How to secure your own AI server?

The Core Problem

Model servers are intended for operation behind an application, not for direct access. Many therefore come with no authentication. If such a server is exposed on the internet, anyone can use it – at your expense and with your data.

This is not a theoretical risk; model servers that are accessible from the internet are actively searched for.

The Measures

Network. Accessible only from the internal network or via VPN. No direct access from the internet.

Pre-authentication. A service in front that authenticates users and forwards requests. This also allows assigning who made which request.

Limitation. Maximum number of requests per user and time period. Protects against errors in your own processes and misuse.

Logging. Who, when, which request – while respecting data protection for employee data.

Encrypted connection, even internally.

Additional Considerations

Inputs can be sensitive. If employees enter confidential content, the logs must be protected accordingly.

Do not use any model without verification. Models from open sources should come from trusted directories.

Co-determination

If requests are logged with personal data, this is a technical setup suitable for behavior monitoring – where a works council exists, implementation is subject to co-determination.