How to Secure Your Own AI Server?

The Fundamental Problem

Model servers are intended to operate behind an application, not for direct access. Many do not come with authentication. If such a server is openly accessible on the internet, anyone can use it – at your expense and with your data.

This is not a theoretical risk; openly accessible model servers are actively sought after.

The Measures

Network. Accessible only from the internal network or via VPN. No direct access from the internet.

Front-end Authentication. A service in front that authenticates users and forwards requests. This allows tracking who requested what.

Limitation. Maximum number of requests per user and time period. Protects against errors in your own processes and against abuse.

Logging. Who, when, which request – while observing data protection regulations for employee data.

Encrypted Connection, even internally.

Additional Considerations

Inputs can be sensitive. If employees enter confidential content, the logs must be adequately protected.

Do not deploy a model without testing. Models from open sources should come from trusted directories.

Co-determination

If requests are logged in a personally identifiable manner, this is a technical setup suitable for behavioral monitoring – where a works council exists, the introduction is subject to co-determination.