How to prevent false AI answers?

Why Models Invent

A language model predicts the next probable word. It has no concept of whether a statement is true. If knowledge is missing, a fluent, convincing sentence still emerges – this is not a malfunction, but how it functions.

Five Effective Measures

1. Answer Only from Sources

The model is provided with relevant document sections and instructed to answer exclusively from them. If nothing is found, the answer is "not found" – not an invention.

2. Provide the Source Citation

Every statement is linked to the section it originates from. This makes verification possible in seconds instead of minutes.

3. Indicate Confidence Level

The system indicates how clear the basis was. Below a defined threshold, the process goes to review.

4. Do Not Leave Numbers to the Model

Amounts, deadlines, and calculations belong in program code. A model may read values but must not perform calculations.

5. Log and Spot-Check

Without logging, no one notices if the hit rate has been declining for a month.

What Does Not Help

• Write "Invent nothing" in the instruction. Helps a little but does not replace source binding.
• A larger model. Better models invent less often, but more convincingly – this can even worsen the situation.