When we build a normal application, we write tests.

For example:

Login Test
API Test
Database Test
Permission Test

But when we add AI to the application, normal tests are not always enough.

The AI can return a different answer for the same type of request.

This is where AI Evals come in.

AI evals are basically tests for AI behaviour.

The idea is simple:

AI Feature
   ↓
Test Cases
   ↓
Run Model
   ↓
Check Result
   ↓
Score

A simple example

Suppose I build an AI customer-support feature.

I can create test cases like:

Question:
How can I reset my password?

Expected:
Give the correct password reset process.

Another test:

Question:
Can I get a refund after 60 days?

Expected:
Do not invent a refund policy.

Now I can run these tests every time I change the model, prompt or system instructions.

Why this is important

With normal code, a small change can easily break a test.

With AI, the problem can be harder to notice.

I may change:

Prompt
Model
Temperature
Tools
Context
RAG data
System instructions

The feature still works.

But the answers may become worse.

Without evals, I may discover the problem only after users start reporting it.

Anthropic describes evals as a way to make behaviour changes visible before they reach production, especially for systems that use tools and multiple steps.

I can create a small eval dataset

For my application, I could keep something like:

tests/ai/

customer-support.json
invoice-assistant.json
sales-assistant.json
product-classification.json

And each test can contain:

Input
Expected Behaviour
Expected Answer
Allowed Variations

For example:

{
  "input": "How can I cancel my subscription?",
  "expected": "Explain the cancellation process",
  "must_not": [
    "invent a refund",
    "invent a support phone number"
  ]
}

The exact format can be different depending on the tool.

The important thing is that the test data stays consistent.

We don't always need exact answers

This is one difference from normal unit testing.

For this:

2 + 2 = 4

the answer must exactly match.

For AI:

How do I reset my password?

there can be many valid answers.

So I can evaluate things like:

Correctness
Relevance
Expected information
Missing information
Wrong information
Safety
Format
Tool usage

This makes AI testing a little more complicated.

LLM-as-a-judge

One common approach is using another model to evaluate the output.

For example:

User Input
    ↓
Model A
    ↓
AI Answer
    ↓
Model B
    ↓
Score

Model B can check:

Is the answer correct?
Is it relevant?
Did it follow the instructions?
Did it invent information?

This is useful, but I would not blindly trust the judge model.

For important features, I would combine model-based evaluation with normal rules.

For example:

Response contains customer ID
Response uses correct currency
Response has required fields
No forbidden text
LLM quality score > threshold

AI evals are useful for RAG too

Suppose I have a company knowledge base.

The flow is:

User Question
   ↓
Search Documents
   ↓
Relevant Context
   ↓
AI Model
   ↓
Answer

There are actually multiple things to test.

First:

Did retrieval find the correct documents?

Then:

Did the AI use those documents?

Then:

Did the final answer contain unsupported information?

So an AI application can have several eval layers instead of one final test.

This is also useful when changing models

Suppose today I use one model.

Tomorrow I want to test another one.

Instead of manually checking 100 questions, I can run:

Test Dataset
      ↓
Model A
      ↓
Score

Test Dataset
      ↓
Model B
      ↓
Score

Now I have actual data to compare the behaviour.

This is much better than testing a model with only two or three questions.

OpenAI has also highlighted the importance of good coding evaluations and reported issues with widely used benchmark tasks, showing that even the evaluation itself needs to be tested carefully.

Evals should run when the prompt changes

This is probably where I would use them most.

For example:

Before:
Prompt v1

After:
Prompt v2

I don't want to check everything manually.

I can run:

AI Eval Suite
   ↓
100 Test Cases
   ↓
Pass / Fail / Score

Then I can see whether the new prompt actually improved the application.

This can become part of CI/CD

For a serious application, I can even put AI evals inside CI.

Something like:

Git Push
   ↓
Tests
   ↓
AI Evals
   ↓
Build
   ↓
Deploy

For example, deployment can fail when an important AI feature drops below a specific quality threshold.

This makes AI behaviour part of the software delivery process instead of something we manually check.

A simple Laravel example

Imagine I have a Laravel API:

POST /api/ai/customer-summary

I can keep test inputs in a file or database.

Then the test runner sends them to my AI service:

Laravel
   ↓
AI Service
   ↓
Model
   ↓
Response
   ↓
Evaluator
   ↓
Score

I can store:

test_case
model
prompt_version
score
passed
created_at

Now I can see when AI quality changes over time.

I would track prompt versions

This is something that becomes important quickly.

For example:

Customer Assistant
Prompt v1
Prompt v2
Prompt v3

Then I can compare:

v1 → 81%
v2 → 88%
v3 → 84%

The exact score depends on my evaluation method, but the history is useful.

Now prompt changes become something I can measure instead of guessing.

AI evals are becoming a bigger engineering area

This is not just a small testing idea anymore.

On September 18, 2026, Anthropic announced a partnership with Accenture to build independent evaluation and red-team capabilities for frontier AI models, with each company planning to invest at least $1 billion over five years.

For normal developers, we obviously don't need that level of infrastructure.

But the direction is interesting.

As AI becomes part of production software, testing AI behaviour becomes an engineering problem of its own.

My basic approach

For a new AI feature, I would start small.

20-50 test cases
+
Expected behaviour
+
Basic rules
+
LLM evaluation

Then run the same tests after:

Prompt changes
Model changes
RAG changes
Tool changes
Code changes

This is already enough to catch many problems.

My takeaway

For normal software:

Code → Tests → Production

For AI software, I think it should become:

Code
  ↓
AI Evals
  ↓
Tests
  ↓
Production

AI can generate impressive answers even when the implementation is not reliable.

So instead of only asking:

"Does my AI work?"

I want to ask:

"Does my AI continue to give the expected result after every change?"

That is what makes AI evals interesting to me.

It is basically bringing the idea of a test suite into AI development.