Demystifying AI Agent Evaluation: Methods for Different Agent Types

Preface

NOTE

The previous article “Agent Evaluation” covered macro-level evaluation concepts. This article dives into specific evaluation methods and perspectives for different types of Agents.

Analysis references:

• Article: https://www.anthropic.com/engineering/demystifying-evals-for-ai-agents
• 𝜏-Bench: https://arxiv.org/abs/2406.12045
• τ2-Bench: https://arxiv.org/abs/2506.07982
• BrowseComp: https://arxiv.org/abs/2504.12516

1. Evaluating Coding Agents

Coding Agents’ main tasks: writing, testing, and debugging code, browsing codebases like human developers. They rely on clearly specified tasks, which means deterministic scorers are ideal for coding Agents.

First evaluation focus: Does the code run? Do tests pass?

Two programming benchmarks:

1. SWE-bench Verified
2. Terminal-Bench

NOTE

Terminal-Bench tests complete compilation processes (end-to-end), not just fixing single compile errors — e.g., deploying web apps, setting up MySQL from scratch. SWE-bench Verified is more like “unit testing”: give the Agent a real problem, it writes fix code, then run the test suite.

Second evaluation focus: Is the Agent’s work process reasonable and efficient?

Beyond just testing results, evaluating the process of task completion is valuable. Two additional evaluation methods:

1. Heuristic-based code quality evaluation: Check code quality with rules rather than just test results — complexity, duplication, naming conventions, security vulnerabilities, performance issues, readability
2. Model-based behavioral evaluation: Use an LLM to evaluate the Agent’s intermediate process

Example: Task A — Query user info from database

• Agent A: Queries all users, filters in memory
• Agent B: Uses WHERE clause for conditional query

Both complete the task, but Agent B is better and more standards-compliant.

Conclusion: Coding Agent evaluation should assess both execution results and execution process.

Complete evaluation case:

task:
  id: "fix-auth-bypass_1"
  desc: "Fix authentication bypass when password field is empty..."
  graders:
    - type: deterministic_tests
      required:
        - test_empty_pw_rejected.js
        - test_null_pw_rejected.js
    - type: llm_rubric
      rubric: prompts/code_quality.md
    - type: static_analysis
      commands:
        - eslint
        - tsc
    - type: state_check
      expect:
        security_logs:
          event_type: "auth_blocked"
    - type: tool_calls
      required:
        - tool: read_file
          params:
            path: "src/auth/*"
        - tool: edit_file
        - tool: run_tests
  tracked_metrics:
    - type: transcript
      metrics:
        - n_turns
        - n_toolcalls
        - n_total_tokens
    - type: latency
      metrics:
        - time_to_first_token
        - output_tokens_per_sec
        - time_to_last_token

2. Evaluating Conversational Agents

Conversational agents interact with users in domains like support, sales, or coaching. Unlike traditional chatbots, they maintain state, use tools, and take actions mid-conversation.

IMPORTANT

While coding and research agents may also involve multiple user interactions, conversational agents present a unique challenge: the quality of interaction itself is part of what you evaluate.

Effective evaluation relies on verifiable final-state outcomes and rubrics capturing both task completion and interaction quality, often requiring a second LLM to simulate users.

First focus: Verifiable final state — the task the conversational Agent must ultimately complete (refund processing, address changes, quote generation, etc.)

Second focus: Interaction quality is also part of evaluation

Example — Customer refund scenario:

Agent A: “Order number?” → “Refunded.” (Task complete but curt)

Agent B: “I’m sorry for the inconvenience. Which order?” → “I’ve found your order — it qualifies for a refund. Processing now, expect 3-5 business days. Anything else I can help with?” (Task complete, great experience)

Conclusion: Conversational Agent evaluation = final state verification + interaction quality assessment

Multi-dimensional effectiveness criteria:

1. Was the user’s issue resolved? (state check)
2. Completed within 10 conversation turns? (context constraint)
3. Was the tone appropriate? (LLM evaluation)

Notable benchmarks: 𝜏-Bench and τ2-Bench, simulating multi-turn interactions in retail support and airline booking.

3. Evaluating Research Agents

Research Agents collect, synthesize, and analyze information to produce outputs like answers or reports.

Evaluation can’t be as deterministic as coding Agent unit tests. Output quality can only be judged relative to the task, primarily on:

• Comprehensive search and research
• Good and correct sources

Different domains require different standards (market research vs technical investigation).

Research Agent evaluation faces unique challenges: experts may disagree on synthesis completeness, ground truth changes with references, and longer open-ended outputs create more room for errors.

Notable benchmark: BrowseComp — tests whether AI agents can find needles in the open web. Questions are designed to be easy to verify but hard to solve.

General evaluation approach — combine multiple scorer types:

1. Grounding check: Does every claim have source support?
2. Coverage check: Are key insights from sources included and used?
3. Source quality check: Are citations authoritative?

4. Evaluating Computer-Use Agents

Computer-use Agents interact with software through the same interfaces as humans: screenshots, mouse clicks, keyboard input, and scrolling — not through APIs or code execution. They can use any program with a GUI.

Evaluation must check not just UI appearance but whether backend logic executed correctly:

1. WebArena: Tests browser-based tasks using URL and page state checks, plus backend state verification for data-modifying tasks
2. OSWorld: Extends to full OS control — evaluation scripts check file system state, app configs, database content, and UI element properties

A critical design consideration from the official source:

TIP

Browser-use agents must balance token efficiency and latency. DOM-based interaction is fast but token-heavy; screenshot-based interaction is slower but more token-efficient.

Guidance for browser Agent development:

1. If the webpage is text-heavy, reading DOM elements directly is more efficient
2. If the webpage has many DOM elements with scattered text (e.g., e-commerce), screenshots may be more efficient

5. Summary

Regardless of agent type, agent behavior varies between runs, making evaluation results harder to interpret than they initially appear.

Two metrics capture this nuance:

1. pass@k measures the probability of getting at least one correct solution in k attempts.

As k increases, pass@k rises — more “shots on goal” means higher chance of at least 1 success.

50% pass@1 means the model succeeded on half the tasks on its first attempt. In programming, we usually care most about pass@1.

Example: 5 tasks, 3 succeeded at least once within 3 attempts → pass@3 = 60%

2. pass^k measures the probability of all k trials succeeding.

As k increases, pass^k drops — maintaining consistency across more trials is harder.

If your agent has 75% per-trial success rate over 3 trials, all-3-passing probability is (0.75)³ ≈ 42%. Critical for user-facing agents where reliable behavior is expected every time.

• pass@k represents capability — what the Agent can do given enough chances, its boundary
• pass^k represents stability — how reliable the Agent is

NOTE

At k=1, they’re identical. By k=10, they diverge completely: pass@k approaches 100% while pass^k drops to 0%.

Both are useful — which to use depends on product needs: for tools, one success matters (pass@k); for agents, consistency is key (pass^k).