Agent tool correctness
AgentToolCorrectnessJudge checks if an agent called the right tools with valid arguments and interpreted the outputs accurately. It’s invaluable for diagnosing production agents that orchestrate APIs, databases, or internal services.
from opik.evaluation.metrics import AgentToolCorrectnessJudge
payload = """TOOL weather_api(city='Paris') -> 12°C and raining.
AGENT: Responded "Sunny and warm".
"""
metric = AgentToolCorrectnessJudge()
score = metric.score(output=payload)
print(score.value) # 0.0–1.0 after normalisation
print(score.reason)Inputs
Section titled “Inputs”| Argument | Type | Required | Description |
|---|---|---|---|
output |
str |
Yes | Payload describing the task, tool calls, and observed behaviour. |
Configuration
Section titled “Configuration”| Parameter | Default | Notes |
|---|---|---|
model |
gpt-5-nano |
Upgrade to a larger evaluator if analysing lengthy traces. |
temperature |
0.0 |
Keep low for repeatable scoring. |
track |
True |
Controls Opik tracking. |
project_name |
None |
Override logging destination. |
The judge emits an integer between 0 and 10 (scaled to 0–1 by Opik); read score.reason to pinpoint incorrect calls, missing validations, or misinterpreted outputs.