Skip to main content
Opik Documentation
current

Search documentation

Type to search this documentation.

On this pageOverview

Dialogue helpfulness

Measure how helpful an assistant reply is within a dialogue

DialogueHelpfulnessJudge inspects the latest assistant reply in the context of preceding turns. It rewards responses that acknowledge the user’s request, use the available context, and offer actionable guidance.

Scoring a support reply
from opik.evaluation.metrics import DialogueHelpfulnessJudge

turns = """USER: My VPN disconnects every 5 minutes.\nASSISTANT: Try reinstalling the client.\nUSER: I already did.\n"""

metric = DialogueHelpfulnessJudge()
score = metric.score(
    input=turns,
    output="Can you send logs? I'll escalate to network engineering.",
)

print(score.value)
print(score.reason)
Argument Type Required Description
input str Optional Conversation history (alternating USER / ASSISTANT blocks).
conversation list[dict] Optional Structured turns ({"role": "user", "content": "..."} {"role": "assistant", ...}).
output str Yes Latest assistant reply to score.
Parameter Default Notes
model gpt-5-nano Switch to a larger evaluator for complex enterprise workflows.
temperature 0.0 Use low temperature for reproducible benchmarks.
track True Record the evaluation in Opik.
project_name None Set when routing results to a different project.

Integrate this judge into regression suites to catch regressions after prompt changes or upgrades to your assistant model.

Suggest an edit

Propose a replacement for this page. The site team reviews it before applying any changes.

Export
Documentation menu