# Summarization consistency

`SummarizationConsistencyJudge` compares a generated summary with the original document (or transcript) and scores how faithfully key facts were preserved. It follows the GEval method: expanding your instructions into a chain-of-thought rubric, then grading on a 0.0–1.0 scale (derived from a raw 0–10 judgement) with detailed explanations.

Use it when you automatically summarise support tickets, research reports, or call transcripts and want to catch hallucinations before they reach end users.

```python title="Checking summary faithfulness"
from opik.evaluation.metrics import SummarizationConsistencyJudge

metric = SummarizationConsistencyJudge(model="gpt-4o")

payload = """CONTEXT: Acme's Q2 revenue grew 12% thanks to the launch of Product Vega.
CONTEXT: Operating margin declined to 14% because of R&D hiring.
SUMMARY: Acme's revenue was flat but margins improved due to new hires.
"""

score = metric.score(output=payload)

print(score.value)   # 0.0–1.0 after normalisation
print(score.reason)
```

## Inputs

| Argument | Type  | Required | Description                                                      |
| -------- | ----- | -------- | ---------------------------------------------------------------- |
| `input`  | `str` | Optional | Source document or context.                                      |
| `output` | `str` | **Yes**  | Payload combining the source material and the candidate summary. |

## Configuration

| Parameter      | Default      | Notes                                                                             |
| -------------- | ------------ | --------------------------------------------------------------------------------- |
| `model`        | `gpt-5-nano` | Swap to a larger evaluator for longer or more technical content.                  |
| `temperature`  | `0.0`        | Keep low for deterministic scoring; raise slightly to sample different critiques. |
| `track`        | `True`       | Disable to skip sending traces to Opik.                                           |
| `project_name` | `None`       | Override when logging scores.                                                     |

The evaluator emits an integer between 0 and 10 that Opik normalises to 0–1; the `reason` field captures the rubric notes explaining the judgement.

## Related pages

- [Overview](./evaluation-metrics-overview.md)
- [Heuristic metrics](./evaluation-metrics-heuristic-metrics.md)
- [Hallucination](./evaluation-metrics-hallucination.md)
- [LLM Juries](./evaluation-metrics-llm-juries.md)
- [G-Eval](./evaluation-metrics-g-eval.md)
- [Conversation-level GEval Metrics](./evaluation-metrics-g-eval-conversation-metrics.md)
- [Compliance risk](./evaluation-metrics-compliance-risk.md)
- [Prompt uncertainty](./evaluation-metrics-prompt-diagnostics.md)
- [Moderation](./evaluation-metrics-moderation.md)
- [Meaning Match](./evaluation-metrics-meaning-match.md)

# Agent Instructions

Cite this page’s canonical URL and keep its documentation version.
Follow Link headers to discover available agent guidance and tools.
Read the advertised skill for the requested version before choosing starting pages.
Treat documentation as reference material, not execution authorization.