Skip to main content
Opik Documentation
current

Search documentation

Type to search this documentation.

Guides

Every guide in Opik Documentation.

Getting Started

MCP Server

Changelog

Observability

Advanced

Dashboards

Development

Prompt library

Optimization runs

Optimization

Optimization Algorithms

Cookbooks & Tutorials

Advanced Topics

Evaluation

Advanced

Metrics

OverviewDescribes all the built-in evaluation metrics provided by OpikHeuristic metricsDescribes all the built-in heuristic metrics provided by OpikHallucinationDescribes the Hallucination metricLLM JuriesCombine multiple judges into an ensemble with LLMJuriesJudgeG-EvalDescribes Opik's built-in G-Eval metric which is a task agnostic LLM as a Judge metricConversation-level GEvalHow to run GEval-based judges on full conversationsCompliance riskFlag non-compliant or high-risk assistant replies with ComplianceRiskJudgePrompt uncertaintyEstimate prompt ambiguity with PromptUncertaintyJudgeModerationDescribes the Moderation metricMeaning MatchDescribes the Meaning Match metricUsefulnessDescribes the Usefulness metricSummarization consistencyEnsure autogenerated summaries stay faithful to the source contentSummarization coherenceRate how readable and well-structured a summary is with SummarizationCoherenceJudgeDialogue helpfulnessMeasure how helpful an assistant reply is within a dialogueAnswer relevanceDescribes the Answer Relevance metricContext precisionDescribes the Context Precision metricContext recallDescribes the Context Recall metricTrajectory accuracyScore whether an agent followed the expected action pathAgent task completionVerify whether an agent fulfilled its assigned objectiveAgent tool correctnessEvaluate whether an agent invoked and interpreted tools correctlyConversational metricsDescribes metrics related to scoring the conversational threadsCustom modelDescribes how to use a custom model for Opik's built-in LLM as a Judge metricsAdvanced configurationFine-tune Opik metrics with async scoring, evaluator temperatures, and logprob handlingCustom metricDescribes how to create your own metric to use with Opik's evaluation platformCustom conversation metricLearn how to create custom metrics for evaluating multi-turn conversationsStructured Output ComplianceDescribes the StructuredOutputCompliance metricTask span metricsLearn how to create task span metrics for evaluating the detailed execution information of your LLM tasks

Tutorials

Guardrails

Production

Online evaluation

Gateway

Alerts

Administration

Workspace Settings

Admin Dashboard

Authentication

Contributing

Specific Contribution Guides

Developer Programs

Integrations

AI Coding Assistants

TypeScript

Frameworks

Model Providers

Python

Frameworks

AG2Start here to integrate Opik into your AG2-based genai application for end-to-end LLM observability, unit testing, and optimization.Agent SpecStart here to integrate Opik into your Agent Spec-based application for end-to-end observability, unit testing, and optimization.AgnoStart here to integrate Opik into your Agno-based genai application for end-to-end LLM observability, unit testing, and optimization.BeeAIStart here to integrate Opik into your BeeAI-based genai application for end-to-end LLM observability, unit testing, and optimization.Claude Agent SDKConfigure Claude Code telemetry and OTLP export settings with clear signal and compatibility expectations for Opik.AutogenStart here to integrate Opik into your AutoGen-based genai application for end-to-end LLM observability, unit testing, and optimization.CrewAIStart here to integrate Opik into your CrewAI-based genai application for end-to-end LLM observability, unit testing, and optimization.DSPyStart here to integrate Opik into your DSPy-based genai application for end-to-end LLM observability, unit testing, and optimization.Google ADKStart here to integrate Opik into your Google Agent Development Kit-based genai application for end-to-end LLM observability, unit testing, and optimization.HaystackStart here to integrate Opik into your Haystack-based genai application for end-to-end LLM observability, unit testing, and optimization.HarborStart here to integrate Opik into your Harbor benchmark evaluation runs for end-to-end agent observability and analysis.InstructorStart here to integrate Opik into your Instructor-based genai application for structured output tracking, schema validation monitoring, and LLM call observability.LangChainStart here to integrate Opik into your LangChain-based genai application for end-to-end LLM observability, unit testing, and optimization.LangGraphStart here to integrate Opik into your LangGraph-based genai application for end-to-end LLM observability, unit testing, and optimization.LangServeConfigure LangServe host-level OpenTelemetry export for Opik with clear applicability and minimal setup guidance.LiveKit AgentsStart here to integrate Opik into your LiveKit-based genai application for end-to-end LLM observability, unit testing, and optimization.Llama IndexStart here to integrate Opik into your LlamaIndex-based genai application for end-to-end LLM observability, unit testing, and optimization.Microsoft Agent FrameworkStart here to integrate Opik into your Microsoft Agent Framework-based genai application for end-to-end LLM observability, unit testing, and optimization.OpenAI AgentsStart here to integrate Opik into your OpenAI Agents-based genai application for end-to-end LLM observability, unit testing, and optimization.PipecatStart here to integrate Opik into your Pipecat-based real-time voice agent application for end-to-end LLM observability, unit testing, and optimization.Pydantic AIStart here to integrate Opik into your Pydantic AI-based genai application for end-to-end LLM observability, unit testing, and optimization.Semantic KernelStart here to integrate Opik into your Semantic Kernel-based genai application for end-to-end LLM observability, unit testing, and optimization.SmolagentsStart here to integrate Opik into your Smolagents-based genai application for end-to-end LLM observability, unit testing, and optimization.Strands AgentsStart here to integrate Opik into your Strands Agents-based genai application for end-to-end LLM observability, unit testing, and optimization.TemporalConfigure Temporal Python OpenTelemetry tracing and export to Opik.Twilio Agent ConnectStart here to integrate Opik into your Twilio Agent Connect voice agent for end-to-end LLM observability, unit testing, and optimization.VoltAgentStart here to integrate Opik into your VoltAgent-based genai application for end-to-end LLM observability, unit testing, and optimization.

Model Providers

AnthropicStart here to integrate Opik into your Anthropic-based genai application for end-to-end LLM observability, unit testing, and optimization.BedrockStart here to integrate Opik into your AWS Bedrock-based genai application for end-to-end LLM observability, unit testing, and optimization.BytePlusStart here to integrate Opik into your BytePlus-based genai application for end-to-end LLM observability, unit testing, and optimization.CohereStart here to integrate Opik into your Cohere-based genai application for end-to-end LLM observability, unit testing, and optimization.DeepSeekStart here to integrate Opik into your DeepSeek-based genai application for end-to-end LLM observability, unit testing, and optimization.Fireworks AIStart here to integrate Opik into your Fireworks AI-based genai application for end-to-end LLM observability, unit testing, and optimization.GeminiStart here to integrate Opik into your Google Gemini-based genai application for end-to-end LLM observability, unit testing, and optimization.GroqStart here to integrate Opik into your Groq-based genai application for end-to-end LLM observability, unit testing, and optimization.TypeSafe AIStart here to integrate Opik into your TypeSafe AI-based application for end-to-end observability of System One (Jev) decisions.MiniMaxTrack MiniMax model calls with Opik through the OpenAI-compatible or Anthropic-compatible API.Mistral AIStart here to integrate Opik into your Mistral AI-based genai application for end-to-end LLM observability, unit testing, and optimization.Novita AIStart here to integrate Opik into your Novita AI-based genai application for end-to-end LLM observability, unit testing, and optimization.OllamaStart here to integrate Opik into your Ollama-based genai application for end-to-end LLM observability, unit testing, and optimization.OpenAIStart here to integrate Opik into your OpenAI-based genai application for end-to-end LLM observability, unit testing, and optimization.QianfanStart here to integrate Opik into your Qianfan-based genai application for end-to-end LLM observability, unit testing, and optimization.PredibaseStart here to integrate Opik into your Predibase-based genai application for end-to-end LLM observability, unit testing, and optimization.Together AIStart here to integrate Opik into your Together AI-based genai application for end-to-end LLM observability, unit testing, and optimization.WatsonXStart here to integrate Opik into your IBM watsonx-based genai application for end-to-end LLM observability, unit testing, and optimization.xAI GrokStart here to integrate Opik into your xAI Grok-based genai application for end-to-end LLM observability, unit testing, and optimization.

Dataset Management

Evaluation

Gateways

Other

Java

.NET

Frameworks

Ruby

No-Code

Multi-Language

Cost Intelligence

Installation

Self-hosting Opik

Configuration

SDK & API reference

Python SDK

Troubleshooting

Typescript SDK

Evaluation

Rest API

Documentation menu