Getting Started
MCP Server
Changelog
Observability
OverviewGetting startedConceptsLearn about the core concepts of Opik's tracing system, including traces, spans, and threads, and how they work together to provide comprehensive observability for your LLM applications.Debugging agents with OllieDiagnostics
Advanced
Dashboards
Development
Prompt library
Optimization runs
Opik Agent OptimizerLearn about Opik's automated LLM prompt and agent optimization SDK. Discover MetaPrompt, Few-shot Bayesian, and Evolutionary optimization algorithms for enhanced performance.Optimization StudioRun prompt optimizations from the Opik UI with datasets, metrics, and visual progress tracking.QuickstartInstall the Agent Optimizer SDK, run your first optimization, and inspect the results in under 10 minutes.Quickstart notebookLearn how to use Opik Agent Optimizer with the HotPotQA dataset through an interactive notebook, covering setup, configuration, and optimization techniques.FAQFind answers to common questions about Opik Agent Optimizer, including optimizer selection, configuration, usage, and best practices.ChangelogRelease notes focused on Agent Optimizer features.Known IssuesKnown runtime issues when running Opik Optimizer with current dependencies.
Optimization
ConceptsLearn about the core concepts of Opik Agent Optimizer, including key terms, evaluation processes, optimization workflows, and best practices for effective LLM optimization.Configure LLM ProvidersLearn how to configure different LLM providers like OpenAI, Anthropic, Gemini, Azure, and Ollama for use with the Opik Agent Optimizer.Define datasetsDesign, version, and validate datasets used for optimization runs.Define metricsCreate reliable metrics and composite objectives for Agent Optimizer runs.Optimize promptsPick the right optimizer, run experiments, and ship better prompts.Optimize tools (MCP)Improve MCP tool and parameter descriptions without changing prompt text.Optimize agentsConnect the Agent Optimizer to Agent Frameworks.Optimize multimodalRun optimizations for prompts that combine text, images, and other modalities.Dashboard resultsAnalyze optimization trials, failure modes, and improvements inside Opik.
Optimization Algorithms
OverviewCompare Agent Optimizer algorithms and pick the right one for your workload.BenchmarksCompare algorithm performance across common datasets and learn how to run the benchmark suite locally.MetaPromptLearn how to use the MetaPrompt Optimizer to refine and improve your LLM prompts through systematic analysis and iterative refinement.HRPOLearn how to use HRPO (Hierarchical Reflective Prompt Optimizer) to improve prompts through systematic root cause analysis of failure modes and targeted refinement.Few-Shot BayesianLearn how to use the Few-Shot Bayesian Optimizer to find optimal few-shot examples for your chat-based prompts using Bayesian optimization techniques.EvolutionaryLearn how to use the Evolutionary Optimizer to discover optimal prompts through genetic algorithms, with support for multi-objective optimization and LLM-driven genetic operations.GEPAUse the external GEPA package through Opik's `GepaOptimizer` to optimize a single system prompt for single-turn tasks with a reflection model.ParameterLearn how to use the Parameter Optimizer to find optimal LLM call parameters (temperature, top_p, etc.) using Bayesian optimization without changing your prompt.Tool OptimizationLearn how to optimize prompts that leverage MCP (Model Context Protocol) tools and function calling capabilities across supported optimizers.
Cookbooks & Tutorials
Synthetic data optimizerLearn how to generate synthetic Q&A data from Opik traces and optimize prompts using the MetaPromptOptimizer through an interactive notebook.ARC-AGI tutorialEnd-to-end ARC-AGI optimization walkthrough with Opik Agent Optimizer.Multimodal agent tutorialMultimodal agent optimization tutorial inspired by a self-driving car example.
Advanced Topics
Extending optimizersLearn how to extend Opik's optimization framework with custom algorithms and contribute to the project's development.Custom metricsBuild specialized metrics, integrate external models, and reuse them across optimizations.Custom optimizer promptsLearn how to override and customize the internal prompt templates used by Opik optimizers to add domain constraints, safety requirements, or custom formatting.Sampling controlsUse n_samples, n_samples_minibatch, and the n parameter to trade cost for stability and exploration.Multiple completions (n)Learn how to use the n parameter to generate multiple candidate outputs per evaluation, reducing variance and improving optimization results through best-of-k selection.Chaining optimizersRun multiple optimizers in sequence to balance exploration and fine-tuning.API ReferenceTechnical SDK reference guide
Evaluation
Advanced
Building Test SuitesDatasets & ExperimentsStep by step guide on how to evaluate your LLM applicationResume an interrupted evaluationContinue an evaluation from where it stoppedManage datasetsGuides you through the process of creating and managing datasetsEvaluate agent trajectoriesStep-by-step guide to evaluate agent trajectoriesEvaluate multi-turn agentsStep-by-step guide to evaluate multi-turn agentsAnnotation QueuesEnable subject matter experts to review and annotate agent outputs with easy queues, invitations, and a clean annotation UIManually logging experimentsStep by step guide to logging evaluation results using Python SDK and REST APIExporting experiment resultsExport a full experiment result set to CSV with the Python SDK
Metrics
OverviewDescribes all the built-in evaluation metrics provided by OpikHeuristic metricsDescribes all the built-in heuristic metrics provided by OpikHallucinationDescribes the Hallucination metricLLM JuriesCombine multiple judges into an ensemble with LLMJuriesJudgeG-EvalDescribes Opik's built-in G-Eval metric which is a task agnostic LLM as a Judge metricConversation-level GEvalHow to run GEval-based judges on full conversationsCompliance riskFlag non-compliant or high-risk assistant replies with ComplianceRiskJudgePrompt uncertaintyEstimate prompt ambiguity with PromptUncertaintyJudgeModerationDescribes the Moderation metricMeaning MatchDescribes the Meaning Match metricUsefulnessDescribes the Usefulness metricSummarization consistencyEnsure autogenerated summaries stay faithful to the source contentSummarization coherenceRate how readable and well-structured a summary is with SummarizationCoherenceJudgeDialogue helpfulnessMeasure how helpful an assistant reply is within a dialogueAnswer relevanceDescribes the Answer Relevance metricContext precisionDescribes the Context Precision metricContext recallDescribes the Context Recall metricTrajectory accuracyScore whether an agent followed the expected action pathAgent task completionVerify whether an agent fulfilled its assigned objectiveAgent tool correctnessEvaluate whether an agent invoked and interpreted tools correctlyConversational metricsDescribes metrics related to scoring the conversational threadsCustom modelDescribes how to use a custom model for Opik's built-in LLM as a Judge metricsAdvanced configurationFine-tune Opik metrics with async scoring, evaluator temperatures, and logprob handlingCustom metricDescribes how to create your own metric to use with Opik's evaluation platformCustom conversation metricLearn how to create custom metrics for evaluating multi-turn conversationsStructured Output ComplianceDescribes the StructuredOutputCompliance metricTask span metricsLearn how to create task span metrics for evaluating the detailed execution information of your LLM tasks
Tutorials
Guardrails
OverviewProtect your LLM application in real time by blocking unsafe inputs and outputsPoliciesDefine your checks once in Opik and reference them by name from your applicationGuardsPrompt injection modelOpik Guard, our purpose-built detection model, evaluated on attacks it has never seenCustom modelsFine-tune a language model for a check the built-in guards do not coverGuardrails serverInstall, configure, and run the backend that powers the model-based guards
Production
Online evaluation
Gateway
Alerts
Administration
OverviewMulti-user collaboration with enterprise-grade access controlRoles and PermissionsUnderstanding and configuring workspace roles and the permission modelData AnonymizationMask sensitive values in trace data for users who do not have permission to see them
Workspace Settings
OverviewUnderstanding workspace-level configuration options in OpikFeedback DefinitionsConfigure custom feedback types for evaluating LLM outputsAI ProvidersConfigure connections to Large Language Model providersWorkspace PreferencesConfigure workspace-wide behavior settingsWorkspace MembersInvite users, manage members, and control workspace-level access
Admin Dashboard
OverviewHow to access and use the organization administration dashboardWorkspacesCreate and manage workspaces within your organizationUsersManage users and organization rolesOrganization SettingsManage organization-wide configurationService AccountsCreating and managing service accounts for automated systems and integrations
Authentication
OverviewUnderstanding authentication options and choosing the right approachSAML SSOSetting up SAML-based single sign-on with your identity providerOIDC SSOSetting up OpenID Connect authentication with your identity providerJWT AuthenticationSetting up JWT token authentication for programmatic and service access
Contributing
Specific Contribution Guides
Developer Programs
Integrations
AI Coding Assistants
OverviewHow to get Claude Code, Cursor, GitHub Copilot and OpenAI Codex sessions into Opik, and when Cost Intelligence or the MCP server is the better fit.MCP ServerClaude CodeLog Claude Code sessions to Opik with the Opik plugin, or export Claude Code's native OpenTelemetry spans to Opik, and know which one to pick.CursorStart here to integrate Opik into your Cursor-based genai application for end-to-end LLM observability, unit testing, and optimization.GitHub CopilotSend GitHub Copilot Chat (VS Code) and Copilot CLI sessions to Opik over OpenTelemetry, with clear content-capture and endpoint expectations.OpenAI CodexConfigure OpenAI Codex OpenTelemetry export to send trace telemetry to Opik.
TypeScript
Frameworks
BeeAIStart here to integrate Opik into your BeeAI-based genai application for end-to-end LLM observability, unit testing, and optimization.LangChainStart here to integrate Opik into your LangChain-based genai application for end-to-end LLM observability, unit testing, and optimization.MastraStart here to integrate Opik into your Mastra-based genai application for end-to-end LLM observability, unit testing, and optimization.Vercel AI SDKStart here to integrate Opik into your Vercel AI SDK-based genai application for end-to-end LLM observability, unit testing, and optimization.
Model Providers
Cloudflare Workers AIStart here to integrate Opik into your Cloudflare Workers AI-based genai application for end-to-end LLM observability, unit testing, and optimization.GeminiStart here to integrate Opik into your Google Gemini-based genai application for end-to-end LLM observability, unit testing, and optimization.OpenAIStart here to integrate Opik into your OpenAI-based genai application for end-to-end LLM observability, unit testing, and optimization.
Python
Frameworks
AG2Start here to integrate Opik into your AG2-based genai application for end-to-end LLM observability, unit testing, and optimization.Agent SpecStart here to integrate Opik into your Agent Spec-based application for end-to-end observability, unit testing, and optimization.AgnoStart here to integrate Opik into your Agno-based genai application for end-to-end LLM observability, unit testing, and optimization.BeeAIStart here to integrate Opik into your BeeAI-based genai application for end-to-end LLM observability, unit testing, and optimization.Claude Agent SDKConfigure Claude Code telemetry and OTLP export settings with clear signal and compatibility expectations for Opik.AutogenStart here to integrate Opik into your AutoGen-based genai application for end-to-end LLM observability, unit testing, and optimization.CrewAIStart here to integrate Opik into your CrewAI-based genai application for end-to-end LLM observability, unit testing, and optimization.DSPyStart here to integrate Opik into your DSPy-based genai application for end-to-end LLM observability, unit testing, and optimization.Google ADKStart here to integrate Opik into your Google Agent Development Kit-based genai application for end-to-end LLM observability, unit testing, and optimization.HaystackStart here to integrate Opik into your Haystack-based genai application for end-to-end LLM observability, unit testing, and optimization.HarborStart here to integrate Opik into your Harbor benchmark evaluation runs for end-to-end agent observability and analysis.InstructorStart here to integrate Opik into your Instructor-based genai application for structured output tracking, schema validation monitoring, and LLM call observability.LangChainStart here to integrate Opik into your LangChain-based genai application for end-to-end LLM observability, unit testing, and optimization.LangGraphStart here to integrate Opik into your LangGraph-based genai application for end-to-end LLM observability, unit testing, and optimization.LangServeConfigure LangServe host-level OpenTelemetry export for Opik with clear applicability and minimal setup guidance.LiveKit AgentsStart here to integrate Opik into your LiveKit-based genai application for end-to-end LLM observability, unit testing, and optimization.Llama IndexStart here to integrate Opik into your LlamaIndex-based genai application for end-to-end LLM observability, unit testing, and optimization.Microsoft Agent FrameworkStart here to integrate Opik into your Microsoft Agent Framework-based genai application for end-to-end LLM observability, unit testing, and optimization.OpenAI AgentsStart here to integrate Opik into your OpenAI Agents-based genai application for end-to-end LLM observability, unit testing, and optimization.PipecatStart here to integrate Opik into your Pipecat-based real-time voice agent application for end-to-end LLM observability, unit testing, and optimization.Pydantic AIStart here to integrate Opik into your Pydantic AI-based genai application for end-to-end LLM observability, unit testing, and optimization.Semantic KernelStart here to integrate Opik into your Semantic Kernel-based genai application for end-to-end LLM observability, unit testing, and optimization.SmolagentsStart here to integrate Opik into your Smolagents-based genai application for end-to-end LLM observability, unit testing, and optimization.Strands AgentsStart here to integrate Opik into your Strands Agents-based genai application for end-to-end LLM observability, unit testing, and optimization.TemporalConfigure Temporal Python OpenTelemetry tracing and export to Opik.Twilio Agent ConnectStart here to integrate Opik into your Twilio Agent Connect voice agent for end-to-end LLM observability, unit testing, and optimization.VoltAgentStart here to integrate Opik into your VoltAgent-based genai application for end-to-end LLM observability, unit testing, and optimization.
Model Providers
AnthropicStart here to integrate Opik into your Anthropic-based genai application for end-to-end LLM observability, unit testing, and optimization.BedrockStart here to integrate Opik into your AWS Bedrock-based genai application for end-to-end LLM observability, unit testing, and optimization.BytePlusStart here to integrate Opik into your BytePlus-based genai application for end-to-end LLM observability, unit testing, and optimization.CohereStart here to integrate Opik into your Cohere-based genai application for end-to-end LLM observability, unit testing, and optimization.DeepSeekStart here to integrate Opik into your DeepSeek-based genai application for end-to-end LLM observability, unit testing, and optimization.Fireworks AIStart here to integrate Opik into your Fireworks AI-based genai application for end-to-end LLM observability, unit testing, and optimization.GeminiStart here to integrate Opik into your Google Gemini-based genai application for end-to-end LLM observability, unit testing, and optimization.GroqStart here to integrate Opik into your Groq-based genai application for end-to-end LLM observability, unit testing, and optimization.TypeSafe AIStart here to integrate Opik into your TypeSafe AI-based application for end-to-end observability of System One (Jev) decisions.MiniMaxTrack MiniMax model calls with Opik through the OpenAI-compatible or Anthropic-compatible API.Mistral AIStart here to integrate Opik into your Mistral AI-based genai application for end-to-end LLM observability, unit testing, and optimization.Novita AIStart here to integrate Opik into your Novita AI-based genai application for end-to-end LLM observability, unit testing, and optimization.OllamaStart here to integrate Opik into your Ollama-based genai application for end-to-end LLM observability, unit testing, and optimization.OpenAIStart here to integrate Opik into your OpenAI-based genai application for end-to-end LLM observability, unit testing, and optimization.QianfanStart here to integrate Opik into your Qianfan-based genai application for end-to-end LLM observability, unit testing, and optimization.PredibaseStart here to integrate Opik into your Predibase-based genai application for end-to-end LLM observability, unit testing, and optimization.Together AIStart here to integrate Opik into your Together AI-based genai application for end-to-end LLM observability, unit testing, and optimization.WatsonXStart here to integrate Opik into your IBM watsonx-based genai application for end-to-end LLM observability, unit testing, and optimization.xAI GrokStart here to integrate Opik into your xAI Grok-based genai application for end-to-end LLM observability, unit testing, and optimization.
Dataset Management
Evaluation
Gateways
Opik LLM GatewayStart here to integrate Opik LLM Gateway into your genai application for centralized LLM provider access using a consistent API format.Kong AI GatewayStart here to integrate Kong AI Gateway with Opik for production-ready LLM gateway capabilities with enterprise features.AISuiteStart here to integrate Opik into your AISuite-based genai application for end-to-end LLM observability, unit testing, and optimization.HeliconeLearn how to integrate Helicone with Opik to log and monitor your LLM traffic using the standard Helicone integration setup.LiteLLMStart here to integrate Opik into your LiteLLM-based genai application for end-to-end LLM observability, unit testing, and optimization.OpenRouterStart here to integrate Opik into your OpenRouter-based genai application for end-to-end LLM observability, unit testing, and optimization.PortkeyStart here to integrate Portkey with Opik for enterprise-grade LLM gateway capabilities with advanced routing and fallback features.TrueFoundryStart here to integrate the TrueFoundry AI Gateway with Opik and trace every LLM and MCP call that passes through the gateway.Vercel AI GatewayStart here to integrate Vercel AI Gateway with Opik for unified access to multiple AI providers with edge-optimized performance.
Other
Java
.NET
Frameworks
Ruby
No-Code
DifyStart here to integrate Opik into your Dify-based genai application for end-to-end LLM observability, unit testing, and optimization.FlowiseStart here to integrate Opik into your Flowise-based genai application for end-to-end LLM observability, unit testing, and optimization.HermesInstall and configure the opik-hermes plugin to export LLM, tool, and agent traces to Opik.LangflowStart here to integrate Opik into your Langflow applications for end-to-end LLM observability.n8nStart here to integrate Opik into your n8n-based workflow automation for end-to-end LLM observability.OpenClawInstall and configure the OpenClaw community plugin to export LLM, tool, and agent traces to Opik.OpenWebUIStart here to integrate Opik into your OpenWebUI-based AI application for end-to-end LLM observability and monitoring.Prompt FlowConfigure Prompt Flow tracing with OpenTelemetry endpoint settings compatible with Opik deployment modes.
Multi-Language
Cost Intelligence
Installation
Self-hosting Opik
OverviewHigh-level overview on how to self-host OpikMigrating to Opik 2.0Enable the background jobs that move your existing 1.x data into projectsLocal deploymentDescribes how to run Opik locally using Docker ComposeKubernetes deploymentDescribes how to run Opik on a Kubernetes clusterHelm chart migrationGuide for migrating from Bitnami charts and imagesClickHouse cold-tier migrationEnabling tiered (cold) ClickHouse storage on a multi-replica clusterPlatform ArchitectureHigh-level overview on Opik's Platform ArchitectureScaling OpikComprehensive guide for scaling Opik in production environmentsAdvanced clickhouse backupComprehensive guide for ClickHouse backup options in Opik Kubernetes deploymentsTroubleshootingTroubleshooting guide for common issues when running self-hosted Opik deployments.ChangelogBreaking and critical changes for self-hosted deployments
Configuration
Anonymous usage statisticsDescribes the usage statistics that are collected by OpikLarge CSV uploadsConfigure Opik to support large CSV file uploads for datasetsDataset versioning migrationUnderstanding the automated dataset versioning migration processLLM model registryUUIDv7 ingestion validationBound the timestamp a client may embed in a trace or span idMCP OAuthLet coding agents sign in to your deployment through the browser
SDK & API reference
Python SDK
Troubleshooting
Typescript SDK
Opik TSA guided CLI tool for quickly integrating Opik SDK into your Node.js/TypeScript projects with automatic configuration and setup.PromptsOpik Query Language (OQL)