Skip to main content
Opik Documentation

Search documentation

Type to search this documentation.

On this pageOverview

Models

The TypeScript SDK provides flexible model configuration through direct integration with the Vercel AI SDK. You can use models from multiple providers with a simple, unified interface.

Overview

Section titled “Overview”

The TypeScript SDK supports three ways to configure models for evaluation and prompt generation:

  1. Model ID strings - Simple string identifiers (e.g., "gpt-5-nano", "claude-3-5-sonnet-latest")
  2. LanguageModel instances - Pre-configured Vercel AI SDK models with custom settings
  3. OpikBaseModel implementations - Custom model integrations for unsupported providers

Quick Start

Section titled “Quick Start”

Using Model ID Strings

Section titled “Using Model ID Strings”

The simplest approach is to pass a model ID string directly:

TypeScript
import { evaluatePrompt } from "opik";
import { Hallucination } from "opik";

// OpenAI model
await evaluatePrompt({
  dataset,
  messages: [{ role: "user", content: "{{input}}" }],
  model: "gpt-5-nano",
  projectName: "my-project",
});

// Anthropic model
await evaluatePrompt({
  dataset,
  messages: [{ role: "user", content: "{{input}}" }],
  model: "claude-3-5-sonnet-latest",
  projectName: "my-project",
});

// Google Gemini model
await evaluatePrompt({
  dataset,
  messages: [{ role: "user", content: "{{input}}" }],
  model: "gemini-2.0-flash",
  projectName: "my-project",
});

// Use in metrics
const metric = new Hallucination({ model: "gpt-5-nano" });

Using LanguageModel Instances

Section titled “Using LanguageModel Instances”

For advanced scenarios, use LanguageModel instances from Vercel AI SDK:

TypeScript
import { openai } from "@ai-sdk/openai";
import { evaluatePrompt } from "opik";

// Create a LanguageModel instance
const customModel = openai("gpt-5-nano");

await evaluatePrompt({
  dataset,
  messages: [{ role: "user", content: "{{input}}" }],
  model: customModel,
  projectName: "my-project",
});

Generation Parameters

Section titled “Generation Parameters”

Parameters for Metrics

Section titled “Parameters for Metrics”

All LLM Judge metrics support these generation parameters directly in the constructor:

TypeScript
import { Hallucination } from "opik";

const metric = new Hallucination({
  model: "gpt-5-nano",
  temperature: 0.3, // Control randomness (0.0-2.0)
  seed: 42, // For reproducible outputs
  maxTokens: 1000, // Maximum response length
});

// Use in evaluation
const score = await metric.score({
  input: "What is the capital of France?",
  output: "The capital of France is Paris.",
  context: ["France is a country in Western Europe."],
});

For advanced generation parameters, use modelSettings:

TypeScript
import { Hallucination } from "opik";

const metric = new Hallucination({
  model: "gpt-5-nano",
  temperature: 0.5,
  modelSettings: {
    topP: 0.9, // Nucleus sampling
    topK: 50, // Top-K sampling
    presencePenalty: 0.1, // Reduce repetition
    frequencyPenalty: 0.2, // Reduce phrase repetition
    stopSequences: ["END"], // Custom stop sequences
  },
});

Parameters for evaluatePrompt

Section titled “Parameters for evaluatePrompt”

The evaluatePrompt function supports only temperature and seed:

TypeScript
import { evaluatePrompt } from "opik";
import { Hallucination, AnswerRelevance } from "opik";

await evaluatePrompt({
  dataset,
  messages: [{ role: "user", content: "{{input}}" }],
  model: "gpt-5-nano",
  temperature: 0.7,
  seed: 42,
  scoringMetrics: [
    new Hallucination({
      model: "gpt-5-nano",
      temperature: 0.3, // Full parameter support in metrics
      seed: 12345,
      maxTokens: 1000,
    }),
  ],
  projectName: "my-project",
});

Note: For full control over all Vercel AI SDK parameters, create a LanguageModel instance with your desired configuration and pass it to the model parameter. See Using LanguageModel Instances below.

Supported Providers

Section titled “Supported Providers”

OpenAI

Section titled “OpenAI”

OpenAI models are supported through the @ai-sdk/openai package.

Example model IDs:

TypeScript
"gpt-5-nano";
"gpt-5-mini";
"gpt-5";

Usage:

TypeScript
import { evaluatePrompt } from "opik";

await evaluatePrompt({
  dataset,
  messages: [{ role: "user", content: "{{input}}" }],
  model: "gpt-5-nano",
  projectName: "my-project",
});

For a complete list of available models, see the Vercel AI SDK OpenAI provider documentation.

Anthropic

Section titled “Anthropic”

Anthropic's Claude models are supported through the @ai-sdk/anthropic package.

Example model IDs:

TypeScript
"claude-3-5-sonnet-latest";
"claude-3-5-haiku-latest";

Usage:

TypeScript
await evaluatePrompt({
  dataset,
  messages: [{ role: "user", content: "{{input}}" }],
  model: "claude-3-5-sonnet-latest",
  projectName: "my-project",
});

For a complete list of available models, see the Vercel AI SDK Anthropic provider documentation.

Google Gemini

Section titled “Google Gemini”

Google's Gemini models are supported through the @ai-sdk/google package.

Example model IDs:

TypeScript
"gemini-2.0-flash";
"gemini-1.5-pro";

Usage:

TypeScript
await evaluatePrompt({
  dataset,
  messages: [{ role: "user", content: "{{input}}" }],
  model: "gemini-2.0-flash",
  projectName: "my-project",
});

For a complete list of available models, see the Vercel AI SDK Google provider documentation.

Using Models in Opik

Section titled “Using Models in Opik”

Using LanguageModel Instances

Section titled “Using LanguageModel Instances”

For advanced scenarios requiring full Vercel AI SDK features (such as structured outputs, custom headers, or provider-specific parameters), create LanguageModel instances directly:

TypeScript
import { openai } from "@ai-sdk/openai";
import { anthropic } from "@ai-sdk/anthropic";
import { evaluatePrompt } from "opik";
import { Hallucination } from "opik";

// Create models with advanced configuration
const genModel = openai("gpt-5-nano", {
  structuredOutputs: true, // Provider-specific feature
});

const evalModel = anthropic("claude-3-5-sonnet-latest");

// Use different models for generation and evaluation
await evaluatePrompt({
  dataset,
  messages: [{ role: "user", content: "{{input}}" }],
  model: genModel,
  scoringMetrics: [new Hallucination({ model: evalModel })],
  projectName: "my-project",
});

This approach gives you full control over Vercel AI SDK parameters that aren't exposed through Opik's simple interface.

Using Models with Metrics

Section titled “Using Models with Metrics”

LLM Judge metrics accept model configuration:

With Model ID String

Section titled “With Model ID String”
TypeScript
import { Hallucination, AnswerRelevance } from "opik";

// Use different models for different metrics
const hallucinationMetric = new Hallucination({ model: "gpt-5-nano" });
const relevanceMetric = new AnswerRelevance({
  model: "claude-3-5-sonnet-latest",
});

await evaluatePrompt({
  dataset,
  messages: [{ role: "user", content: "{{input}}" }],
  model: "gpt-5-nano",
  scoringMetrics: [hallucinationMetric, relevanceMetric],
  projectName: "my-project",
});

With LanguageModel Instance

Section titled “With LanguageModel Instance”
TypeScript
import { openai } from "@ai-sdk/openai";
import { Hallucination } from "opik";

// Create model for metric evaluation
const judgeModel = openai("gpt-5-nano");

const metric = new Hallucination({ model: judgeModel });

await evaluatePrompt({
  dataset,
  messages: [{ role: "user", content: "{{input}}" }],
  model: "gpt-5-nano",
  scoringMetrics: [metric],
  projectName: "my-project",
});

Custom Model Implementation

Section titled “Custom Model Implementation”

For unsupported providers, implement the OpikBaseModel interface:

OpikBaseModel Interface

Section titled “OpikBaseModel Interface”
TypeScript
abstract class OpikBaseModel {
  constructor(public readonly modelName: string) {}

  /**
   * Generate a string response from a text prompt
   */
  abstract generateString(input: string): Promise<string>;

  /**
   * Generate a response from messages with provider-specific format
   */
  abstract generateProviderResponse(messages: OpikMessage[]): Promise<unknown>;
}

Example Implementation

Section titled “Example Implementation”
TypeScript
import { OpikBaseModel, OpikMessage } from "opik";

class CustomProviderModel extends OpikBaseModel {
  private apiKey: string;
  private baseUrl: string;

  constructor(modelName: string, apiKey: string, baseUrl: string) {
    super(modelName);
    this.apiKey = apiKey;
    this.baseUrl = baseUrl;
  }

  async generateString(input: string): Promise<string> {
    const messages: OpikMessage[] = [
      {
        role: "user",
        content: input,
      },
    ];

    const response = await this.generateProviderResponse(messages);
    // Extract text from provider response format
    return response.choices[0].message.content;
  }

  async generateProviderResponse(messages: OpikMessage[]): Promise<unknown> {
    const response = await fetch(`${this.baseUrl}/chat/completions`, {
      method: "POST",
      headers: {
        Authorization: `Bearer ${this.apiKey}`,
        "Content-Type": "application/json",
      },
      body: JSON.stringify({
        model: this.modelName,
        messages: messages,
      }),
    });

    if (!response.ok) {
      throw new Error(`API request failed: ${response.statusText}`);
    }

    return response.json();
  }
}

// Usage
const customModel = new CustomProviderModel(
  "custom-model-v1",
  process.env.CUSTOM_API_KEY,
  "https://api.custom-provider.com"
);

await evaluatePrompt({
  dataset,
  messages: [{ role: "user", content: "{{input}}" }],
  model: customModel,
  projectName: "my-project",
});

Model Resolution

Section titled “Model Resolution”

The SDK automatically resolves models in this order:

  1. If a string is provided: Auto-detects provider and creates appropriate model
  2. If LanguageModel is provided: Uses the instance directly
  3. If OpikBaseModel is provided: Uses the custom implementation
  4. If undefined: Defaults to "gpt-5-nano"
TypeScript
// String → Auto-detected as OpenAI
model: "gpt-5-nano";

// LanguageModel → Used directly
import { openai } from "@ai-sdk/openai";
model: openai("gpt-5-nano");

// Custom implementation
model: new CustomProviderModel("my-model", apiKey, baseUrl);

// Undefined → Defaults to "gpt-5-nano"
model: undefined;

Best Practices

Section titled “Best Practices”

1. Use Model ID Strings for Simplicity

Section titled “1. Use Model ID Strings for Simplicity”

For most use cases, use model ID strings directly:

TypeScript
import { Hallucination } from "opik";

const metric = new Hallucination({ model: "gpt-5-nano" });

The Opik SDK handles model configuration internally for optimal evaluation performance.

2. Match Model Capabilities to Task

Section titled “2. Match Model Capabilities to Task”

Choose models based on task requirements:

TypeScript
// Complex reasoning: GPT-5, Claude Sonnet
model: "gpt-5";
model: "claude-3-5-sonnet-latest";

// Fast responses: GPT-5-nano, Gemini Flash
model: "gpt-5-nano";
model: "gemini-2.0-flash";

// Long context: Claude, Gemini
model: "claude-3-5-sonnet-latest"; // 200K context
model: "gemini-1.5-pro"; // 1M context

3. Use Different Models for Tasks and Metrics

Section titled “3. Use Different Models for Tasks and Metrics”

Optimize costs by using different models:

TypeScript
await evaluatePrompt({
  dataset,
  messages: [{ role: "user", content: "{{input}}" }],
  model: "gpt-5-nano", // Cheaper for generation
  scoringMetrics: [
    new Hallucination({ model: "gpt-5-mini" }), // More accurate for evaluation
  ],
  projectName: "my-project",
});

4. Configure API Keys

Section titled “4. Configure API Keys”

Set up environment variables for each provider:

Bash
# OpenAI
export OPENAI_API_KEY="sk-..."

# Anthropic
export ANTHROPIC_API_KEY="sk-ant-..."

# Google
export GOOGLE_API_KEY="..."

5. Handle Rate Limits

Section titled “5. Handle Rate Limits”

Use appropriate worker counts to avoid rate limits:

TypeScript
await evaluatePrompt({
  dataset,
  messages: [{ role: "user", content: "{{input}}" }],
  model: "gpt-5-nano",
  taskWorkers: 5, // Limit parallel requests
  projectName: "my-project",
});

Troubleshooting

Section titled “Troubleshooting”

API Key Not Found

Section titled “API Key Not Found”
TypeScript
// Error: API key not found for provider

// Solution: Set environment variable
process.env.OPENAI_API_KEY = "sk-...";

Model Not Supported

Section titled “Model Not Supported”
TypeScript
// Error: Unsupported model ID

// Solution: Use custom implementation
class MyModel extends OpikBaseModel {
  // ... implementation
}

Rate Limit Errors

Section titled “Rate Limit Errors”
TypeScript
// Error: Rate limit exceeded

// Solution: Reduce worker count
await evaluatePrompt({
  dataset,
  messages: [{ role: "user", content: "{{input}}" }],
  model: "gpt-5-nano",
  taskWorkers: 3, // Reduce from default 10
  projectName: "my-project",
});

See Also

Section titled “See Also”
Suggest an edit

Propose a replacement for this page. The site team reviews it before applying any changes.

Export
Documentation menu