Skip to main content
Opik Documentation
current

Search documentation

Type to search this documentation.

On this pageOverview

Known Issues

Known runtime issues when running Opik Optimizer with current dependencies.

Rate limiter errors (pyrate-limiter 4.x)

If pyrate-limiter 4.x is installed you may see TypeError: Limiter.__init__() got an unexpected keyword argument 'raise_when_fail'. That version dropped the legacy flag our optimizer still passes.

Workaround: pin pyrate-limiter to a 3.x release:

Bash
pip install "pyrate-limiter>=3.0.0,<4.0.0"

Fixed in: 3.0.0 (2026-01-26). Upgrade the SDK to remove the legacy flag entirely.

tqdm / rich progress error (tqdm >= 4.71)

convert_tqdm_to_rich.<locals>._tqdm_to_track() missing 1 required positional argument: 'iterable' comes from tqdm >= 4.71 changing the wrapper signature we rely on.

Workaround: pin tqdm to 4.70.0:

Bash
pip install tqdm==4.70.0

Fixed in: 3.0.0 (2026-01-26).

Pydantic serialization warnings

PydanticSerializationUnexpectedValue is emitted when LiteLLM serializes Message objects with fewer fields than the schema (an upstream change in LiteLLM/Pydantic v2). We suppress the warning because the payload is still valid.

Workaround: avoid the affected LiteLLM builds:

Bash
pip install --upgrade "litellm<1.81.1"

Fixed in: 3.0.0 (2026-01-26).

litellm → OpenAI connection errors (1.81.x)

litellm.InternalServerError: OpenAIException - Connection error. has been reproducible against LiteLLM 1.81.*. These releases can break the OpenAI evaluation flow inside Opik Optimizer.

Workaround:

Bash
pip install --upgrade "litellm<1.81.0"

Fixed in: 3.0.0 (2026-01-26).

ValueError: Prompt must be a ChatPrompt object

This error occurs when you pass an incorrect type to the optimizer's optimize_prompt() method.

Solution: Ensure you're using the ChatPrompt class to define your prompt:

Python
from opik_optimizer import ChatPrompt

prompt = ChatPrompt(
    messages=[
        {"role": "system", "content": "Your system prompt here"},
        {"role": "user", "content": "Your user prompt with {variable}"},
    ],
    model="gpt-4",
)
ValueError: Dataset must be a Dataset object

This error occurs when the dataset passed to the optimizer is not a proper Dataset object.

Solution: Use the Dataset class to create your dataset:

Python
import opik

client = opik.Opik()
dataset = client.get_or_create_dataset(name="your-dataset-name", project_name="my-project")
dataset.insert(
    [
        {"input": "example 1", "output": "expected 1"},
        {"input": "example 2", "output": "expected 2"},
    ]
)
ValueError: Metric must be a function

This error occurs when the metric parameter is not callable or doesn't have the correct signature.

Solution: Ensure your metric is a function that takes dataset_item and llm_output as arguments and returns a ScoreResult:

Python
from opik.evaluation.metrics import ScoreResult

def my_metric(dataset_item, llm_output):
    # Your scoring logic here
    score = calculate_score(dataset_item, llm_output)
    return ScoreResult(
        name="my-metric",
        value=score,
        reason="Explanation for the score",
    )
ValueError: Missing required key in prompt

This error occurs when your prompt template contains placeholders (e.g., {variable}) that don't match your dataset fields.

Solution: Ensure all placeholders in your prompt match the keys in your dataset:

Python
# Prompt with {question} placeholder
prompt = ChatPrompt(
    user="Answer: {question}",
    model="gpt-4",
)

# Dataset must have 'question' field
dataset = Dataset.from_list(
    [
        {"question": "What is AI?", "output": "..."},
    ]
)
ImportError: gepa package is required for GepaOptimizer

This error occurs when trying to use the GepaOptimizer without the required gepa package installed.

Solution: Install the gepa package:

Bash
pip install gepa
Exception: Make sure you have provider API key set

This error typically occurs when the LLM provider API key is not configured in your environment.

Solution: Set the appropriate environment variable for your LLM provider:

Bash
# For OpenAI
export OPENAI_API_KEY="your-api-key"

# For Anthropic
export ANTHROPIC_API_KEY="your-api-key"

# For other providers, check the LiteLLM documentation
Suggest an edit

Propose a replacement for this page. The site team reviews it before applying any changes.

Export
Documentation menu