Opik Agent Optimizer API Reference

The Opik Agent Optimizer SDK provides a comprehensive set of tools for optimizing LLM prompts and agents. This reference guide documents the standardized API that all optimizers follow, ensuring consistency and interoperability across different optimization algorithms.

• Standardized API: All optimizers follow the same interface for optimize_prompt() methods
• Multiple Algorithms: Support for various optimization strategies including evolutionary, few-shot, meta-prompt, and GEPA
• MCP Support: Built-in support for Model Context Protocol tool calling and optimization
• Consistent Results: All optimizers return standardized OptimizationResult objects
• Counter Tracking: Built-in LLM and tool call counters for monitoring usage
• Backward Compatibility: All original parameters preserved through kwargs extraction
• Deprecation Warnings: Clear warnings for deprecated parameters with migration guidance

The SDK provides several optimizer classes that all inherit from BaseOptimizer and implement the same standardized interface:

• ParameterOptimizer: Optimizes LLM call parameters (temperature, top_p, etc.) using Bayesian optimization
• FewShotBayesianOptimizer: Uses few-shot learning with Bayesian optimization
• MetaPromptOptimizer: Employs meta-prompting techniques for optimization
• EvolutionaryOptimizer: Uses genetic algorithms for prompt evolution
• GepaOptimizer: Leverages GEPA (Genetic-Pareto) optimization approach
• HRPO (Hierarchical Reflective Prompt Optimizer): Uses hierarchical root cause analysis for targeted prompt refinement

All optimizers implement these core methods with identical signatures:

def optimize_prompt(
    self,
    prompt: ChatPrompt | dict[str, ChatPrompt],
    dataset: Dataset,
    metric: MetricFunction,
    agent: OptimizableAgent | None = None,
    experiment_config: dict | None = None,
    n_samples: int | None = None,
    auto_continue: bool = False,
    project_name: str | None = None,
    optimization_id: str | None = None,
    validation_dataset: Dataset | None = None,
    max_trials: int = 10,
    allow_tool_use: bool = True,
    **kwargs: Any,
) -> OptimizationResult

The following parameters are deprecated and will be removed in future versions:

• num_threads in optimizer constructors: Use n_threads instead

# ❌ Deprecated
optimizer = FewShotBayesianOptimizer(
    model="gpt-4o-mini",
    num_threads=16,  # Deprecated
)

# ✅ Correct
optimizer = FewShotBayesianOptimizer(
    model="gpt-4o-mini",
    n_threads=16,  # Use n_threads instead
)

FewShotBayesianOptimizer(
    model: str = 'openai/gpt-5-nano',
    model_parameters: dict[str, typing.Any] | None = None,
    min_examples: int = 2,
    max_examples: int = 8,
    n_threads: int = 12,
    verbose: int = 1,
    seed: int = 42,
    name: str | None = None,
    enable_columnar_selection: bool = True,
    enable_diversity: bool = True,
    enable_multivariate_tpe: bool = True,
    enable_optuna_pruning: bool = True,
    prompt_overrides: dict[str, str] | collections.abc.Callable[[opik_optimizer.utils.prompt_library.PromptLibrary], None] | None = None,
    skip_perfect_score: bool = True,
    perfect_score: float = 0.95
)

Parameters:

• model (str, default: openai/gpt-5-nano) —

• model_parameters (dict[str, typing.Any] | None) —

• min_examples (int, default: 2) —

• max_examples (int, default: 8) —

• n_threads (int, default: 12) —

• verbose (int, default: 1) —

• seed (int, default: 42) —

• name (str | None) —

• enable_columnar_selection (bool, default: True) —

• enable_diversity (bool, default: True) —

• enable_multivariate_tpe (bool, default: True) —

• enable_optuna_pruning (bool, default: True) —

• prompt_overrides (dict[str, str] | collections.abc.Callable[[opik_optimizer.utils.prompt_library.PromptLibrary], None] | None) —

• skip_perfect_score (bool, default: True) —

• perfect_score (float, default: 0.95) —

begin_round(
    context: OptimizationContext,
    extras: Any
)

Parameters:

• context (OptimizationContext) —

• extras (Any) —

cleanup()

evaluate(
    context: OptimizationContext,
    prompts: dict,
    experiment_config: dict[str, typing.Any] | None = None,
    sampling_tag: str | None = None
)

Parameters:

• context (OptimizationContext) — Optimization context for this run.

• prompts (dict) — Dict of named prompts to evaluate (e.g., {"main": ChatPrompt(...)}). Single-prompt optimizations use a dict with one entry.

• experiment_config (dict[str, typing.Any] | None) — Optional experiment configuration.

• sampling_tag (str | None) — Optional sampling tag for deterministic subsampling per candidate.

evaluate_prompt(
    prompt: opik_optimizer.api_objects.chat_prompt.ChatPrompt | dict[str, opik_optimizer.api_objects.chat_prompt.ChatPrompt],
    dataset: Dataset,
    metric: MetricFunction,
    agent: opik_optimizer.agents.optimizable_agent.OptimizableAgent | None = None,
    n_threads: int | None = None,
    verbose: int = 1,
    dataset_item_ids: list[str] | None = None,
    experiment_config: dict | None = None,
    n_samples: int | float | str | None = None,
    n_samples_strategy: str | None = None,
    seed: int | None = None,
    return_evaluation_result: bool = False,
    allow_tool_use: bool = False,
    use_evaluate_on_dict_items: bool | None = None,
    sampling_tag: str | None = None
)

Parameters:

• prompt (opik_optimizer.api_objects.chat_prompt.ChatPrompt | dict[str, opik_optimizer.api_objects.chat_prompt.ChatPrompt]) —

• dataset (Dataset) —

• metric (MetricFunction) —

• agent (opik_optimizer.agents.optimizable_agent.OptimizableAgent | None) —

• n_threads (int | None) —

• verbose (int, default: 1) —

• dataset_item_ids (list[str] | None) —

• experiment_config (dict | None) —

• n_samples (int | float | str | None) —

• n_samples_strategy (str | None) —

• seed (int | None) —

• return_evaluation_result (bool, default: False) —

• allow_tool_use (bool, default: False) —

• use_evaluate_on_dict_items (bool | None) —

• sampling_tag (str | None) —

evaluate_with_result(
    context: OptimizationContext,
    prompts: dict,
    experiment_config: dict[str, typing.Any] | None = None,
    empty_score: float | None = None,
    n_samples: int | float | str | None = None,
    n_samples_strategy: str | None = None,
    sampling_tag: str | None = None
)

Parameters:

• context (OptimizationContext) —

• prompts (dict) —

• experiment_config (dict[str, typing.Any] | None) —

• empty_score (float | None) —

• n_samples (int | float | str | None) —

• n_samples_strategy (str | None) —

• sampling_tag (str | None) —

finish_candidate(
    context: OptimizationContext,
    candidate_handle: Any,
    score: float | None,
    metrics: dict[str, typing.Any] | None = None,
    extras: dict[str, typing.Any] | None = None,
    candidates: list[dict[str, typing.Any]] | None = None,
    dataset: str | None = None,
    dataset_split: str | None = None,
    trial_index: int | None = None,
    timestamp: str | None = None,
    round_handle: typing.Any | None = None
)

Parameters:

• context (OptimizationContext) —

• candidate_handle (Any) —

• score (float | None) —

• metrics (dict[str, typing.Any] | None) —

• extras (dict[str, typing.Any] | None) —

• candidates (list[dict[str, typing.Any]] | None) —

• dataset (str | None) —

• dataset_split (str | None) —

• trial_index (int | None) —

• timestamp (str | None) —

• round_handle (typing.Any | None) —

finish_round(
    round_handle: Any,
    context: opik_optimizer.core.state.OptimizationContext | None = None,
    best_score: float | None = None,
    best_candidate: typing.Any | None = None,
    best_prompt: typing.Any | None = None,
    stop_reason: str | None = None,
    extras: dict[str, typing.Any] | None = None,
    candidates: list[dict[str, typing.Any]] | None = None,
    timestamp: str | None = None,
    dataset_split: str | None = None,
    pareto_front: list[dict[str, typing.Any]] | None = None,
    selection_meta: dict[str, typing.Any] | None = None
)

Parameters:

• round_handle (Any) —

• context (opik_optimizer.core.state.OptimizationContext | None) —

• best_score (float | None) —

• best_candidate (typing.Any | None) —

• best_prompt (typing.Any | None) —

• stop_reason (str | None) —

• extras (dict[str, typing.Any] | None) —

• candidates (list[dict[str, typing.Any]] | None) —

• timestamp (str | None) —

• dataset_split (str | None) —

• pareto_front (list[dict[str, typing.Any]] | None) —

• selection_meta (dict[str, typing.Any] | None) —

get_config(
    context: OptimizationContext
)

Parameters:

• context (OptimizationContext) —

get_default_prompt(
    key: str
)

Parameters:

• key (str) — The prompt key to retrieve

get_history_entries()

get_history_rounds()

get_metadata(
    context: OptimizationContext
)

Parameters:

• context (OptimizationContext) —

get_optimizer_metadata()

get_prompt(
    key: str,
    fmt: Any
)

Parameters:

• key (str) — The prompt key to retrieve

• fmt (Any) —

list_prompts()

on_trial(
    context: OptimizationContext,
    prompts: dict,
    score: float,
    prev_best_score: float | None = None
)

Parameters:

• context (OptimizationContext) —

• prompts (dict) —

• score (float) —

• prev_best_score (float | None) —

optimize_mcp(
    args: Any,
    kwargs: Any
)

Parameters:

• args (Any) —

• kwargs (Any) —

optimize_prompt(
    prompt: opik_optimizer.api_objects.chat_prompt.ChatPrompt | dict[str, opik_optimizer.api_objects.chat_prompt.ChatPrompt],
    dataset: Dataset,
    metric: MetricFunction,
    agent: opik_optimizer.agents.optimizable_agent.OptimizableAgent | None = None,
    experiment_config: dict | None = None,
    n_samples: int | float | str | None = None,
    n_samples_minibatch: int | None = None,
    n_samples_strategy: str | None = None,
    auto_continue: bool = False,
    project_name: str | None = None,
    optimization_id: str | None = None,
    validation_dataset: opik.api_objects.dataset.dataset.Dataset | None = None,
    max_trials: int = 10,
    allow_tool_use: bool = True,
    optimize_prompts: bool | str | list[str] | None = 'system',
    optimize_tools: bool | dict[str, bool] | None = None,
    args: Any,
    kwargs: Any
)

Parameters:

• prompt (opik_optimizer.api_objects.chat_prompt.ChatPrompt | dict[str, opik_optimizer.api_objects.chat_prompt.ChatPrompt]) — The prompt to optimize (single ChatPrompt or dict of prompts)

• dataset (Dataset) — Opik dataset (training set - used for feedback/context) TODO/FIXME: This parameter will be deprecated in favor of dataset_training. For now, it serves as the training dataset parameter.

• metric (MetricFunction) — A metric function with signature (dataset_item, llm_output) -> float

• agent (opik_optimizer.agents.optimizable_agent.OptimizableAgent | None) — Optional agent for prompt execution (defaults to LiteLLMAgent)

• experiment_config (dict | None) — Optional configuration for the experiment

• n_samples (int | float | str | None) — Number of samples to use for evaluation

• n_samples_minibatch (int | None) — Optional number of samples for inner-loop minibatches

• n_samples_strategy (str | None) — Sampling strategy name (default "random_sorted")

• auto_continue (bool, default: False) — Whether to continue optimization automatically

• project_name (str | None) — Opik project name for logging traces (defaults to OPIK_PROJECT_NAME env or "Optimization")

• optimization_id (str | None) — Optional ID to use when creating the Opik optimization run

• validation_dataset (opik.api_objects.dataset.dataset.Dataset | None) — Optional validation dataset for ranking candidates

• max_trials (int, default: 10) — Maximum number of optimization trials

• allow_tool_use (bool, default: True) — Whether tools may be executed during evaluation (default True)

• optimize_prompts (bool | str | list[str] | None, default: system) — Which prompt roles to allow for optimization

• optimize_tools (bool | dict[str, bool] | None) — Optional tool optimization selector. Only supported by optimizers that explicitly document tool optimization support.

• args (Any) —

• kwargs (Any) —

post_baseline(
    context: OptimizationContext,
    score: float
)

Parameters:

• context (OptimizationContext) —

• score (float) —

post_optimize(
    context: OptimizationContext,
    result: OptimizationResult
)

Parameters:

• context (OptimizationContext) —

• result (OptimizationResult) —

post_round(
    round_handle: Any,
    context: opik_optimizer.core.state.OptimizationContext | None = None,
    best_score: float | None = None,
    best_candidate: typing.Any | None = None,
    best_prompt: typing.Any | None = None,
    stop_reason: str | None = None,
    extras: dict[str, typing.Any] | None = None,
    candidates: list[dict[str, typing.Any]] | None = None,
    timestamp: str | None = None,
    dataset_split: str | None = None,
    pareto_front: list[dict[str, typing.Any]] | None = None,
    selection_meta: dict[str, typing.Any] | None = None
)

Parameters:

• round_handle (Any) —

• context (opik_optimizer.core.state.OptimizationContext | None) —

• best_score (float | None) —

• best_candidate (typing.Any | None) —

• best_prompt (typing.Any | None) —

• stop_reason (str | None) —

• extras (dict[str, typing.Any] | None) —

• candidates (list[dict[str, typing.Any]] | None) —

• timestamp (str | None) —

• dataset_split (str | None) —

• pareto_front (list[dict[str, typing.Any]] | None) —

• selection_meta (dict[str, typing.Any] | None) —

post_trial(
    context: OptimizationContext,
    candidate_handle: Any,
    score: float | None,
    metrics: dict[str, typing.Any] | None = None,
    extras: dict[str, typing.Any] | None = None,
    candidates: list[dict[str, typing.Any]] | None = None,
    dataset: str | None = None,
    dataset_split: str | None = None,
    trial_index: int | None = None,
    timestamp: str | None = None,
    round_handle: typing.Any | None = None
)

Parameters:

• context (OptimizationContext) —

• candidate_handle (Any) —

• score (float | None) —

• metrics (dict[str, typing.Any] | None) —

• extras (dict[str, typing.Any] | None) —

• candidates (list[dict[str, typing.Any]] | None) —

• dataset (str | None) —

• dataset_split (str | None) —

• trial_index (int | None) —

• timestamp (str | None) —

• round_handle (typing.Any | None) —

pre_baseline(
    context: OptimizationContext
)

Parameters:

• context (OptimizationContext) —

pre_optimize(
    context: OptimizationContext
)

Parameters:

• context (OptimizationContext) — The optimization context

pre_round(
    context: OptimizationContext,
    extras: Any
)

Parameters:

• context (OptimizationContext) —

• extras (Any) —

pre_trial(
    context: OptimizationContext,
    candidate: Any,
    round_handle: typing.Any | None = None
)

Parameters:

• context (OptimizationContext) —

• candidate (Any) —

• round_handle (typing.Any | None) —

record_candidate_entry(
    prompt_or_payload: Any,
    score: float | None = None,
    id: str | None = None,
    metrics: dict[str, typing.Any] | None = None,
    notes: str | None = None,
    extra: dict[str, typing.Any] | None = None,
    context: opik_optimizer.core.state.OptimizationContext | None = None
)

Parameters:

• prompt_or_payload (Any) —

• score (float | None) —

• id (str | None) —

• metrics (dict[str, typing.Any] | None) —

• notes (str | None) —

• extra (dict[str, typing.Any] | None) —

• context (opik_optimizer.core.state.OptimizationContext | None) —

run_optimization(
    context: OptimizationContext
)

Parameters:

• context (OptimizationContext) — The optimization context with prompts, dataset, metric, etc.

set_default_dataset_split(
    dataset_split: str | None
)

Parameters:

• dataset_split (str | None) —

set_pareto_front(
    pareto_front: list[dict[str, typing.Any]] | None
)

Parameters:

• pareto_front (list[dict[str, typing.Any]] | None) —

set_selection_meta(
    selection_meta: dict[str, typing.Any] | None
)

Parameters:

• selection_meta (dict[str, typing.Any] | None) —

start_candidate(
    context: OptimizationContext,
    candidate: Any,
    round_handle: typing.Any | None = None
)

Parameters:

• context (OptimizationContext) —

• candidate (Any) —

• round_handle (typing.Any | None) —

with_dataset_split(
    dataset_split: str | None
)

Parameters:

• dataset_split (str | None) —

GepaOptimizer(
    model: str = 'openai/gpt-5-nano',
    model_parameters: dict[str, typing.Any] | None = None,
    n_threads: int = 12,
    verbose: int = 1,
    seed: int = 42,
    name: str | None = None,
    skip_perfect_score: bool = True,
    perfect_score: float = 0.95,
    prompt_overrides: dict[str, str] | collections.abc.Callable[[opik_optimizer.utils.prompt_library.PromptLibrary], None] | None = None
)

Parameters:

• model (str, default: openai/gpt-5-nano) —

• model_parameters (dict[str, typing.Any] | None) —

• n_threads (int, default: 12) —

• verbose (int, default: 1) —

• seed (int, default: 42) —

• name (str | None) —

• skip_perfect_score (bool, default: True) —

• perfect_score (float, default: 0.95) —

• prompt_overrides (dict[str, str] | collections.abc.Callable[[opik_optimizer.utils.prompt_library.PromptLibrary], None] | None) —

begin_round(
    context: OptimizationContext,
    extras: Any
)

Parameters:

• context (OptimizationContext) —

• extras (Any) —

cleanup()

evaluate(
    context: OptimizationContext,
    prompts: dict,
    experiment_config: dict[str, typing.Any] | None = None,
    sampling_tag: str | None = None
)

Parameters:

• context (OptimizationContext) — Optimization context for this run.

• prompts (dict) — Dict of named prompts to evaluate (e.g., {"main": ChatPrompt(...)}). Single-prompt optimizations use a dict with one entry.

• experiment_config (dict[str, typing.Any] | None) — Optional experiment configuration.

• sampling_tag (str | None) — Optional sampling tag for deterministic subsampling per candidate.

evaluate_prompt(
    prompt: opik_optimizer.api_objects.chat_prompt.ChatPrompt | dict[str, opik_optimizer.api_objects.chat_prompt.ChatPrompt],
    dataset: Dataset,
    metric: MetricFunction,
    agent: opik_optimizer.agents.optimizable_agent.OptimizableAgent | None = None,
    n_threads: int | None = None,
    verbose: int = 1,
    dataset_item_ids: list[str] | None = None,
    experiment_config: dict | None = None,
    n_samples: int | float | str | None = None,
    n_samples_strategy: str | None = None,
    seed: int | None = None,
    return_evaluation_result: bool = False,
    allow_tool_use: bool | None = None,
    use_evaluate_on_dict_items: bool | None = None,
    sampling_tag: str | None = None
)

Parameters:

• prompt (opik_optimizer.api_objects.chat_prompt.ChatPrompt | dict[str, opik_optimizer.api_objects.chat_prompt.ChatPrompt]) —

• dataset (Dataset) —

• metric (MetricFunction) —

• agent (opik_optimizer.agents.optimizable_agent.OptimizableAgent | None) —

• n_threads (int | None) —

• verbose (int, default: 1) —

• dataset_item_ids (list[str] | None) —

• experiment_config (dict | None) —

• n_samples (int | float | str | None) —

• n_samples_strategy (str | None) —

• seed (int | None) —

• return_evaluation_result (bool, default: False) —

• allow_tool_use (bool | None) —

• use_evaluate_on_dict_items (bool | None) —

• sampling_tag (str | None) —

evaluate_with_result(
    context: OptimizationContext,
    prompts: dict,
    experiment_config: dict[str, typing.Any] | None = None,
    empty_score: float | None = None,
    n_samples: int | float | str | None = None,
    n_samples_strategy: str | None = None,
    sampling_tag: str | None = None
)

Parameters:

• context (OptimizationContext) —

• prompts (dict) —

• experiment_config (dict[str, typing.Any] | None) —

• empty_score (float | None) —

• n_samples (int | float | str | None) —

• n_samples_strategy (str | None) —

• sampling_tag (str | None) —

finish_candidate(
    context: OptimizationContext,
    candidate_handle: Any,
    score: float | None,
    metrics: dict[str, typing.Any] | None = None,
    extras: dict[str, typing.Any] | None = None,
    candidates: list[dict[str, typing.Any]] | None = None,
    dataset: str | None = None,
    dataset_split: str | None = None,
    trial_index: int | None = None,
    timestamp: str | None = None,
    round_handle: typing.Any | None = None
)

Parameters:

• context (OptimizationContext) —

• candidate_handle (Any) —

• score (float | None) —

• metrics (dict[str, typing.Any] | None) —

• extras (dict[str, typing.Any] | None) —

• candidates (list[dict[str, typing.Any]] | None) —

• dataset (str | None) —

• dataset_split (str | None) —

• trial_index (int | None) —

• timestamp (str | None) —

• round_handle (typing.Any | None) —

finish_round(
    round_handle: Any,
    context: opik_optimizer.core.state.OptimizationContext | None = None,
    best_score: float | None = None,
    best_candidate: typing.Any | None = None,
    best_prompt: typing.Any | None = None,
    stop_reason: str | None = None,
    extras: dict[str, typing.Any] | None = None,
    candidates: list[dict[str, typing.Any]] | None = None,
    timestamp: str | None = None,
    dataset_split: str | None = None,
    pareto_front: list[dict[str, typing.Any]] | None = None,
    selection_meta: dict[str, typing.Any] | None = None
)

Parameters:

• round_handle (Any) —

• context (opik_optimizer.core.state.OptimizationContext | None) —

• best_score (float | None) —

• best_candidate (typing.Any | None) —

• best_prompt (typing.Any | None) —

• stop_reason (str | None) —

• extras (dict[str, typing.Any] | None) —

• candidates (list[dict[str, typing.Any]] | None) —

• timestamp (str | None) —

• dataset_split (str | None) —

• pareto_front (list[dict[str, typing.Any]] | None) —

• selection_meta (dict[str, typing.Any] | None) —

get_config(
    context: OptimizationContext
)

Parameters:

• context (OptimizationContext) —

get_default_prompt(
    key: str
)

Parameters:

• key (str) — The prompt key to retrieve

get_history_entries()

get_history_rounds()

get_metadata(
    context: OptimizationContext
)

Parameters:

• context (OptimizationContext) —

get_optimizer_metadata()

get_prompt(
    key: str,
    fmt: Any
)

Parameters:

• key (str) — The prompt key to retrieve

• fmt (Any) —

list_prompts()

on_trial(
    context: OptimizationContext,
    prompts: dict,
    score: float,
    prev_best_score: float | None = None
)

Parameters:

• context (OptimizationContext) —

• prompts (dict) —

• score (float) —

• prev_best_score (float | None) —

optimize_mcp(
    args: Any,
    kwargs: Any
)

Parameters:

• args (Any) —

• kwargs (Any) —

optimize_prompt(
    prompt: opik_optimizer.api_objects.chat_prompt.ChatPrompt | dict[str, opik_optimizer.api_objects.chat_prompt.ChatPrompt],
    dataset: Dataset,
    metric: MetricFunction,
    agent: opik_optimizer.agents.optimizable_agent.OptimizableAgent | None = None,
    experiment_config: dict | None = None,
    n_samples: int | float | str | None = None,
    n_samples_minibatch: int | None = None,
    n_samples_strategy: str | None = None,
    auto_continue: bool = False,
    project_name: str | None = None,
    optimization_id: str | None = None,
    validation_dataset: opik.api_objects.dataset.dataset.Dataset | None = None,
    max_trials: int = 10,
    allow_tool_use: bool = True,
    optimize_prompts: bool | str | list[str] | None = 'system',
    optimize_tools: bool | dict[str, bool] | None = None,
    args: Any,
    kwargs: Any
)

Parameters:

• prompt (opik_optimizer.api_objects.chat_prompt.ChatPrompt | dict[str, opik_optimizer.api_objects.chat_prompt.ChatPrompt]) — The prompt to optimize (single ChatPrompt or dict of prompts)

• dataset (Dataset) — Opik dataset (training set - used for feedback/context) TODO/FIXME: This parameter will be deprecated in favor of dataset_training. For now, it serves as the training dataset parameter.

• metric (MetricFunction) — A metric function with signature (dataset_item, llm_output) -> float

• agent (opik_optimizer.agents.optimizable_agent.OptimizableAgent | None) — Optional agent for prompt execution (defaults to LiteLLMAgent)

• experiment_config (dict | None) — Optional configuration for the experiment

• n_samples (int | float | str | None) — Number of samples to use for evaluation

• n_samples_minibatch (int | None) — Optional number of samples for inner-loop minibatches

• n_samples_strategy (str | None) — Sampling strategy name (default "random_sorted")

• auto_continue (bool, default: False) — Whether to continue optimization automatically

• project_name (str | None) — Opik project name for logging traces (defaults to OPIK_PROJECT_NAME env or "Optimization")

• optimization_id (str | None) — Optional ID to use when creating the Opik optimization run

• validation_dataset (opik.api_objects.dataset.dataset.Dataset | None) — Optional validation dataset for ranking candidates

• max_trials (int, default: 10) — Maximum number of optimization trials

• allow_tool_use (bool, default: True) — Whether tools may be executed during evaluation (default True)

• optimize_prompts (bool | str | list[str] | None, default: system) — Which prompt roles to allow for optimization

• optimize_tools (bool | dict[str, bool] | None) — Optional tool optimization selector. Only supported by optimizers that explicitly document tool optimization support.

• args (Any) —

• kwargs (Any) —

post_baseline(
    context: OptimizationContext,
    score: float
)

Parameters:

• context (OptimizationContext) —

• score (float) —

post_optimize(
    context: OptimizationContext,
    result: OptimizationResult
)

Parameters:

• context (OptimizationContext) —

• result (OptimizationResult) —

post_round(
    round_handle: Any,
    context: opik_optimizer.core.state.OptimizationContext | None = None,
    best_score: float | None = None,
    best_candidate: typing.Any | None = None,
    best_prompt: typing.Any | None = None,
    stop_reason: str | None = None,
    extras: dict[str, typing.Any] | None = None,
    candidates: list[dict[str, typing.Any]] | None = None,
    timestamp: str | None = None,
    dataset_split: str | None = None,
    pareto_front: list[dict[str, typing.Any]] | None = None,
    selection_meta: dict[str, typing.Any] | None = None
)

Parameters:

• round_handle (Any) —

• context (opik_optimizer.core.state.OptimizationContext | None) —

• best_score (float | None) —

• best_candidate (typing.Any | None) —

• best_prompt (typing.Any | None) —

• stop_reason (str | None) —

• extras (dict[str, typing.Any] | None) —

• candidates (list[dict[str, typing.Any]] | None) —

• timestamp (str | None) —

• dataset_split (str | None) —

• pareto_front (list[dict[str, typing.Any]] | None) —

• selection_meta (dict[str, typing.Any] | None) —

post_trial(
    context: OptimizationContext,
    candidate_handle: Any,
    score: float | None,
    metrics: dict[str, typing.Any] | None = None,
    extras: dict[str, typing.Any] | None = None,
    candidates: list[dict[str, typing.Any]] | None = None,
    dataset: str | None = None,
    dataset_split: str | None = None,
    trial_index: int | None = None,
    timestamp: str | None = None,
    round_handle: typing.Any | None = None
)

Parameters:

• context (OptimizationContext) —

• candidate_handle (Any) —

• score (float | None) —

• metrics (dict[str, typing.Any] | None) —

• extras (dict[str, typing.Any] | None) —

• candidates (list[dict[str, typing.Any]] | None) —

• dataset (str | None) —

• dataset_split (str | None) —

• trial_index (int | None) —

• timestamp (str | None) —

• round_handle (typing.Any | None) —

pre_baseline(
    context: OptimizationContext
)

Parameters:

• context (OptimizationContext) —

pre_optimize(
    context: OptimizationContext
)

Parameters:

• context (OptimizationContext) —

pre_round(
    context: OptimizationContext,
    extras: Any
)

Parameters:

• context (OptimizationContext) —

• extras (Any) —

pre_trial(
    context: OptimizationContext,
    candidate: Any,
    round_handle: typing.Any | None = None
)

Parameters:

• context (OptimizationContext) —

• candidate (Any) —

• round_handle (typing.Any | None) —

record_candidate_entry(
    prompt_or_payload: Any,
    score: float | None = None,
    id: str | None = None,
    metrics: dict[str, typing.Any] | None = None,
    notes: str | None = None,
    extra: dict[str, typing.Any] | None = None,
    context: opik_optimizer.core.state.OptimizationContext | None = None
)

Parameters:

• prompt_or_payload (Any) —

• score (float | None) —

• id (str | None) —

• metrics (dict[str, typing.Any] | None) —

• notes (str | None) —

• extra (dict[str, typing.Any] | None) —

• context (opik_optimizer.core.state.OptimizationContext | None) —

run_optimization(
    context: OptimizationContext
)

Parameters:

• context (OptimizationContext) — The optimization context with prompts, dataset, metric, etc.

set_default_dataset_split(
    dataset_split: str | None
)

Parameters:

• dataset_split (str | None) —

set_pareto_front(
    pareto_front: list[dict[str, typing.Any]] | None
)

Parameters:

• pareto_front (list[dict[str, typing.Any]] | None) —

set_selection_meta(
    selection_meta: dict[str, typing.Any] | None
)

Parameters:

• selection_meta (dict[str, typing.Any] | None) —

start_candidate(
    context: OptimizationContext,
    candidate: Any,
    round_handle: typing.Any | None = None
)

Parameters:

• context (OptimizationContext) —

• candidate (Any) —

• round_handle (typing.Any | None) —

with_dataset_split(
    dataset_split: str | None
)

Parameters:

• dataset_split (str | None) —

MetaPromptOptimizer(
    model: str = 'openai/gpt-5-nano',
    model_parameters: dict[str, typing.Any] | None = None,
    prompts_per_round: int = 4,
    enable_context: bool = True,
    num_task_examples: int = 5,
    task_context_columns: list[str] | None = None,
    n_threads: int = 12,
    verbose: int = 1,
    seed: int = 42,
    name: str | None = None,
    use_hall_of_fame: bool = True,
    prompt_overrides: dict[str, str] | collections.abc.Callable[[opik_optimizer.utils.prompt_library.PromptLibrary], None] | None = None,
    skip_perfect_score: bool = True,
    perfect_score: float = 0.95
)

Parameters:

• model (str, default: openai/gpt-5-nano) —

• model_parameters (dict[str, typing.Any] | None) —

• prompts_per_round (int, default: 4) —

• enable_context (bool, default: True) —

• num_task_examples (int, default: 5) —

• task_context_columns (list[str] | None) —

• n_threads (int, default: 12) —

• verbose (int, default: 1) —

• seed (int, default: 42) —

• name (str | None) —

• use_hall_of_fame (bool, default: True) —

• prompt_overrides (dict[str, str] | collections.abc.Callable[[opik_optimizer.utils.prompt_library.PromptLibrary], None] | None) —

• skip_perfect_score (bool, default: True) —

• perfect_score (float, default: 0.95) —

begin_round(
    context: OptimizationContext,
    extras: Any
)

Parameters:

• context (OptimizationContext) —

• extras (Any) —

cleanup()

evaluate(
    context: OptimizationContext,
    prompts: dict,
    experiment_config: dict[str, typing.Any] | None = None,
    sampling_tag: str | None = None
)

Parameters:

• context (OptimizationContext) — Optimization context for this run.

• prompts (dict) — Dict of named prompts to evaluate (e.g., {"main": ChatPrompt(...)}). Single-prompt optimizations use a dict with one entry.

• experiment_config (dict[str, typing.Any] | None) — Optional experiment configuration.

• sampling_tag (str | None) — Optional sampling tag for deterministic subsampling per candidate.

evaluate_prompt(
    prompt: opik_optimizer.api_objects.chat_prompt.ChatPrompt | dict[str, opik_optimizer.api_objects.chat_prompt.ChatPrompt],
    dataset: Dataset,
    metric: MetricFunction,
    agent: opik_optimizer.agents.optimizable_agent.OptimizableAgent | None = None,
    n_threads: int | None = None,
    verbose: int = 1,
    dataset_item_ids: list[str] | None = None,
    experiment_config: dict | None = None,
    n_samples: int | float | str | None = None,
    n_samples_strategy: str | None = None,
    seed: int | None = None,
    return_evaluation_result: bool = False,
    allow_tool_use: bool | None = None,
    use_evaluate_on_dict_items: bool | None = None,
    sampling_tag: str | None = None
)

Parameters:

• prompt (opik_optimizer.api_objects.chat_prompt.ChatPrompt | dict[str, opik_optimizer.api_objects.chat_prompt.ChatPrompt]) —

• dataset (Dataset) —

• metric (MetricFunction) —

• agent (opik_optimizer.agents.optimizable_agent.OptimizableAgent | None) —

• n_threads (int | None) —

• verbose (int, default: 1) —

• dataset_item_ids (list[str] | None) —

• experiment_config (dict | None) —

• n_samples (int | float | str | None) —

• n_samples_strategy (str | None) —

• seed (int | None) —

• return_evaluation_result (bool, default: False) —

• allow_tool_use (bool | None) —

• use_evaluate_on_dict_items (bool | None) —

• sampling_tag (str | None) —

evaluate_with_result(
    context: OptimizationContext,
    prompts: dict,
    experiment_config: dict[str, typing.Any] | None = None,
    empty_score: float | None = None,
    n_samples: int | float | str | None = None,
    n_samples_strategy: str | None = None,
    sampling_tag: str | None = None
)

Parameters:

• context (OptimizationContext) —

• prompts (dict) —

• experiment_config (dict[str, typing.Any] | None) —

• empty_score (float | None) —

• n_samples (int | float | str | None) —

• n_samples_strategy (str | None) —

• sampling_tag (str | None) —

finish_candidate(
    context: OptimizationContext,
    candidate_handle: Any,
    score: float | None,
    metrics: dict[str, typing.Any] | None = None,
    extras: dict[str, typing.Any] | None = None,
    candidates: list[dict[str, typing.Any]] | None = None,
    dataset: str | None = None,
    dataset_split: str | None = None,
    trial_index: int | None = None,
    timestamp: str | None = None,
    round_handle: typing.Any | None = None
)

Parameters:

• context (OptimizationContext) —

• candidate_handle (Any) —

• score (float | None) —

• metrics (dict[str, typing.Any] | None) —

• extras (dict[str, typing.Any] | None) —

• candidates (list[dict[str, typing.Any]] | None) —

• dataset (str | None) —

• dataset_split (str | None) —

• trial_index (int | None) —

• timestamp (str | None) —

• round_handle (typing.Any | None) —

finish_round(
    round_handle: Any,
    context: opik_optimizer.core.state.OptimizationContext | None = None,
    best_score: float | None = None,
    best_candidate: typing.Any | None = None,
    best_prompt: typing.Any | None = None,
    stop_reason: str | None = None,
    extras: dict[str, typing.Any] | None = None,
    candidates: list[dict[str, typing.Any]] | None = None,
    timestamp: str | None = None,
    dataset_split: str | None = None,
    pareto_front: list[dict[str, typing.Any]] | None = None,
    selection_meta: dict[str, typing.Any] | None = None
)

Parameters:

• round_handle (Any) —

• context (opik_optimizer.core.state.OptimizationContext | None) —

• best_score (float | None) —

• best_candidate (typing.Any | None) —

• best_prompt (typing.Any | None) —

• stop_reason (str | None) —

• extras (dict[str, typing.Any] | None) —

• candidates (list[dict[str, typing.Any]] | None) —

• timestamp (str | None) —

• dataset_split (str | None) —

• pareto_front (list[dict[str, typing.Any]] | None) —

• selection_meta (dict[str, typing.Any] | None) —

get_config(
    context: OptimizationContext
)

Parameters:

• context (OptimizationContext) —

get_default_prompt(
    key: str
)

Parameters:

• key (str) — The prompt key to retrieve

get_history_entries()

get_history_rounds()

get_metadata(
    context: OptimizationContext
)

Parameters:

• context (OptimizationContext) —

get_optimizer_metadata()

get_prompt(
    key: str,
    fmt: Any
)

Parameters:

• key (str) — The prompt key to retrieve

• fmt (Any) —

list_prompts()

on_trial(
    context: OptimizationContext,
    prompts: dict,
    score: float,
    prev_best_score: float | None = None
)

Parameters:

• context (OptimizationContext) —

• prompts (dict) —

• score (float) —

• prev_best_score (float | None) —

optimize_mcp(
    args: Any,
    kwargs: Any
)

Parameters:

• args (Any) —

• kwargs (Any) —

optimize_prompt(
    prompt: opik_optimizer.api_objects.chat_prompt.ChatPrompt | dict[str, opik_optimizer.api_objects.chat_prompt.ChatPrompt],
    dataset: Dataset,
    metric: MetricFunction,
    agent: opik_optimizer.agents.optimizable_agent.OptimizableAgent | None = None,
    experiment_config: dict | None = None,
    n_samples: int | float | str | None = None,
    n_samples_minibatch: int | None = None,
    n_samples_strategy: str | None = None,
    auto_continue: bool = False,
    project_name: str | None = None,
    optimization_id: str | None = None,
    validation_dataset: opik.api_objects.dataset.dataset.Dataset | None = None,
    max_trials: int = 10,
    allow_tool_use: bool = True,
    optimize_prompts: bool | str | list[str] | None = 'system',
    optimize_tools: bool | dict[str, bool] | None = None,
    args: Any,
    kwargs: Any
)

Parameters:

• prompt (opik_optimizer.api_objects.chat_prompt.ChatPrompt | dict[str, opik_optimizer.api_objects.chat_prompt.ChatPrompt]) — The prompt to optimize (single ChatPrompt or dict of prompts)

• dataset (Dataset) — Opik dataset (training set - used for feedback/context) TODO/FIXME: This parameter will be deprecated in favor of dataset_training. For now, it serves as the training dataset parameter.

• metric (MetricFunction) — A metric function with signature (dataset_item, llm_output) -> float

• agent (opik_optimizer.agents.optimizable_agent.OptimizableAgent | None) — Optional agent for prompt execution (defaults to LiteLLMAgent)

• experiment_config (dict | None) — Optional configuration for the experiment

• n_samples (int | float | str | None) — Number of samples to use for evaluation

• n_samples_minibatch (int | None) — Optional number of samples for inner-loop minibatches

• n_samples_strategy (str | None) — Sampling strategy name (default "random_sorted")

• auto_continue (bool, default: False) — Whether to continue optimization automatically

• project_name (str | None) — Opik project name for logging traces (defaults to OPIK_PROJECT_NAME env or "Optimization")

• optimization_id (str | None) — Optional ID to use when creating the Opik optimization run

• validation_dataset (opik.api_objects.dataset.dataset.Dataset | None) — Optional validation dataset for ranking candidates

• max_trials (int, default: 10) — Maximum number of optimization trials

• allow_tool_use (bool, default: True) — Whether tools may be executed during evaluation (default True)

• optimize_prompts (bool | str | list[str] | None, default: system) — Which prompt roles to allow for optimization

• optimize_tools (bool | dict[str, bool] | None) — Optional tool optimization selector. Only supported by optimizers that explicitly document tool optimization support.

• args (Any) —

• kwargs (Any) —

post_baseline(
    context: OptimizationContext,
    score: float
)

Parameters:

• context (OptimizationContext) —

• score (float) —

post_optimize(
    context: OptimizationContext,
    result: OptimizationResult
)

Parameters:

• context (OptimizationContext) —

• result (OptimizationResult) —

post_round(
    round_handle: Any,
    context: opik_optimizer.core.state.OptimizationContext | None = None,
    best_score: float | None = None,
    best_candidate: typing.Any | None = None,
    best_prompt: typing.Any | None = None,
    stop_reason: str | None = None,
    extras: dict[str, typing.Any] | None = None,
    candidates: list[dict[str, typing.Any]] | None = None,
    timestamp: str | None = None,
    dataset_split: str | None = None,
    pareto_front: list[dict[str, typing.Any]] | None = None,
    selection_meta: dict[str, typing.Any] | None = None
)

Parameters:

• round_handle (Any) —

• context (opik_optimizer.core.state.OptimizationContext | None) —

• best_score (float | None) —

• best_candidate (typing.Any | None) —

• best_prompt (typing.Any | None) —

• stop_reason (str | None) —

• extras (dict[str, typing.Any] | None) —

• candidates (list[dict[str, typing.Any]] | None) —

• timestamp (str | None) —

• dataset_split (str | None) —

• pareto_front (list[dict[str, typing.Any]] | None) —

• selection_meta (dict[str, typing.Any] | None) —

post_trial(
    context: OptimizationContext,
    candidate_handle: Any,
    score: float | None,
    metrics: dict[str, typing.Any] | None = None,
    extras: dict[str, typing.Any] | None = None,
    candidates: list[dict[str, typing.Any]] | None = None,
    dataset: str | None = None,
    dataset_split: str | None = None,
    trial_index: int | None = None,
    timestamp: str | None = None,
    round_handle: typing.Any | None = None
)

Parameters:

• context (OptimizationContext) —

• candidate_handle (Any) —

• score (float | None) —

• metrics (dict[str, typing.Any] | None) —

• extras (dict[str, typing.Any] | None) —

• candidates (list[dict[str, typing.Any]] | None) —

• dataset (str | None) —

• dataset_split (str | None) —

• trial_index (int | None) —

• timestamp (str | None) —

• round_handle (typing.Any | None) —

pre_baseline(
    context: OptimizationContext
)

Parameters:

• context (OptimizationContext) —

pre_optimize(
    context: OptimizationContext
)

Parameters:

• context (OptimizationContext) — The optimization context

pre_round(
    context: OptimizationContext,
    extras: Any
)

Parameters:

• context (OptimizationContext) —

• extras (Any) —

pre_trial(
    context: OptimizationContext,
    candidate: Any,
    round_handle: typing.Any | None = None
)

Parameters:

• context (OptimizationContext) —

• candidate (Any) —

• round_handle (typing.Any | None) —

record_candidate_entry(
    prompt_or_payload: Any,
    score: float | None = None,
    id: str | None = None,
    metrics: dict[str, typing.Any] | None = None,
    notes: str | None = None,
    extra: dict[str, typing.Any] | None = None,
    context: opik_optimizer.core.state.OptimizationContext | None = None
)

Parameters:

• prompt_or_payload (Any) —

• score (float | None) —

• id (str | None) —

• metrics (dict[str, typing.Any] | None) —

• notes (str | None) —

• extra (dict[str, typing.Any] | None) —

• context (opik_optimizer.core.state.OptimizationContext | None) —

run_optimization(
    context: OptimizationContext
)

Parameters:

• context (OptimizationContext) — The optimization context with prompts, dataset, metric, etc.

set_default_dataset_split(
    dataset_split: str | None
)

Parameters:

• dataset_split (str | None) —

set_pareto_front(
    pareto_front: list[dict[str, typing.Any]] | None
)

Parameters:

• pareto_front (list[dict[str, typing.Any]] | None) —

set_selection_meta(
    selection_meta: dict[str, typing.Any] | None
)

Parameters:

• selection_meta (dict[str, typing.Any] | None) —

start_candidate(
    context: OptimizationContext,
    candidate: Any,
    round_handle: typing.Any | None = None
)

Parameters:

• context (OptimizationContext) —

• candidate (Any) —

• round_handle (typing.Any | None) —

with_dataset_split(
    dataset_split: str | None
)

Parameters:

• dataset_split (str | None) —

EvolutionaryOptimizer(
    model: str = 'openai/gpt-5-nano',
    model_parameters: dict[str, typing.Any] | None = None,
    population_size: int = 30,
    num_generations: int = 15,
    mutation_rate: float = 0.2,
    crossover_rate: float = 0.8,
    tournament_size: int = 4,
    elitism_size: int = 3,
    adaptive_mutation: bool = True,
    enable_moo: bool = True,
    enable_llm_crossover: bool = True,
    enable_semantic_crossover: bool = False,
    output_style_guidance: str | None = None,
    infer_output_style: bool = False,
    n_threads: int = 12,
    verbose: int = 1,
    seed: int = 42,
    name: str | None = None,
    prompt_overrides: dict[str, str] | collections.abc.Callable[[opik_optimizer.utils.prompt_library.PromptLibrary], None] | None = None,
    skip_perfect_score: bool = True,
    perfect_score: float = 0.95
)

Parameters:

• model (str, default: openai/gpt-5-nano) —

• model_parameters (dict[str, typing.Any] | None) —

• population_size (int, default: 30) —

• num_generations (int, default: 15) —

• mutation_rate (float, default: 0.2) —

• crossover_rate (float, default: 0.8) —

• tournament_size (int, default: 4) —

• elitism_size (int, default: 3) —

• adaptive_mutation (bool, default: True) —

• enable_moo (bool, default: True) —

• enable_llm_crossover (bool, default: True) —

• enable_semantic_crossover (bool, default: False) —

• output_style_guidance (str | None) —

• infer_output_style (bool, default: False) —

• n_threads (int, default: 12) —

• verbose (int, default: 1) —

• seed (int, default: 42) —

• name (str | None) —

• prompt_overrides (dict[str, str] | collections.abc.Callable[[opik_optimizer.utils.prompt_library.PromptLibrary], None] | None) —

• skip_perfect_score (bool, default: True) —

• perfect_score (float, default: 0.95) —

begin_round(
    context: OptimizationContext,
    extras: Any
)

Parameters:

• context (OptimizationContext) —

• extras (Any) —

cleanup()

evaluate(
    context: OptimizationContext,
    prompts: dict,
    experiment_config: dict[str, typing.Any] | None = None,
    sampling_tag: str | None = None
)

Parameters:

• context (OptimizationContext) — Optimization context for this run.

• prompts (dict) — Dict of named prompts to evaluate (e.g., {"main": ChatPrompt(...)}). Single-prompt optimizations use a dict with one entry.

• experiment_config (dict[str, typing.Any] | None) — Optional experiment configuration.

• sampling_tag (str | None) — Optional sampling tag for deterministic subsampling per candidate.

evaluate_prompt(
    prompt: opik_optimizer.api_objects.chat_prompt.ChatPrompt | dict[str, opik_optimizer.api_objects.chat_prompt.ChatPrompt],
    dataset: Dataset,
    metric: MetricFunction,
    agent: opik_optimizer.agents.optimizable_agent.OptimizableAgent | None = None,
    n_threads: int | None = None,
    verbose: int = 1,
    dataset_item_ids: list[str] | None = None,
    experiment_config: dict | None = None,
    n_samples: int | float | str | None = None,
    n_samples_strategy: str | None = None,
    seed: int | None = None,
    return_evaluation_result: bool = False,
    allow_tool_use: bool | None = None,
    use_evaluate_on_dict_items: bool | None = None,
    sampling_tag: str | None = None
)

Parameters:

• prompt (opik_optimizer.api_objects.chat_prompt.ChatPrompt | dict[str, opik_optimizer.api_objects.chat_prompt.ChatPrompt]) —

• dataset (Dataset) —

• metric (MetricFunction) —

• agent (opik_optimizer.agents.optimizable_agent.OptimizableAgent | None) —

• n_threads (int | None) —

• verbose (int, default: 1) —

• dataset_item_ids (list[str] | None) —

• experiment_config (dict | None) —

• n_samples (int | float | str | None) —

• n_samples_strategy (str | None) —

• seed (int | None) —

• return_evaluation_result (bool, default: False) —

• allow_tool_use (bool | None) —

• use_evaluate_on_dict_items (bool | None) —

• sampling_tag (str | None) —

evaluate_with_result(
    context: OptimizationContext,
    prompts: dict,
    experiment_config: dict[str, typing.Any] | None = None,
    empty_score: float | None = None,
    n_samples: int | float | str | None = None,
    n_samples_strategy: str | None = None,
    sampling_tag: str | None = None
)

Parameters:

• context (OptimizationContext) —

• prompts (dict) —

• experiment_config (dict[str, typing.Any] | None) —

• empty_score (float | None) —

• n_samples (int | float | str | None) —

• n_samples_strategy (str | None) —

• sampling_tag (str | None) —

finish_candidate(
    context: OptimizationContext,
    candidate_handle: Any,
    score: float | None,
    metrics: dict[str, typing.Any] | None = None,
    extras: dict[str, typing.Any] | None = None,
    candidates: list[dict[str, typing.Any]] | None = None,
    dataset: str | None = None,
    dataset_split: str | None = None,
    trial_index: int | None = None,
    timestamp: str | None = None,
    round_handle: typing.Any | None = None
)

Parameters:

• context (OptimizationContext) —

• candidate_handle (Any) —

• score (float | None) —

• metrics (dict[str, typing.Any] | None) —

• extras (dict[str, typing.Any] | None) —

• candidates (list[dict[str, typing.Any]] | None) —

• dataset (str | None) —

• dataset_split (str | None) —

• trial_index (int | None) —

• timestamp (str | None) —

• round_handle (typing.Any | None) —

finish_round(
    round_handle: Any,
    context: opik_optimizer.core.state.OptimizationContext | None = None,
    best_score: float | None = None,
    best_candidate: typing.Any | None = None,
    best_prompt: typing.Any | None = None,
    stop_reason: str | None = None,
    extras: dict[str, typing.Any] | None = None,
    candidates: list[dict[str, typing.Any]] | None = None,
    timestamp: str | None = None,
    dataset_split: str | None = None,
    pareto_front: list[dict[str, typing.Any]] | None = None,
    selection_meta: dict[str, typing.Any] | None = None
)

Parameters:

• round_handle (Any) —

• context (opik_optimizer.core.state.OptimizationContext | None) —

• best_score (float | None) —

• best_candidate (typing.Any | None) —

• best_prompt (typing.Any | None) —

• stop_reason (str | None) —

• extras (dict[str, typing.Any] | None) —

• candidates (list[dict[str, typing.Any]] | None) —

• timestamp (str | None) —

• dataset_split (str | None) —

• pareto_front (list[dict[str, typing.Any]] | None) —

• selection_meta (dict[str, typing.Any] | None) —

get_config(
    context: OptimizationContext
)

Parameters:

• context (OptimizationContext) —

get_default_prompt(
    key: str
)

Parameters:

• key (str) — The prompt key to retrieve

get_history_entries()

get_history_rounds()

get_metadata(
    context: OptimizationContext
)

Parameters:

• context (OptimizationContext) —

get_optimizer_metadata()

get_prompt(
    key: str,
    fmt: Any
)

Parameters:

• key (str) — The prompt key to retrieve

• fmt (Any) —

list_prompts()

on_trial(
    context: OptimizationContext,
    prompts: dict,
    score: float,
    prev_best_score: float | None = None
)

Parameters:

• context (OptimizationContext) —

• prompts (dict) —

• score (float) —

• prev_best_score (float | None) —

optimize_mcp(
    args: Any,
    kwargs: Any
)

Parameters:

• args (Any) —

• kwargs (Any) —

optimize_prompt(
    prompt: opik_optimizer.api_objects.chat_prompt.ChatPrompt | dict[str, opik_optimizer.api_objects.chat_prompt.ChatPrompt],
    dataset: Dataset,
    metric: MetricFunction,
    agent: opik_optimizer.agents.optimizable_agent.OptimizableAgent | None = None,
    experiment_config: dict | None = None,
    n_samples: int | float | str | None = None,
    n_samples_minibatch: int | None = None,
    n_samples_strategy: str | None = None,
    auto_continue: bool = False,
    project_name: str | None = None,
    optimization_id: str | None = None,
    validation_dataset: opik.api_objects.dataset.dataset.Dataset | None = None,
    max_trials: int = 10,
    allow_tool_use: bool = True,
    optimize_prompts: bool | str | list[str] | None = 'system',
    optimize_tools: bool | dict[str, bool] | None = None,
    args: Any,
    kwargs: Any
)

Parameters:

• prompt (opik_optimizer.api_objects.chat_prompt.ChatPrompt | dict[str, opik_optimizer.api_objects.chat_prompt.ChatPrompt]) — The prompt to optimize (single ChatPrompt or dict of prompts)

• dataset (Dataset) — Opik dataset (training set - used for feedback/context) TODO/FIXME: This parameter will be deprecated in favor of dataset_training. For now, it serves as the training dataset parameter.

• metric (MetricFunction) — A metric function with signature (dataset_item, llm_output) -> float

• agent (opik_optimizer.agents.optimizable_agent.OptimizableAgent | None) — Optional agent for prompt execution (defaults to LiteLLMAgent)

• experiment_config (dict | None) — Optional configuration for the experiment

• n_samples (int | float | str | None) — Number of samples to use for evaluation

• n_samples_minibatch (int | None) — Optional number of samples for inner-loop minibatches

• n_samples_strategy (str | None) — Sampling strategy name (default "random_sorted")

• auto_continue (bool, default: False) — Whether to continue optimization automatically

• project_name (str | None) — Opik project name for logging traces (defaults to OPIK_PROJECT_NAME env or "Optimization")

• optimization_id (str | None) — Optional ID to use when creating the Opik optimization run

• validation_dataset (opik.api_objects.dataset.dataset.Dataset | None) — Optional validation dataset for ranking candidates

• max_trials (int, default: 10) — Maximum number of optimization trials

• allow_tool_use (bool, default: True) — Whether tools may be executed during evaluation (default True)

• optimize_prompts (bool | str | list[str] | None, default: system) — Which prompt roles to allow for optimization

• optimize_tools (bool | dict[str, bool] | None) — Optional tool optimization selector. Only supported by optimizers that explicitly document tool optimization support.

• args (Any) —

• kwargs (Any) —

post_baseline(
    context: OptimizationContext,
    score: float
)

Parameters:

• context (OptimizationContext) —

• score (float) —

post_optimize(
    context: OptimizationContext,
    result: OptimizationResult
)

Parameters:

• context (OptimizationContext) —

• result (OptimizationResult) —

post_round(
    round_handle: Any,
    context: opik_optimizer.core.state.OptimizationContext | None = None,
    best_score: float | None = None,
    best_candidate: typing.Any | None = None,
    best_prompt: typing.Any | None = None,
    stop_reason: str | None = None,
    extras: dict[str, typing.Any] | None = None,
    candidates: list[dict[str, typing.Any]] | None = None,
    timestamp: str | None = None,
    dataset_split: str | None = None,
    pareto_front: list[dict[str, typing.Any]] | None = None,
    selection_meta: dict[str, typing.Any] | None = None
)

Parameters:

• round_handle (Any) —

• context (opik_optimizer.core.state.OptimizationContext | None) —

• best_score (float | None) —

• best_candidate (typing.Any | None) —

• best_prompt (typing.Any | None) —

• stop_reason (str | None) —

• extras (dict[str, typing.Any] | None) —

• candidates (list[dict[str, typing.Any]] | None) —

• timestamp (str | None) —

• dataset_split (str | None) —

• pareto_front (list[dict[str, typing.Any]] | None) —

• selection_meta (dict[str, typing.Any] | None) —

post_trial(
    context: OptimizationContext,
    candidate_handle: Any,
    score: float | None,
    metrics: dict[str, typing.Any] | None = None,
    extras: dict[str, typing.Any] | None = None,
    candidates: list[dict[str, typing.Any]] | None = None,
    dataset: str | None = None,
    dataset_split: str | None = None,
    trial_index: int | None = None,
    timestamp: str | None = None,
    round_handle: typing.Any | None = None
)

Parameters:

• context (OptimizationContext) —

• candidate_handle (Any) —

• score (float | None) —

• metrics (dict[str, typing.Any] | None) —

• extras (dict[str, typing.Any] | None) —

• candidates (list[dict[str, typing.Any]] | None) —

• dataset (str | None) —

• dataset_split (str | None) —

• trial_index (int | None) —

• timestamp (str | None) —

• round_handle (typing.Any | None) —

pre_baseline(
    context: OptimizationContext
)

Parameters:

• context (OptimizationContext) —

pre_optimize(
    context: OptimizationContext
)

Parameters:

• context (OptimizationContext) —

pre_round(
    context: OptimizationContext,
    extras: Any
)

Parameters:

• context (OptimizationContext) —

• extras (Any) —

pre_trial(
    context: OptimizationContext,
    candidate: Any,
    round_handle: typing.Any | None = None
)

Parameters:

• context (OptimizationContext) —

• candidate (Any) —

• round_handle (typing.Any | None) —

record_candidate_entry(
    prompt_or_payload: Any,
    score: float | None = None,
    id: str | None = None,
    metrics: dict[str, typing.Any] | None = None,
    notes: str | None = None,
    extra: dict[str, typing.Any] | None = None,
    context: opik_optimizer.core.state.OptimizationContext | None = None
)

Parameters:

• prompt_or_payload (Any) —

• score (float | None) —

• id (str | None) —

• metrics (dict[str, typing.Any] | None) —

• notes (str | None) —

• extra (dict[str, typing.Any] | None) —

• context (opik_optimizer.core.state.OptimizationContext | None) —

run_optimization(
    context: OptimizationContext
)

Parameters:

• context (OptimizationContext) — The optimization context with prompts, dataset, metric, etc.

set_default_dataset_split(
    dataset_split: str | None
)

Parameters:

• dataset_split (str | None) —

set_pareto_front(
    pareto_front: list[dict[str, typing.Any]] | None
)

Parameters:

• pareto_front (list[dict[str, typing.Any]] | None) —

set_selection_meta(
    selection_meta: dict[str, typing.Any] | None
)

Parameters:

• selection_meta (dict[str, typing.Any] | None) —

start_candidate(
    context: OptimizationContext,
    candidate: Any,
    round_handle: typing.Any | None = None
)

Parameters:

• context (OptimizationContext) —

• candidate (Any) —

• round_handle (typing.Any | None) —

with_dataset_split(
    dataset_split: str | None
)

Parameters:

• dataset_split (str | None) —

HierarchicalReflectiveOptimizer(
    model: str = 'openai/gpt-5-nano',
    model_parameters: dict[str, typing.Any] | None = None,
    reasoning_model: str | None = None,
    reasoning_model_parameters: dict[str, typing.Any] | None = None,
    max_parallel_batches: int = 5,
    batch_size: int = 25,
    convergence_threshold: float = 0.01,
    n_threads: int = 12,
    verbose: int = 1,
    seed: int = 42,
    name: str | None = None,
    prompt_overrides: dict[str, str] | collections.abc.Callable[[opik_optimizer.utils.prompt_library.PromptLibrary], None] | None = None,
    skip_perfect_score: bool = True,
    perfect_score: float = 0.95
)

Parameters:

• model (str, default: openai/gpt-5-nano) —

• model_parameters (dict[str, typing.Any] | None) —

• reasoning_model (str | None) —

• reasoning_model_parameters (dict[str, typing.Any] | None) —

• max_parallel_batches (int, default: 5) —

• batch_size (int, default: 25) —

• convergence_threshold (float, default: 0.01) —

• n_threads (int, default: 12) —

• verbose (int, default: 1) —

• seed (int, default: 42) —

• name (str | None) —

• prompt_overrides (dict[str, str] | collections.abc.Callable[[opik_optimizer.utils.prompt_library.PromptLibrary], None] | None) —

• skip_perfect_score (bool, default: True) —

• perfect_score (float, default: 0.95) —

begin_round(
    context: OptimizationContext,
    extras: Any
)

Parameters:

• context (OptimizationContext) —

• extras (Any) —

cleanup()

evaluate(
    context: OptimizationContext,
    prompts: dict,
    experiment_config: dict[str, typing.Any] | None = None,
    sampling_tag: str | None = None
)

Parameters:

• context (OptimizationContext) — Optimization context for this run.

• prompts (dict) — Dict of named prompts to evaluate (e.g., {"main": ChatPrompt(...)}). Single-prompt optimizations use a dict with one entry.

• experiment_config (dict[str, typing.Any] | None) — Optional experiment configuration.

• sampling_tag (str | None) — Optional sampling tag for deterministic subsampling per candidate.

evaluate_prompt(
    prompt: opik_optimizer.api_objects.chat_prompt.ChatPrompt | dict[str, opik_optimizer.api_objects.chat_prompt.ChatPrompt],
    dataset: Dataset,
    metric: MetricFunction,
    agent: opik_optimizer.agents.optimizable_agent.OptimizableAgent | None = None,
    n_threads: int | None = None,
    verbose: int = 1,
    dataset_item_ids: list[str] | None = None,
    experiment_config: dict | None = None,
    n_samples: int | float | str | None = None,
    n_samples_strategy: str | None = None,
    seed: int | None = None,
    return_evaluation_result: bool = False,
    allow_tool_use: bool | None = None,
    use_evaluate_on_dict_items: bool | None = None,
    sampling_tag: str | None = None
)

Parameters:

• prompt (opik_optimizer.api_objects.chat_prompt.ChatPrompt | dict[str, opik_optimizer.api_objects.chat_prompt.ChatPrompt]) —

• dataset (Dataset) —

• metric (MetricFunction) —

• agent (opik_optimizer.agents.optimizable_agent.OptimizableAgent | None) —

• n_threads (int | None) —

• verbose (int, default: 1) —

• dataset_item_ids (list[str] | None) —

• experiment_config (dict | None) —

• n_samples (int | float | str | None) —

• n_samples_strategy (str | None) —

• seed (int | None) —

• return_evaluation_result (bool, default: False) —

• allow_tool_use (bool | None) —

• use_evaluate_on_dict_items (bool | None) —

• sampling_tag (str | None) —

evaluate_with_result(
    context: OptimizationContext,
    prompts: dict,
    experiment_config: dict[str, typing.Any] | None = None,
    empty_score: float | None = None,
    n_samples: int | float | str | None = None,
    n_samples_strategy: str | None = None,
    sampling_tag: str | None = None
)

Parameters:

• context (OptimizationContext) —

• prompts (dict) —

• experiment_config (dict[str, typing.Any] | None) —

• empty_score (float | None) —

• n_samples (int | float | str | None) —

• n_samples_strategy (str | None) —

• sampling_tag (str | None) —

finish_candidate(
    context: OptimizationContext,
    candidate_handle: Any,
    score: float | None,
    metrics: dict[str, typing.Any] | None = None,
    extras: dict[str, typing.Any] | None = None,
    candidates: list[dict[str, typing.Any]] | None = None,
    dataset: str | None = None,
    dataset_split: str | None = None,
    trial_index: int | None = None,
    timestamp: str | None = None,
    round_handle: typing.Any | None = None
)

Parameters:

• context (OptimizationContext) —

• candidate_handle (Any) —

• score (float | None) —

• metrics (dict[str, typing.Any] | None) —

• extras (dict[str, typing.Any] | None) —

• candidates (list[dict[str, typing.Any]] | None) —

• dataset (str | None) —

• dataset_split (str | None) —

• trial_index (int | None) —

• timestamp (str | None) —

• round_handle (typing.Any | None) —

finish_round(
    round_handle: Any,
    context: opik_optimizer.core.state.OptimizationContext | None = None,
    best_score: float | None = None,
    best_candidate: typing.Any | None = None,
    best_prompt: typing.Any | None = None,
    stop_reason: str | None = None,
    extras: dict[str, typing.Any] | None = None,
    candidates: list[dict[str, typing.Any]] | None = None,
    timestamp: str | None = None,
    dataset_split: str | None = None,
    pareto_front: list[dict[str, typing.Any]] | None = None,
    selection_meta: dict[str, typing.Any] | None = None
)

Parameters:

• round_handle (Any) —

• context (opik_optimizer.core.state.OptimizationContext | None) —

• best_score (float | None) —

• best_candidate (typing.Any | None) —

• best_prompt (typing.Any | None) —

• stop_reason (str | None) —

• extras (dict[str, typing.Any] | None) —

• candidates (list[dict[str, typing.Any]] | None) —

• timestamp (str | None) —

• dataset_split (str | None) —

• pareto_front (list[dict[str, typing.Any]] | None) —

• selection_meta (dict[str, typing.Any] | None) —

get_config(
    context: OptimizationContext
)

Parameters:

• context (OptimizationContext) —

get_default_prompt(
    key: str
)

Parameters:

• key (str) — The prompt key to retrieve

get_history_entries()

get_history_rounds()

get_metadata(
    context: OptimizationContext
)

Parameters:

• context (OptimizationContext) —

get_optimizer_metadata()

get_prompt(
    key: str,
    fmt: Any
)

Parameters:

• key (str) — The prompt key to retrieve

• fmt (Any) —

list_prompts()

on_trial(
    context: OptimizationContext,
    prompts: dict,
    score: float,
    prev_best_score: float | None = None
)

Parameters:

• context (OptimizationContext) —

• prompts (dict) —

• score (float) —

• prev_best_score (float | None) —

optimize_mcp(
    args: Any,
    kwargs: Any
)

Parameters:

• args (Any) —

• kwargs (Any) —

optimize_prompt(
    prompt: opik_optimizer.api_objects.chat_prompt.ChatPrompt | dict[str, opik_optimizer.api_objects.chat_prompt.ChatPrompt],
    dataset: Dataset,
    metric: MetricFunction,
    agent: opik_optimizer.agents.optimizable_agent.OptimizableAgent | None = None,
    experiment_config: dict | None = None,
    n_samples: int | float | str | None = None,
    n_samples_minibatch: int | None = None,
    n_samples_strategy: str | None = None,
    auto_continue: bool = False,
    project_name: str | None = None,
    optimization_id: str | None = None,
    validation_dataset: opik.api_objects.dataset.dataset.Dataset | None = None,
    max_trials: int = 10,
    allow_tool_use: bool = True,
    optimize_prompts: bool | str | list[str] | None = 'system',
    optimize_tools: bool | dict[str, bool] | None = None,
    args: Any,
    kwargs: Any
)

Parameters:

• prompt (opik_optimizer.api_objects.chat_prompt.ChatPrompt | dict[str, opik_optimizer.api_objects.chat_prompt.ChatPrompt]) — The prompt to optimize (single ChatPrompt or dict of prompts)

• dataset (Dataset) — Opik dataset (training set - used for feedback/context) TODO/FIXME: This parameter will be deprecated in favor of dataset_training. For now, it serves as the training dataset parameter.

• metric (MetricFunction) — A metric function with signature (dataset_item, llm_output) -> float

• agent (opik_optimizer.agents.optimizable_agent.OptimizableAgent | None) — Optional agent for prompt execution (defaults to LiteLLMAgent)

• experiment_config (dict | None) — Optional configuration for the experiment

• n_samples (int | float | str | None) — Number of samples to use for evaluation

• n_samples_minibatch (int | None) — Optional number of samples for inner-loop minibatches

• n_samples_strategy (str | None) — Sampling strategy name (default "random_sorted")

• auto_continue (bool, default: False) — Whether to continue optimization automatically

• project_name (str | None) — Opik project name for logging traces (defaults to OPIK_PROJECT_NAME env or "Optimization")

• optimization_id (str | None) — Optional ID to use when creating the Opik optimization run

• validation_dataset (opik.api_objects.dataset.dataset.Dataset | None) — Optional validation dataset for ranking candidates

• max_trials (int, default: 10) — Maximum number of optimization trials

• allow_tool_use (bool, default: True) — Whether tools may be executed during evaluation (default True)

• optimize_prompts (bool | str | list[str] | None, default: system) — Which prompt roles to allow for optimization

• optimize_tools (bool | dict[str, bool] | None) — Optional tool optimization selector. Only supported by optimizers that explicitly document tool optimization support.

• args (Any) —

• kwargs (Any) —

post_baseline(
    context: OptimizationContext,
    score: float
)

Parameters:

• context (OptimizationContext) —

• score (float) —

post_optimize(
    context: OptimizationContext,
    result: OptimizationResult
)

Parameters:

• context (OptimizationContext) —

• result (OptimizationResult) —

post_round(
    round_handle: Any,
    context: opik_optimizer.core.state.OptimizationContext | None = None,
    best_score: float | None = None,
    best_candidate: typing.Any | None = None,
    best_prompt: typing.Any | None = None,
    stop_reason: str | None = None,
    extras: dict[str, typing.Any] | None = None,
    candidates: list[dict[str, typing.Any]] | None = None,
    timestamp: str | None = None,
    dataset_split: str | None = None,
    pareto_front: list[dict[str, typing.Any]] | None = None,
    selection_meta: dict[str, typing.Any] | None = None
)

Parameters:

• round_handle (Any) —

• context (opik_optimizer.core.state.OptimizationContext | None) —

• best_score (float | None) —

• best_candidate (typing.Any | None) —

• best_prompt (typing.Any | None) —

• stop_reason (str | None) —

• extras (dict[str, typing.Any] | None) —

• candidates (list[dict[str, typing.Any]] | None) —

• timestamp (str | None) —

• dataset_split (str | None) —

• pareto_front (list[dict[str, typing.Any]] | None) —

• selection_meta (dict[str, typing.Any] | None) —

post_trial(
    context: OptimizationContext,
    candidate_handle: Any,
    score: float | None,
    metrics: dict[str, typing.Any] | None = None,
    extras: dict[str, typing.Any] | None = None,
    candidates: list[dict[str, typing.Any]] | None = None,
    dataset: str | None = None,
    dataset_split: str | None = None,
    trial_index: int | None = None,
    timestamp: str | None = None,
    round_handle: typing.Any | None = None
)

Parameters:

• context (OptimizationContext) —

• candidate_handle (Any) —

• score (float | None) —

• metrics (dict[str, typing.Any] | None) —

• extras (dict[str, typing.Any] | None) —

• candidates (list[dict[str, typing.Any]] | None) —

• dataset (str | None) —

• dataset_split (str | None) —

• trial_index (int | None) —

• timestamp (str | None) —

• round_handle (typing.Any | None) —

pre_baseline(
    context: OptimizationContext
)

Parameters:

• context (OptimizationContext) —

pre_optimize(
    context: OptimizationContext
)

Parameters:

• context (OptimizationContext) — The optimization context

pre_round(
    context: OptimizationContext,
    extras: Any
)

Parameters:

• context (OptimizationContext) —

• extras (Any) —

pre_trial(
    context: OptimizationContext,
    candidate: Any,
    round_handle: typing.Any | None = None
)

Parameters:

• context (OptimizationContext) —

• candidate (Any) —

• round_handle (typing.Any | None) —

record_candidate_entry(
    prompt_or_payload: Any,
    score: float | None = None,
    id: str | None = None,
    metrics: dict[str, typing.Any] | None = None,
    notes: str | None = None,
    extra: dict[str, typing.Any] | None = None,
    context: opik_optimizer.core.state.OptimizationContext | None = None
)

Parameters:

• prompt_or_payload (Any) —

• score (float | None) —

• id (str | None) —

• metrics (dict[str, typing.Any] | None) —

• notes (str | None) —

• extra (dict[str, typing.Any] | None) —

• context (opik_optimizer.core.state.OptimizationContext | None) —

run_optimization(
    context: OptimizationContext
)

Parameters:

• context (OptimizationContext) — The optimization context with prompts, dataset, metric, etc.

set_default_dataset_split(
    dataset_split: str | None
)

Parameters:

• dataset_split (str | None) —

set_pareto_front(
    pareto_front: list[dict[str, typing.Any]] | None
)

Parameters:

• pareto_front (list[dict[str, typing.Any]] | None) —

set_selection_meta(
    selection_meta: dict[str, typing.Any] | None
)

Parameters:

• selection_meta (dict[str, typing.Any] | None) —

start_candidate(
    context: OptimizationContext,
    candidate: Any,
    round_handle: typing.Any | None = None
)

Parameters:

• context (OptimizationContext) —

• candidate (Any) —

• round_handle (typing.Any | None) —

with_dataset_split(
    dataset_split: str | None
)

Parameters:

• dataset_split (str | None) —

ParameterOptimizer(
    model: str = 'openai/gpt-5-nano',
    model_parameters: dict[str, typing.Any] | None = None,
    default_n_trials: int = 20,
    local_search_ratio: float = 0.3,
    local_search_scale: float = 0.2,
    n_threads: int = 12,
    verbose: int = 1,
    seed: int = 42,
    name: str | None = None,
    skip_perfect_score: bool = True,
    perfect_score: float = 0.95
)

Parameters:

• model (str, default: openai/gpt-5-nano) —

• model_parameters (dict[str, typing.Any] | None) —

• default_n_trials (int, default: 20) —

• local_search_ratio (float, default: 0.3) —

• local_search_scale (float, default: 0.2) —

• n_threads (int, default: 12) —

• verbose (int, default: 1) —

• seed (int, default: 42) —

• name (str | None) —

• skip_perfect_score (bool, default: True) —

• perfect_score (float, default: 0.95) —

begin_round(
    context: OptimizationContext,
    extras: Any
)

Parameters:

• context (OptimizationContext) —

• extras (Any) —

cleanup()

evaluate(
    context: OptimizationContext,
    prompts: dict,
    experiment_config: dict[str, typing.Any] | None = None,
    sampling_tag: str | None = None
)

Parameters:

• context (OptimizationContext) — Optimization context for this run.

• prompts (dict) — Dict of named prompts to evaluate (e.g., {"main": ChatPrompt(...)}). Single-prompt optimizations use a dict with one entry.

• experiment_config (dict[str, typing.Any] | None) — Optional experiment configuration.

• sampling_tag (str | None) — Optional sampling tag for deterministic subsampling per candidate.

evaluate_prompt(
    prompt: opik_optimizer.api_objects.chat_prompt.ChatPrompt | dict[str, opik_optimizer.api_objects.chat_prompt.ChatPrompt],
    dataset: Dataset,
    metric: MetricFunction,
    agent: opik_optimizer.agents.optimizable_agent.OptimizableAgent | None = None,
    n_threads: int | None = None,
    verbose: int = 1,
    dataset_item_ids: list[str] | None = None,
    experiment_config: dict | None = None,
    n_samples: int | float | str | None = None,
    n_samples_strategy: str | None = None,
    seed: int | None = None,
    return_evaluation_result: bool = False,
    allow_tool_use: bool | None = None,
    use_evaluate_on_dict_items: bool | None = None,
    sampling_tag: str | None = None
)

Parameters:

• prompt (opik_optimizer.api_objects.chat_prompt.ChatPrompt | dict[str, opik_optimizer.api_objects.chat_prompt.ChatPrompt]) —

• dataset (Dataset) —

• metric (MetricFunction) —

• agent (opik_optimizer.agents.optimizable_agent.OptimizableAgent | None) —

• n_threads (int | None) —

• verbose (int, default: 1) —

• dataset_item_ids (list[str] | None) —

• experiment_config (dict | None) —

• n_samples (int | float | str | None) —

• n_samples_strategy (str | None) —

• seed (int | None) —

• return_evaluation_result (bool, default: False) —

• allow_tool_use (bool | None) —

• use_evaluate_on_dict_items (bool | None) —

• sampling_tag (str | None) —

evaluate_with_result(
    context: OptimizationContext,
    prompts: dict,
    experiment_config: dict[str, typing.Any] | None = None,
    empty_score: float | None = None,
    n_samples: int | float | str | None = None,
    n_samples_strategy: str | None = None,
    sampling_tag: str | None = None
)

Parameters:

• context (OptimizationContext) —

• prompts (dict) —

• experiment_config (dict[str, typing.Any] | None) —

• empty_score (float | None) —

• n_samples (int | float | str | None) —

• n_samples_strategy (str | None) —

• sampling_tag (str | None) —

finish_candidate(
    context: OptimizationContext,
    candidate_handle: Any,
    score: float | None,
    metrics: dict[str, typing.Any] | None = None,
    extras: dict[str, typing.Any] | None = None,
    candidates: list[dict[str, typing.Any]] | None = None,
    dataset: str | None = None,
    dataset_split: str | None = None,
    trial_index: int | None = None,
    timestamp: str | None = None,
    round_handle: typing.Any | None = None
)

Parameters:

• context (OptimizationContext) —

• candidate_handle (Any) —

• score (float | None) —

• metrics (dict[str, typing.Any] | None) —

• extras (dict[str, typing.Any] | None) —

• candidates (list[dict[str, typing.Any]] | None) —

• dataset (str | None) —

• dataset_split (str | None) —

• trial_index (int | None) —

• timestamp (str | None) —

• round_handle (typing.Any | None) —

finish_round(
    round_handle: Any,
    context: opik_optimizer.core.state.OptimizationContext | None = None,
    best_score: float | None = None,
    best_candidate: typing.Any | None = None,
    best_prompt: typing.Any | None = None,
    stop_reason: str | None = None,
    extras: dict[str, typing.Any] | None = None,
    candidates: list[dict[str, typing.Any]] | None = None,
    timestamp: str | None = None,
    dataset_split: str | None = None,
    pareto_front: list[dict[str, typing.Any]] | None = None,
    selection_meta: dict[str, typing.Any] | None = None
)

Parameters:

• round_handle (Any) —

• context (opik_optimizer.core.state.OptimizationContext | None) —

• best_score (float | None) —

• best_candidate (typing.Any | None) —

• best_prompt (typing.Any | None) —

• stop_reason (str | None) —

• extras (dict[str, typing.Any] | None) —

• candidates (list[dict[str, typing.Any]] | None) —

• timestamp (str | None) —

• dataset_split (str | None) —

• pareto_front (list[dict[str, typing.Any]] | None) —

• selection_meta (dict[str, typing.Any] | None) —

get_config(
    context: OptimizationContext
)

Parameters:

• context (OptimizationContext) —

get_default_prompt(
    key: str
)

Parameters:

• key (str) — The prompt key to retrieve

get_history_entries()

get_history_rounds()

get_metadata(
    context: OptimizationContext
)

Parameters:

• context (OptimizationContext) —

get_optimizer_metadata()

get_prompt(
    key: str,
    fmt: Any
)

Parameters:

• key (str) — The prompt key to retrieve

• fmt (Any) —

list_prompts()

on_trial(
    context: OptimizationContext,
    prompts: dict,
    score: float,
    prev_best_score: float | None = None
)

Parameters:

• context (OptimizationContext) —

• prompts (dict) —

• score (float) —

• prev_best_score (float | None) —

optimize_mcp(
    args: Any,
    kwargs: Any
)

Parameters:

• args (Any) —

• kwargs (Any) —

optimize_parameter(
    prompt: opik_optimizer.api_objects.chat_prompt.ChatPrompt | dict[str, opik_optimizer.api_objects.chat_prompt.ChatPrompt],
    dataset: Dataset,
    metric: MetricFunction,
    parameter_space: opik_optimizer.algorithms.parameter_optimizer.ops.search_ops.ParameterSearchSpace | collections.abc.Mapping[str, typing.Any],
    validation_dataset: opik.api_objects.dataset.dataset.Dataset | None = None,
    experiment_config: dict | None = None,
    max_trials: int | None = None,
    n_samples: int | float | str | None = None,
    n_samples_minibatch: int | None = None,
    n_samples_strategy: str | None = None,
    agent: opik_optimizer.agents.optimizable_agent.OptimizableAgent | None = None,
    project_name: str = 'Optimization',
    sampler: optuna.samplers._base.BaseSampler | None = None,
    callbacks: list[collections.abc.Callable[[optuna.study.study.Study, optuna.trial._frozen.FrozenTrial], None]] | None = None,
    timeout: float | None = None,
    local_trials: int | None = None,
    local_search_scale: float | None = None,
    optimization_id: str | None = None
)

Parameters:

• prompt (opik_optimizer.api_objects.chat_prompt.ChatPrompt | dict[str, opik_optimizer.api_objects.chat_prompt.ChatPrompt]) — The prompt or dict of prompts to evaluate with tuned parameters. When a dict is provided, parameters are optimized independently for each prompt.

• dataset (Dataset) — Dataset providing evaluation examples

• metric (MetricFunction) — Objective function to maximize

• parameter_space (opik_optimizer.algorithms.parameter_optimizer.ops.search_ops.ParameterSearchSpace | collections.abc.Mapping[str, typing.Any]) — Definition of the search space for tunable parameters. For multi-prompt, params without a prefix are expanded per prompt. Params already prefixed (e.g., 'analyze.temperature') are kept as-is.

• validation_dataset (opik.api_objects.dataset.dataset.Dataset | None) — Optional validation dataset. Note: Due to the internal implementation of ParameterOptimizer, this parameter is currently not fully utilized and we recommend not using it for this optimizer.

• experiment_config (dict | None) — Optional experiment metadata

• max_trials (int | None) — Total number of trials (if None, uses default_n_trials)

• n_samples (int | float | str | None) — Number of dataset samples to evaluate per trial (None for all)

• n_samples_minibatch (int | None) — Optional number of samples for inner-loop minibatches

• n_samples_strategy (str | None) — Sampling strategy name (default "random_sorted")

• agent (opik_optimizer.agents.optimizable_agent.OptimizableAgent | None) — Optional custom agent instance to execute evaluations

• project_name (str, default: Optimization) — Opik project name for logging traces (default: "Optimization")

• sampler (optuna.samplers._base.BaseSampler | None) — Optuna sampler to use (default: TPESampler with seed)

• callbacks (list[collections.abc.Callable[[optuna.study.study.Study, optuna.trial._frozen.FrozenTrial], None]] | None) — List of callback functions for Optuna study

• timeout (float | None) — Maximum time in seconds for optimization

• local_trials (int | None) — Number of trials for local search (overrides local_search_ratio)

• local_search_scale (float | None) — Scale factor for local search narrowing (0.0-1.0)

• optimization_id (str | None) — Optional ID to use when creating the Opik optimization run; when provided it must be a valid UUIDv7 string.

post_baseline(
    context: OptimizationContext,
    score: float
)

Parameters:

• context (OptimizationContext) —

• score (float) —

post_optimize(
    context: OptimizationContext,
    result: OptimizationResult
)

Parameters:

• context (OptimizationContext) —

• result (OptimizationResult) —

post_round(
    round_handle: Any,
    context: opik_optimizer.core.state.OptimizationContext | None = None,
    best_score: float | None = None,
    best_candidate: typing.Any | None = None,
    best_prompt: typing.Any | None = None,
    stop_reason: str | None = None,
    extras: dict[str, typing.Any] | None = None,
    candidates: list[dict[str, typing.Any]] | None = None,
    timestamp: str | None = None,
    dataset_split: str | None = None,
    pareto_front: list[dict[str, typing.Any]] | None = None,
    selection_meta: dict[str, typing.Any] | None = None
)

Parameters:

• round_handle (Any) —

• context (opik_optimizer.core.state.OptimizationContext | None) —

• best_score (float | None) —

• best_candidate (typing.Any | None) —

• best_prompt (typing.Any | None) —

• stop_reason (str | None) —

• extras (dict[str, typing.Any] | None) —

• candidates (list[dict[str, typing.Any]] | None) —

• timestamp (str | None) —

• dataset_split (str | None) —

• pareto_front (list[dict[str, typing.Any]] | None) —

• selection_meta (dict[str, typing.Any] | None) —

post_trial(
    context: OptimizationContext,
    candidate_handle: Any,
    score: float | None,
    metrics: dict[str, typing.Any] | None = None,
    extras: dict[str, typing.Any] | None = None,
    candidates: list[dict[str, typing.Any]] | None = None,
    dataset: str | None = None,
    dataset_split: str | None = None,
    trial_index: int | None = None,
    timestamp: str | None = None,
    round_handle: typing.Any | None = None
)

Parameters:

• context (OptimizationContext) —

• candidate_handle (Any) —

• score (float | None) —

• metrics (dict[str, typing.Any] | None) —

• extras (dict[str, typing.Any] | None) —

• candidates (list[dict[str, typing.Any]] | None) —

• dataset (str | None) —

• dataset_split (str | None) —

• trial_index (int | None) —

• timestamp (str | None) —

• round_handle (typing.Any | None) —

pre_baseline(
    context: OptimizationContext
)

Parameters:

• context (OptimizationContext) —

pre_optimize(
    context: OptimizationContext
)

Parameters:

• context (OptimizationContext) — The optimization context

pre_round(
    context: OptimizationContext,
    extras: Any
)

Parameters:

• context (OptimizationContext) —

• extras (Any) —

pre_trial(
    context: OptimizationContext,
    candidate: Any,
    round_handle: typing.Any | None = None
)

Parameters:

• context (OptimizationContext) —

• candidate (Any) —

• round_handle (typing.Any | None) —

record_candidate_entry(
    prompt_or_payload: Any,
    score: float | None = None,
    id: str | None = None,
    metrics: dict[str, typing.Any] | None = None,
    notes: str | None = None,
    extra: dict[str, typing.Any] | None = None,
    context: opik_optimizer.core.state.OptimizationContext | None = None
)

Parameters:

• prompt_or_payload (Any) —

• score (float | None) —

• id (str | None) —

• metrics (dict[str, typing.Any] | None) —

• notes (str | None) —

• extra (dict[str, typing.Any] | None) —

• context (opik_optimizer.core.state.OptimizationContext | None) —

set_default_dataset_split(
    dataset_split: str | None
)

Parameters:

• dataset_split (str | None) —

set_pareto_front(
    pareto_front: list[dict[str, typing.Any]] | None
)

Parameters:

• pareto_front (list[dict[str, typing.Any]] | None) —

set_selection_meta(
    selection_meta: dict[str, typing.Any] | None
)

Parameters:

• selection_meta (dict[str, typing.Any] | None) —

start_candidate(
    context: OptimizationContext,
    candidate: Any,
    round_handle: typing.Any | None = None
)

Parameters:

• context (OptimizationContext) —

• candidate (Any) —

• round_handle (typing.Any | None) —

with_dataset_split(
    dataset_split: str | None
)

Parameters:

• dataset_split (str | None) —

ParameterSearchSpace(
    parameters: list[opik_optimizer.algorithms.parameter_optimizer.ops.search_ops.ParameterSpec] = PydanticUndefined
)

Parameters:

• parameters (list[opik_optimizer.algorithms.parameter_optimizer.ops.search_ops.ParameterSpec], default: PydanticUndefined) —

ParameterSpec(
    name: <class 'str'>,
    description: str | None = None,
    distribution: <enum 'ParameterType'>,
    low: float | None = None,
    high: float | None = None,
    step: float | None = None,
    scale: Literal['linear', 'log'] = 'linear',
    choices: list[Any] | None = None,
    target: str | collections.abc.Sequence[str] | None = None,
    default: Any | None = None
)

Parameters:

• name (<class 'str'>, default: PydanticUndefined) —

• description (str | None) —

• distribution (<enum 'ParameterType'>, default: PydanticUndefined) —

• low (float | None) —

• high (float | None) —

• step (float | None) —

• scale (Literal['linear', 'log'], default: linear) —

• choices (list[Any] | None) —

• target (str | collections.abc.Sequence[str] | None) —

• default (Any | None) —

ParameterType(
    args: Any,
    kwds: Any
)

Parameters:

• args (Any) —

• kwds (Any) —

BaseOptimizer(
    model: str,
    verbose: int = 1,
    seed: int = 42,
    model_parameters: dict[str, typing.Any] | None = None,
    reasoning_model: str | None = None,
    reasoning_model_parameters: dict[str, typing.Any] | None = None,
    name: str | None = None,
    skip_perfect_score: bool = True,
    perfect_score: float = 0.95,
    prompt_overrides: dict[str, str] | collections.abc.Callable[[opik_optimizer.utils.prompt_library.PromptLibrary], None] | None = None,
    display: opik_optimizer.utils.display.run.RunDisplay | None = None
)

Parameters:

• model (str) —

• verbose (int, default: 1) —

• seed (int, default: 42) —

• model_parameters (dict[str, typing.Any] | None) —

• reasoning_model (str | None) —

• reasoning_model_parameters (dict[str, typing.Any] | None) —

• name (str | None) —

• skip_perfect_score (bool, default: True) —

• perfect_score (float, default: 0.95) —

• prompt_overrides (dict[str, str] | collections.abc.Callable[[opik_optimizer.utils.prompt_library.PromptLibrary], None] | None) —

• display (opik_optimizer.utils.display.run.RunDisplay | None) —

begin_round(
    context: OptimizationContext,
    extras: Any
)

Parameters:

• context (OptimizationContext) —

• extras (Any) —

cleanup()

evaluate(
    context: OptimizationContext,
    prompts: dict,
    experiment_config: dict[str, typing.Any] | None = None,
    sampling_tag: str | None = None
)

Parameters:

• context (OptimizationContext) — Optimization context for this run.

• prompts (dict) — Dict of named prompts to evaluate (e.g., {"main": ChatPrompt(...)}). Single-prompt optimizations use a dict with one entry.

• experiment_config (dict[str, typing.Any] | None) — Optional experiment configuration.

• sampling_tag (str | None) — Optional sampling tag for deterministic subsampling per candidate.

evaluate_prompt(
    prompt: opik_optimizer.api_objects.chat_prompt.ChatPrompt | dict[str, opik_optimizer.api_objects.chat_prompt.ChatPrompt],
    dataset: Dataset,
    metric: MetricFunction,
    agent: opik_optimizer.agents.optimizable_agent.OptimizableAgent | None = None,
    n_threads: int | None = None,
    verbose: int = 1,
    dataset_item_ids: list[str] | None = None,
    experiment_config: dict | None = None,
    n_samples: int | float | str | None = None,
    n_samples_strategy: str | None = None,
    seed: int | None = None,
    return_evaluation_result: bool = False,
    allow_tool_use: bool | None = None,
    use_evaluate_on_dict_items: bool | None = None,
    sampling_tag: str | None = None
)

Parameters:

• prompt (opik_optimizer.api_objects.chat_prompt.ChatPrompt | dict[str, opik_optimizer.api_objects.chat_prompt.ChatPrompt]) —

• dataset (Dataset) —

• metric (MetricFunction) —

• agent (opik_optimizer.agents.optimizable_agent.OptimizableAgent | None) —

• n_threads (int | None) —

• verbose (int, default: 1) —

• dataset_item_ids (list[str] | None) —

• experiment_config (dict | None) —

• n_samples (int | float | str | None) —

• n_samples_strategy (str | None) —

• seed (int | None) —

• return_evaluation_result (bool, default: False) —

• allow_tool_use (bool | None) —

• use_evaluate_on_dict_items (bool | None) —

• sampling_tag (str | None) —

evaluate_with_result(
    context: OptimizationContext,
    prompts: dict,
    experiment_config: dict[str, typing.Any] | None = None,
    empty_score: float | None = None,
    n_samples: int | float | str | None = None,
    n_samples_strategy: str | None = None,
    sampling_tag: str | None = None
)

Parameters:

• context (OptimizationContext) —

• prompts (dict) —

• experiment_config (dict[str, typing.Any] | None) —

• empty_score (float | None) —

• n_samples (int | float | str | None) —

• n_samples_strategy (str | None) —

• sampling_tag (str | None) —

finish_candidate(
    context: OptimizationContext,
    candidate_handle: Any,
    score: float | None,
    metrics: dict[str, typing.Any] | None = None,
    extras: dict[str, typing.Any] | None = None,
    candidates: list[dict[str, typing.Any]] | None = None,
    dataset: str | None = None,
    dataset_split: str | None = None,
    trial_index: int | None = None,
    timestamp: str | None = None,
    round_handle: typing.Any | None = None
)

Parameters:

• context (OptimizationContext) —

• candidate_handle (Any) —

• score (float | None) —

• metrics (dict[str, typing.Any] | None) —

• extras (dict[str, typing.Any] | None) —

• candidates (list[dict[str, typing.Any]] | None) —

• dataset (str | None) —

• dataset_split (str | None) —

• trial_index (int | None) —

• timestamp (str | None) —

• round_handle (typing.Any | None) —

finish_round(
    round_handle: Any,
    context: opik_optimizer.core.state.OptimizationContext | None = None,
    best_score: float | None = None,
    best_candidate: typing.Any | None = None,
    best_prompt: typing.Any | None = None,
    stop_reason: str | None = None,
    extras: dict[str, typing.Any] | None = None,
    candidates: list[dict[str, typing.Any]] | None = None,
    timestamp: str | None = None,
    dataset_split: str | None = None,
    pareto_front: list[dict[str, typing.Any]] | None = None,
    selection_meta: dict[str, typing.Any] | None = None
)

Parameters:

• round_handle (Any) —

• context (opik_optimizer.core.state.OptimizationContext | None) —

• best_score (float | None) —

• best_candidate (typing.Any | None) —

• best_prompt (typing.Any | None) —

• stop_reason (str | None) —

• extras (dict[str, typing.Any] | None) —

• candidates (list[dict[str, typing.Any]] | None) —

• timestamp (str | None) —

• dataset_split (str | None) —

• pareto_front (list[dict[str, typing.Any]] | None) —

• selection_meta (dict[str, typing.Any] | None) —

get_config(
    context: OptimizationContext
)

Parameters:

• context (OptimizationContext) —

get_default_prompt(
    key: str
)

Parameters:

• key (str) — The prompt key to retrieve

get_history_entries()

get_history_rounds()

get_metadata(
    context: OptimizationContext
)

Parameters:

• context (OptimizationContext) —

get_prompt(
    key: str,
    fmt: Any
)

Parameters:

• key (str) — The prompt key to retrieve

• fmt (Any) —

list_prompts()

on_trial(
    context: OptimizationContext,
    prompts: dict,
    score: float,
    prev_best_score: float | None = None
)

Parameters:

• context (OptimizationContext) —

• prompts (dict) —

• score (float) —

• prev_best_score (float | None) —

optimize_mcp(
    args: Any,
    kwargs: Any
)

Parameters:

• args (Any) —

• kwargs (Any) —

optimize_prompt(
    prompt: opik_optimizer.api_objects.chat_prompt.ChatPrompt | dict[str, opik_optimizer.api_objects.chat_prompt.ChatPrompt],
    dataset: Dataset,
    metric: MetricFunction,
    agent: opik_optimizer.agents.optimizable_agent.OptimizableAgent | None = None,
    experiment_config: dict | None = None,
    n_samples: int | float | str | None = None,
    n_samples_minibatch: int | None = None,
    n_samples_strategy: str | None = None,
    auto_continue: bool = False,
    project_name: str | None = None,
    optimization_id: str | None = None,
    validation_dataset: opik.api_objects.dataset.dataset.Dataset | None = None,
    max_trials: int = 10,
    allow_tool_use: bool = True,
    optimize_prompts: bool | str | list[str] | None = 'system',
    optimize_tools: bool | dict[str, bool] | None = None,
    args: Any,
    kwargs: Any
)

Parameters:

• prompt (opik_optimizer.api_objects.chat_prompt.ChatPrompt | dict[str, opik_optimizer.api_objects.chat_prompt.ChatPrompt]) — The prompt to optimize (single ChatPrompt or dict of prompts)

• dataset (Dataset) — Opik dataset (training set - used for feedback/context) TODO/FIXME: This parameter will be deprecated in favor of dataset_training. For now, it serves as the training dataset parameter.

• metric (MetricFunction) — A metric function with signature (dataset_item, llm_output) -> float

• agent (opik_optimizer.agents.optimizable_agent.OptimizableAgent | None) — Optional agent for prompt execution (defaults to LiteLLMAgent)

• experiment_config (dict | None) — Optional configuration for the experiment

• n_samples (int | float | str | None) — Number of samples to use for evaluation

• n_samples_minibatch (int | None) — Optional number of samples for inner-loop minibatches

• n_samples_strategy (str | None) — Sampling strategy name (default "random_sorted")

• auto_continue (bool, default: False) — Whether to continue optimization automatically

• project_name (str | None) — Opik project name for logging traces (defaults to OPIK_PROJECT_NAME env or "Optimization")

• optimization_id (str | None) — Optional ID to use when creating the Opik optimization run

• validation_dataset (opik.api_objects.dataset.dataset.Dataset | None) — Optional validation dataset for ranking candidates

• max_trials (int, default: 10) — Maximum number of optimization trials

• allow_tool_use (bool, default: True) — Whether tools may be executed during evaluation (default True)

• optimize_prompts (bool | str | list[str] | None, default: system) — Which prompt roles to allow for optimization

• optimize_tools (bool | dict[str, bool] | None) — Optional tool optimization selector. Only supported by optimizers that explicitly document tool optimization support.

• args (Any) —

• kwargs (Any) —

post_baseline(
    context: OptimizationContext,
    score: float
)

Parameters:

• context (OptimizationContext) —

• score (float) —

post_optimize(
    context: OptimizationContext,
    result: OptimizationResult
)

Parameters:

• context (OptimizationContext) —

• result (OptimizationResult) —

post_round(
    round_handle: Any,
    context: opik_optimizer.core.state.OptimizationContext | None = None,
    best_score: float | None = None,
    best_candidate: typing.Any | None = None,
    best_prompt: typing.Any | None = None,
    stop_reason: str | None = None,
    extras: dict[str, typing.Any] | None = None,
    candidates: list[dict[str, typing.Any]] | None = None,
    timestamp: str | None = None,
    dataset_split: str | None = None,
    pareto_front: list[dict[str, typing.Any]] | None = None,
    selection_meta: dict[str, typing.Any] | None = None
)

Parameters:

• round_handle (Any) —

• context (opik_optimizer.core.state.OptimizationContext | None) —

• best_score (float | None) —

• best_candidate (typing.Any | None) —

• best_prompt (typing.Any | None) —

• stop_reason (str | None) —

• extras (dict[str, typing.Any] | None) —

• candidates (list[dict[str, typing.Any]] | None) —

• timestamp (str | None) —

• dataset_split (str | None) —

• pareto_front (list[dict[str, typing.Any]] | None) —

• selection_meta (dict[str, typing.Any] | None) —

post_trial(
    context: OptimizationContext,
    candidate_handle: Any,
    score: float | None,
    metrics: dict[str, typing.Any] | None = None,
    extras: dict[str, typing.Any] | None = None,
    candidates: list[dict[str, typing.Any]] | None = None,
    dataset: str | None = None,
    dataset_split: str | None = None,
    trial_index: int | None = None,
    timestamp: str | None = None,
    round_handle: typing.Any | None = None
)

Parameters:

• context (OptimizationContext) —

• candidate_handle (Any) —

• score (float | None) —

• metrics (dict[str, typing.Any] | None) —

• extras (dict[str, typing.Any] | None) —

• candidates (list[dict[str, typing.Any]] | None) —

• dataset (str | None) —

• dataset_split (str | None) —

• trial_index (int | None) —

• timestamp (str | None) —

• round_handle (typing.Any | None) —

pre_baseline(
    context: OptimizationContext
)

Parameters:

• context (OptimizationContext) —

pre_optimize(
    context: OptimizationContext
)

Parameters:

• context (OptimizationContext) — The optimization context

pre_round(
    context: OptimizationContext,
    extras: Any
)

Parameters:

• context (OptimizationContext) —

• extras (Any) —

pre_trial(
    context: OptimizationContext,
    candidate: Any,
    round_handle: typing.Any | None = None
)

Parameters:

• context (OptimizationContext) —

• candidate (Any) —

• round_handle (typing.Any | None) —

record_candidate_entry(
    prompt_or_payload: Any,
    score: float | None = None,
    id: str | None = None,
    metrics: dict[str, typing.Any] | None = None,
    notes: str | None = None,
    extra: dict[str, typing.Any] | None = None,
    context: opik_optimizer.core.state.OptimizationContext | None = None
)

Parameters:

• prompt_or_payload (Any) —

• score (float | None) —

• id (str | None) —

• metrics (dict[str, typing.Any] | None) —

• notes (str | None) —

• extra (dict[str, typing.Any] | None) —

• context (opik_optimizer.core.state.OptimizationContext | None) —

run_optimization(
    context: OptimizationContext
)

Parameters:

• context (OptimizationContext) — The optimization context with prompts, dataset, metric, etc.

set_default_dataset_split(
    dataset_split: str | None
)

Parameters:

• dataset_split (str | None) —

set_pareto_front(
    pareto_front: list[dict[str, typing.Any]] | None
)

Parameters:

• pareto_front (list[dict[str, typing.Any]] | None) —

set_selection_meta(
    selection_meta: dict[str, typing.Any] | None
)

Parameters:

• selection_meta (dict[str, typing.Any] | None) —

start_candidate(
    context: OptimizationContext,
    candidate: Any,
    round_handle: typing.Any | None = None
)

Parameters:

• context (OptimizationContext) —

• candidate (Any) —

• round_handle (typing.Any | None) —

with_dataset_split(
    dataset_split: str | None
)

Parameters:

• dataset_split (str | None) —

ChatPrompt(
    name: str = 'chat-prompt',
    system: str | None = None,
    user: str | None = None,
    messages: list[dict[str, typing.Any]] | None = None,
    tools: list[dict[str, typing.Any]] | collections.abc.Mapping[str, typing.Any] | None = None,
    function_map: collections.abc.Mapping[str, collections.abc.Callable[..., typing.Any]] | None = None,
    model: str = 'openai/gpt-5-nano',
    model_parameters: dict[str, typing.Any] | None = None,
    model_kwargs: dict[str, typing.Any] | None = None,
    kwargs: Any
)

Parameters:

• name (str, default: chat-prompt) —

• system (str | None) — the system prompt

• user (str | None) —

• messages (list[dict[str, typing.Any]] | None) — a list of dictionaries with role/content, with a content containing {input-dataset-field}

• tools (list[dict[str, typing.Any]] | collections.abc.Mapping[str, typing.Any] | None) —

• function_map (collections.abc.Mapping[str, collections.abc.Callable[..., typing.Any]] | None) —

• model (str, default: openai/gpt-5-nano) —

• model_parameters (dict[str, typing.Any] | None) —

• model_kwargs (dict[str, typing.Any] | None) —

• kwargs (Any) —

copy()

get_messages(
    dataset_item: dict[str, typing.Any] | None = None
)

Parameters:

• dataset_item (dict[str, typing.Any] | None) —

replace_in_messages(
    messages: list,
    label: str,
    value: str
)

Parameters:

• messages (list) —

• label (str) —

• value (str) —

set_messages(
    messages: list
)

Parameters:

• messages (list) —

to_dict()

AlgorithmResult(
    best_prompts: dict,
    best_score: float,
    history: Sequence = <factory>,
    metadata: dict = <factory>
)

Parameters:

• best_prompts (dict) —

• best_score (float) —

• history (Sequence, default: ) —

• metadata (dict, default: ) —

OptimizationResult(
    schema_version: <class 'str'> = 'v1',
    details_version: <class 'str'> = 'v1',
    optimizer: <class 'str'> = 'Optimizer',
    prompt: opik_optimizer.api_objects.chat_prompt.ChatPrompt | dict[str, opik_optimizer.api_objects.chat_prompt.ChatPrompt],
    score: <class 'float'>,
    metric_name: <class 'str'>,
    optimization_id: str | None = None,
    dataset_id: str | None = None,
    initial_prompt: opik_optimizer.api_objects.chat_prompt.ChatPrompt | dict[str, opik_optimizer.api_objects.chat_prompt.ChatPrompt] | None = None,
    initial_score: float | None = None,
    details: dict[str, Any] = PydanticUndefined,
    history: list[dict[str, Any]] = [],
    llm_calls: int | None = None,
    llm_calls_tools: int | None = None,
    llm_cost_total: float | None = None,
    llm_token_usage_total: dict[str, int] | None = None
)

Parameters:

• schema_version (<class 'str'>, default: v1) —

• details_version (<class 'str'>, default: v1) —

• optimizer (<class 'str'>, default: Optimizer) —

• prompt (opik_optimizer.api_objects.chat_prompt.ChatPrompt | dict[str, opik_optimizer.api_objects.chat_prompt.ChatPrompt], default: PydanticUndefined) —

• score (<class 'float'>, default: PydanticUndefined) —

• metric_name (<class 'str'>, default: PydanticUndefined) —

• optimization_id (str | None) —

• dataset_id (str | None) —

• initial_prompt (opik_optimizer.api_objects.chat_prompt.ChatPrompt | dict[str, opik_optimizer.api_objects.chat_prompt.ChatPrompt] | None) —

• initial_score (float | None) —

• details (dict[str, Any], default: PydanticUndefined) —

• history (list[dict[str, Any]], default: []) —

• llm_calls (int | None) —

• llm_calls_tools (int | None) —

• llm_cost_total (float | None) —

• llm_token_usage_total (dict[str, int] | None) —

OptimizationContext(
    prompts: dict,
    initial_prompts: dict,
    is_single_prompt_optimization: bool,
    dataset: Dataset,
    evaluation_dataset: Dataset,
    validation_dataset: opik.api_objects.dataset.dataset.Dataset | None,
    metric: MetricFunction,
    agent: opik_optimizer.agents.optimizable_agent.OptimizableAgent | None,
    optimization: opik.api_objects.optimization.optimization.Optimization | None,
    optimization_id: str | None,
    experiment_config: dict[str, typing.Any] | None,
    n_samples: int | float | str | None,
    max_trials: int,
    project_name: str,
    n_samples_minibatch: int | None = None,
    n_samples_strategy: str = 'random_sorted',
    allow_tool_use: bool = True,
    baseline_score: float | None = None,
    extra_params: dict = <factory>,
    trials_completed: int = 0,
    should_stop: bool = False,
    finish_reason: Optional = None,
    current_best_score: float | None = None,
    current_best_prompt: dict[str, opik_optimizer.api_objects.chat_prompt.ChatPrompt] | None = None,
    dataset_split: str | None = None,
    scoring_health: dict[str, int] | None = None
)

Parameters:

• prompts (dict) —

• initial_prompts (dict) —

• is_single_prompt_optimization (bool) —

• dataset (Dataset) —

• evaluation_dataset (Dataset) —

• validation_dataset (opik.api_objects.dataset.dataset.Dataset | None) —

• metric (MetricFunction) —

• agent (opik_optimizer.agents.optimizable_agent.OptimizableAgent | None) —

• optimization (opik.api_objects.optimization.optimization.Optimization | None) —

• optimization_id (str | None) —

• experiment_config (dict[str, typing.Any] | None) —

• n_samples (int | float | str | None) —

• max_trials (int) —

• project_name (str) —

• n_samples_minibatch (int | None) —

• n_samples_strategy (str, default: random_sorted) —

• allow_tool_use (bool, default: True) —

• baseline_score (float | None) —

• extra_params (dict, default: ) —

• trials_completed (int, default: 0) —

• should_stop (bool, default: False) —

• finish_reason (Optional) —

• current_best_score (float | None) —

• current_best_prompt (dict[str, opik_optimizer.api_objects.chat_prompt.ChatPrompt] | None) —

• dataset_split (str | None) —

• scoring_health (dict[str, int] | None) —

OptimizationHistoryState(
    context: Any = None
)

Parameters:

• context (Any) —

clear()

end_round(
    round_handle: Any,
    best_score: float | None = None,
    best_candidate: typing.Any | None = None,
    best_prompt: typing.Any | None = None,
    stop_reason: str | None = None,
    extras: dict[str, typing.Any] | None = None,
    candidates: list[dict[str, typing.Any]] | None = None,
    timestamp: str | None = None,
    pareto_front: list[dict[str, typing.Any]] | None = None,
    selection_meta: dict[str, typing.Any] | None = None,
    dataset_split: str | None = None
)

Parameters:

• round_handle (Any) —

• best_score (float | None) —

• best_candidate (typing.Any | None) —

• best_prompt (typing.Any | None) —

• stop_reason (str | None) —

• extras (dict[str, typing.Any] | None) —

• candidates (list[dict[str, typing.Any]] | None) —

• timestamp (str | None) —

• pareto_front (list[dict[str, typing.Any]] | None) —

• selection_meta (dict[str, typing.Any] | None) —

• dataset_split (str | None) —

finalize_stop(
    stop_reason: str | None = None
)

Parameters:

• stop_reason (str | None) —

get_entries()

get_rounds()

record_trial(
    round_handle: Any,
    score: float | None,
    candidate: typing.Any | None = None,
    trial_index: int | None = None,
    metrics: dict[str, typing.Any] | None = None,
    dataset: str | None = None,
    dataset_split: str | None = None,
    extras: dict[str, typing.Any] | None = None,
    candidates: list[dict[str, typing.Any]] | None = None,
    timestamp: str | None = None,
    stop_reason: str | None = None,
    candidate_id_prefix: str | None = None
)

Parameters:

• round_handle (Any) —

• score (float | None) —

• candidate (typing.Any | None) —

• trial_index (int | None) —

• metrics (dict[str, typing.Any] | None) —

• dataset (str | None) —

• dataset_split (str | None) —

• extras (dict[str, typing.Any] | None) —

• candidates (list[dict[str, typing.Any]] | None) —

• timestamp (str | None) —

• stop_reason (str | None) —

• candidate_id_prefix (str | None) —

set_context(
    context: Any
)

Parameters:

• context (Any) —

set_default_dataset_split(
    dataset_split: str | None
)

Parameters:

• dataset_split (str | None) —

set_pareto_front(
    pareto_front: list[dict[str, typing.Any]] | None
)

Parameters:

• pareto_front (list[dict[str, typing.Any]] | None) —

set_selection_meta(
    selection_meta: dict[str, typing.Any] | None
)

Parameters:

• selection_meta (dict[str, typing.Any] | None) —

start_round(
    round_index: int | None = None,
    extras: dict[str, typing.Any] | None = None,
    timestamp: str | None = None
)

Parameters:

• round_index (int | None) —

• extras (dict[str, typing.Any] | None) —

• timestamp (str | None) —

with_dataset_split(
    dataset_split: str | None
)

Parameters:

• dataset_split (str | None) —

OptimizationRound(
    round_index: int,
    trials: list = <factory>,
    best_score: float | None = None,
    best_so_far: float | None = None,
    best_prompt: typing.Any | None = None,
    best_candidate: typing.Any | None = None,
    candidates: list[dict[str, typing.Any]] | None = None,
    generated_prompts: list[dict[str, typing.Any]] | None = None,
    stop_reason: str | None = None,
    stopped: bool | None = None,
    dataset_split: str | None = None,
    extras: dict[str, typing.Any] | None = None,
    timestamp: str = <factory>
)

Parameters:

• round_index (int) —

• trials (list, default: ) —

• best_score (float | None) —

• best_so_far (float | None) —

• best_prompt (typing.Any | None) —

• best_candidate (typing.Any | None) —

• candidates (list[dict[str, typing.Any]] | None) —

• generated_prompts (list[dict[str, typing.Any]] | None) —

• stop_reason (str | None) —

• stopped (bool | None) —

• dataset_split (str | None) —

• extras (dict[str, typing.Any] | None) —

• timestamp (str, default: ) —

to_dict()

OptimizationTrial(
    trial_index: int | None,
    score: float | None,
    candidate: Any,
    metrics: dict[str, typing.Any] | None = None,
    dataset: str | None = None,
    dataset_split: str | None = None,
    candidate_id: str | None = None,
    extras: dict[str, typing.Any] | None = None,
    timestamp: str = <factory>
)

Parameters:

• trial_index (int | None) —

• score (float | None) —

• candidate (Any) —

• metrics (dict[str, typing.Any] | None) —

• dataset (str | None) —

• dataset_split (str | None) —

• candidate_id (str | None) —

• extras (dict[str, typing.Any] | None) —

• timestamp (str, default: ) —

to_dict()

OptimizableAgent(
    prompt: Any = None,
    project_name: Any = None,
    kwargs: Any
)

Parameters:

• prompt (Any) —

• project_name (Any) —

• kwargs (Any) —

init_agent(
    prompt: Any
)

Parameters:

• prompt (Any) —

init_llm()

invoke(
    messages: list,
    seed: int | None = None
)

Parameters:

• messages (list) — List of message dictionaries

• seed (int | None) — Optional seed for reproducibility

invoke_agent(
    prompts: Any,
    dataset_item: Any,
    allow_tool_use: Any = False,
    seed: Any = None
)

Parameters:

• prompts (Any) —

• dataset_item (Any) —

• allow_tool_use (Any, default: False) —

• seed (Any) —

invoke_agent_candidates(
    prompts: Any,
    dataset_item: Any,
    allow_tool_use: Any = False,
    seed: Any = None
)

Parameters:

• prompts (Any) — Mapping of prompt name to ChatPrompt.

• dataset_item (Any) — Dataset row used to render the prompt messages.

• allow_tool_use (Any, default: False) — Whether tool execution is allowed in this invocation.

• seed (Any) — Optional seed for reproducibility.

invoke_dataset_item(
    dataset_item: dict
)

Parameters:

• dataset_item (dict) —

invoke_prompt(
    prompt: Any,
    dataset_item: Any,
    allow_tool_use: Any = False,
    seed: Any = None
)

Parameters:

• prompt (Any) —

• dataset_item (Any) —

• allow_tool_use (Any, default: False) —

• seed (Any) —

llm_invoke(
    query: str | None = None,
    messages: list[dict[str, str]] | None = None,
    seed: int | None = None,
    allow_tool_use: bool | None = False
)

Parameters:

• query (str | None) —

• messages (list[dict[str, str]] | None) —

• seed (int | None) —

• allow_tool_use (bool | None, default: False) —

MultiMetricObjective(
    metrics: list,
    weights: list[float] | None = None,
    name: str = 'multi_metric_objective',
    reason: str | None = None,
    reason_builder: collections.abc.Callable[[list[_opik._score_result.ScoreResult], list[float], float], str | None] | None = None
)

Parameters:

• metrics (list) —

• weights (list[float] | None) —

• name (str, default: multi_metric_objective) —

• reason (str | None) —

• reason_builder (collections.abc.Callable[[list[_opik._score_result.ScoreResult], list[float], float], str | None] | None) —

ScoringFailedError(
    failed: int,
    total: int,
    objective_metric_name: str | None = None,
    message: str | None = None
)

Parameters:

• failed (int) —

• total (int) —

• objective_metric_name (str | None) —

• message (str | None) —

PromptLibrary(
    defaults: dict,
    overrides: dict[str, str] | collections.abc.Callable[[opik_optimizer.utils.prompt_library.PromptLibrary], None] | None = None
)

Parameters:

• defaults (dict) — Dictionary of default prompt templates

• overrides (dict[str, str] | collections.abc.Callable[[opik_optimizer.utils.prompt_library.PromptLibrary], None] | None) — Optional dict or callable to customize prompts

get(
    key: str,
    fmt: object
)

Parameters:

• key (str) — The prompt key to retrieve

• fmt (object) —