Skip to main content
Opik Documentation
current

Search documentation

Type to search this documentation.

On this pageOverview

Optimize prompts

Pick the right optimizer, run experiments, and ship better prompts.

Use this playbook whenever you need to improve a prompt (single-turn or agentic) and want a repeatable process rather than manual tweaks.

  • Record the current prompt and score using your production metric.
  • Log at least 10 representative dataset rows so the optimizer can generalize.
  • Capture latency and token costs—optimizations should not regress them unexpectedly.
Scenario Recommended optimizer
General prompt copy edits MetaPrompt
Complex failure analysis HRPO
Need diverse candidates Evolutionary
Few-shot heavy prompts Few-Shot Bayesian
Tune sampling params Parameter optimizer
Python
from opik_optimizer import HRPO

optimizer = HRPO(
    model="openai/gpt-4o",
    max_parallel_batches=4,
    seed=42,
)
result = optimizer.optimize_prompt(
    prompt=my_prompt,
    dataset=my_dataset,
    metric=answer_quality,
    max_trials=5,
    n_samples=50,
)
  • Set project_name on the optimizer to group runs by team or initiative.
  • Start with max_trials = 3–5. Increase once you confirm the metric is reliable.
  • Use n_samples to limit cost during early exploration; rerun on the full dataset before promoting a prompt.
  • For optimizers with inner-loop evaluations (HRPO, GEPA), set n_samples_minibatch to keep those steps lightweight.
  • Use n_samples_strategy to keep subsampling deterministic (default: "random_sorted").

Tool optimization is now documented separately. Use it when you want to improve MCP tool descriptions without changing prompt text.

Target specific sections inside a prompt (advanced)

Section titled “Target specific sections inside a prompt (advanced)”

If you need finer control than roles (for example, only optimize a specific assistant message), use prompt_segments to extract and update parts by segment ID.

Intent/Trigger: use segment-level updates when you need to constrain changes to exact message segments.

  • Required parameters: prompt, dataset, metric
  • Optional parameters: segment update args (updates passed to prompt_segments.apply_segment_updates)
  • Minimal valid payload: optimizer.optimize_prompt(prompt=updated_prompt, dataset=my_dataset, metric=answer_quality)
Python
from opik_optimizer.utils import prompt_segments

segments = prompt_segments.extract_prompt_segments(my_prompt)
for segment in segments:
    print(segment.segment_id, segment.role)

# Update only message:1 (second message)
updates = {"message:1": "User question: {user_query}"}
updated_prompt = prompt_segments.apply_segment_updates(my_prompt, updates)

# Use the updated prompt in optimization (the original prompt is unchanged)
result = optimizer.optimize_prompt(
    prompt=updated_prompt,
    dataset=my_dataset,
    metric=answer_quality,
)

You can pass a dict of ChatPrompt objects to optimize a coordinated prompt bundle (for example, a multi-agent setup or system/user prompt pair that must stay in sync). Each key names a prompt and is preserved through optimization.

Python
from opik_optimizer import MetaPromptOptimizer, ChatPrompt

prompts = {
    "researcher": ChatPrompt(
        name="researcher",
        messages=[
            {"role": "system", "content": "Gather facts and cite sources."},
            {"role": "user", "content": "{question}"},
        ],
    ),
    "synthesizer": ChatPrompt(
        name="synthesizer",
        messages=[
            {"role": "system", "content": "Summarize findings clearly."},
            {"role": "user", "content": "{question}"},
        ],
    ),
}

optimizer = MetaPromptOptimizer(model="openai/gpt-4o-mini", prompts_per_round=2)
result = optimizer.optimize_prompt(
    prompt=prompts,
    dataset=my_dataset,
    metric=answer_quality,
    max_trials=3,
)

result.prompt returns a dict keyed by the same names so you can update each agent prompt together.

  • Compare result.score vs. result.initial_score to ensure material improvement.
  • Review the history attribute for regression reasons.
  • Use Dashboard results to visualize per-trial performance.
  1. Export the prompt

    result.prompt returns the best-performing ChatPrompt. Serialize it as JSON and check it into your repo.

  2. Automate regression tests

    Wire the optimizer run into CI with a smaller dataset so future prompt edits have guardrails.

  3. Monitor in production

    Trace the new prompt with Opik tracing to confirm real-world performance matches experiment results.

Suggest an edit

Propose a replacement for this page. The site team reviews it before applying any changes.

Export
Documentation menu