Evolutionary Optimizer: Genetic Algorithms

The EvolutionaryOptimizer uses genetic algorithms to refine and discover effective prompts. It iteratively evolves a population of prompts, applying selection, crossover, and mutation operations to find prompts that maximize a given evaluation metric. This optimizer can also perform multi-objective optimization (e.g., maximizing score while minimizing prompt length) and leverage LLMs for more sophisticated genetic operations.

The EvolutionaryOptimizer is built upon the DEAP library for evolutionary computation. The core concept behind the optimizer is that we evolve a population of prompts over multiple generations to find the best one.

We utilize different techniques to evolve the population of prompts:

• Selection: We select the best prompts from the population to be the parents of the next generation.
• Crossover: We crossover the parents to create the children of the next generation.
• Mutation: We mutate the children to create the new population of prompts.

We repeat this process for a number of generations until we find the best prompt.

You can use the EvolutionaryOptimizer to optimize a prompt:

from opik_optimizer import EvolutionaryOptimizer
from opik.evaluation.metrics import LevenshteinRatio # or any other suitable metric
from opik_optimizer import datasets, ChatPrompt

# 1. Define your evaluation dataset
dataset = datasets.tiny_test() # Replace with your actual dataset

# 2. Configure the evaluation metric
def levenshtein_ratio(dataset_item, llm_output):
    return LevenshteinRatio().score(reference=dataset_item["label"], output=llm_output)

# 3. Define your base prompt and task configuration
initial_prompt = ChatPrompt(
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "{text}"}
    ]
)

# 4. Initialize the EvolutionaryOptimizer
optimizer = EvolutionaryOptimizer(
    model="openai/gpt-4o-mini",
    model_parameters={"temperature": 0.4},
    population_size=20,
    num_generations=10,
)

# 5. Run the optimization
optimization_result = optimizer.optimize_prompt(
    prompt=initial_prompt,
    dataset=dataset,
    metric=levenshtein_ratio,
    n_samples=5
)

# 6. View the results
optimization_result.display()

The optimizer has the following parameters:

• model (str, default: openai/gpt-5-nano) —

• model_parameters (dict[str, typing.Any] | None) —

• population_size (int, default: 30) —

• num_generations (int, default: 15) —

• mutation_rate (float, default: 0.2) —

• crossover_rate (float, default: 0.8) —

• tournament_size (int, default: 4) —

• elitism_size (int, default: 3) —

• adaptive_mutation (bool, default: True) —

• enable_moo (bool, default: True) —

• enable_llm_crossover (bool, default: True) —

• output_style_guidance (str | None) —

• infer_output_style (bool, default: False) —

• n_threads (int, default: 12) —

• verbose (int, default: 1) —

• seed (int, default: 42) —

The optimize_prompt method has the following parameters:

• prompt (ChatPrompt) — The prompt to optimize

• dataset (Dataset) — The dataset to use for evaluation

• metric (Callable) — Metric function to optimize with, should have the arguments dataset_item and llm_output

• experiment_config (dict | None) — Optional experiment configuration

• n_samples (int | float | str | None) — Number of dataset items to use per evaluation. Use counts (e.g., 50), fractions (e.g., 0.1), percentages (e.g., "10%"), or "all"/"full"/None for the full dataset.

• n_samples_minibatch (int | None) — Optional number of samples for inner-loop minibatches (defaults to n_samples).

• n_samples_strategy (str | None) — Sampling strategy for subsampling (default: "random_sorted").

• auto_continue (bool, default: False) — Whether to automatically continue optimization

• agent_class (type[opik_optimizer.optimizable_agent.OptimizableAgent] | None) — Optional agent class to use

• project_name (str, default: Optimization) — Opik project name for logging traces (default: "Optimization")

• max_trials (int, default: 10) —

• mcp_config (opik_optimizer.mcp_utils.mcp_workflow.MCPExecutionConfig | None) — MCP tool calling configuration (default: None)

• args (Any) —

• kwargs (Any) —

There are two models to consider when using the EvolutionaryOptimizer:

• EvolutionaryOptimizer.model: The model used for the evolution of the population of prompts.
• ChatPrompt.model: The model used to evaluate the prompt.

The model parameter accepts any LiteLLM-supported model string (e.g., "gpt-4o", "azure/gpt-4", "anthropic/claude-3-opus", "gemini/gemini-1.5-pro"). You can also pass in extra model parameters using the model_parameters parameter:

optimizer = EvolutionaryOptimizer(
    model="anthropic/claude-3-opus-20240229",
    model_parameters={
        "temperature": 0.7,
        "max_tokens": 4096
    }
)

1. Explore specific Optimizers for algorithm details.
2. Refer to the FAQ for common questions and troubleshooting.
3. Refer to the API Reference for detailed configuration options.