Opik Optimizer

This guide will help you get started with contributing to the Agent Optimizer SDK, our tool for optimizing prompts and improving model performance.

Before you start, please review our general Contribution Overview and the Contributor License Agreement (CLA).

Project Structure

The Agent Optimizer is located in the sdks/opik_optimizer directory. Here's an overview of the key components:

src/: Main source code
benchmarks/: Benchmarking tools and results
notebooks/: Example notebooks and tutorials
tests/: Test files
docs/: Additional documentation
scripts/: Utility scripts
setup.py: Package configuration
requirements.txt: Python dependencies

Setup

Create virtual environment

Bash

python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

Install dependencies

Bash

cd sdks/opik_optimizer
pip install -r requirements.txt
pip install -e .

Run tests

Bash

pytest tests/

:::

Development Workflow

Create branch
Create a new branch for your changes

Make changes

Make your changes

Add tests

Add tests for new functionality

Run tests

Run the test suite

Run benchmarks

Run benchmarks if applicable

Submit PR

Submit a pull request

:::

Testing

We use pytest for testing. When adding new features:

Write unit tests
Write unit tests in the tests/ directory

Run tests

Ensure all tests pass with pytest tests/

:::

Benchmarking

The optimizer includes benchmarking tools to measure performance improvements:

Run benchmarks
bash cd benchmarks python run_benchmark.py

View results

View results in the benchmark_results/ directory

Add benchmarks

Add new benchmarks for new optimization strategies

:::

Documentation

When adding new features or making changes:

Update README
Update the README.md

Add docstrings

Add docstrings for new functions and classes

Add examples

Include examples in the notebooks/ directory

Update docs

Update the main documentation if necessary. See the Documentation Guide for details.

:::

Code Style

We follow PEP 8 guidelines. Before submitting a PR:

Run linter
Run flake8 to check for style issues

Fix issues

Fix any linting errors

Check style

Ensure your code follows Python best practices

:::

Pull Request Process

Fork repository
Fork the repository

Create branch

Create your feature branch

Make changes

Make your changes

Run checks

Run tests and benchmarks

Submit PR

Submit a pull request

:::

Your PR should:

Have a clear description of the changes
Include tests for new functionality
Pass all CI checks
Follow the project's coding standards
Include benchmark results if applicable

Notebooks and Examples

The notebooks/ directory contains examples and tutorials. When adding new features:

Create notebook
Create a new notebook demonstrating the feature

Add explanations

Include clear explanations and comments

Show usage

Show both basic and advanced usage

Add comparisons

Add performance comparisons if relevant

:::

Need Help?

If you need help or have questions:

Open an issue on GitHub
Join our Comet Chat community
Check the existing documentation and notebooks

Remember to review our Contributor License Agreement (CLA) before contributing.