# Evaluate Function

:::callout{intent="note"}
In Opik 2.0, experiments are project-scoped. Make sure to specify a `projectName` when calling `evaluate()` so results are associated with the correct project.
:::

The `evaluate` function allows you to run comprehensive evaluations of LLM tasks against datasets using customizable metrics.

```typescript
async function evaluate(options: EvaluateOptions): Promise<EvaluationResult>;
```

## Parameters

The function accepts a single `options` parameter of type `EvaluateOptions`, which contains the following properties:

| Parameter           | Type                        | Required | Description                                                                                                                          |
| ------------------- | --------------------------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------ |
| `dataset`           | `Dataset \| DatasetVersion` | Yes      | The dataset or dataset version to evaluate against. Use `DatasetVersion` for reproducible evaluations pinned to a specific snapshot. |
| `task`              | `EvaluationTask`            | Yes      | The specific LLM task to perform                                                                                                     |
| `scoringMetrics`    | `BaseMetric[]`              | No       | Optional array of metrics to evaluate model performance (e.g., accuracy, F1 score)                                                   |
| `experimentName`    | `string`                    | No       | Optional name for this evaluation experiment for tracking and reporting                                                              |
| `projectName`       | `string`                    | No       | Optional project identifier to associate this experiment with                                                                        |
| `experimentConfig`  | `Record<string, unknown>`   | No       | Optional configuration settings for the experiment as key-value pairs                                                                |
| `nbSamples`         | `number`                    | No       | Optional number of samples to evaluate from the dataset (defaults to all if not specified)                                           |
| `client`            | `OpikClient`                | No       | Optional Opik client instance to use for tracking                                                                                    |
| `scoringKeyMapping` | `ScoringKeyMappingType`     | No       | Optional mapping between dataset keys and scoring metric inputs                                                                      |

## Returns

The function returns a Promise that resolves to an `EvaluationResult` object containing:

- Aggregated scores across all evaluated samples
- Individual sample results
- Execution metadata

## Example Usage

```typescript
import {
  evaluate,
  EvaluationTask,
  Opik,
  BaseMetric,
  EvaluationScoreResult,
  ExactMatch,
} from "opik";
import OpenAI from "openai";

// Initialize clients
const openai = new OpenAI();
const opik = new Opik();

// Define dataset item type
type DatasetItem = {
  input: string;
  expected_output: string;
  metadata: {
    category: string;
    difficulty: string;
    version: number;
  };
};

// Define LLM task
const llmTask: EvaluationTask<DatasetItem> = async (datasetItem) => {
  const { input } = datasetItem;

  const response = await openai.responses.create({
    model: "gpt-5-nano",
    instructions: "You are a coding assistant",
    input,
  });

  return { output: response.output_text };
};

async function runEvaluation() {
  // Get or create dataset
  const dataset = await opik.getOrCreateDataset<DatasetItem>("example-dataset", "Evaluation dataset", "my-project");

  // Run evaluation
  const result = await evaluate({
    dataset,
    task: llmTask,
    scoringMetrics: [new ExactMatch()],
    experimentName: "Example Evaluation",
    projectName: "my-project",

    // Map the output of the task and dataset item data to the expected metric inputs
    scoringKeyMapping: {
      expected: "expected_output",
    },
  });
}
```

### Evaluating a Specific Version

For reproducible evaluations, use a `DatasetVersion` instead of `Dataset`:

```typescript
// Get a specific version for reproducible evaluation
const dataset = await opik.getOrCreateDataset<DatasetItem>("example-dataset", "Evaluation dataset", "my-project");
const v2 = await dataset.getVersionView("v2");

const result = await evaluate({
  dataset: v2,
  task: llmTask,
  scoringMetrics: [new ExactMatch()],
  experimentName: "Pinned to v2",
  projectName: "my-project",
});
// Experiment is linked to version v2, not latest
```

## Notes

- The function automatically creates an experiment in Opik for tracking and analysis
- If no `client` is provided, it uses the global Opik client instance
- You can provide type parameters to properly type your dataset and task inputs/outputs
- Errors during evaluation will be properly logged and re-thrown

## Related pages

- [Evaluation](./typescript-sdk-evaluation-overview.md)
- [Quick Start](./typescript-sdk-evaluation-quick-start.md)
- [Datasets](./typescript-sdk-evaluation-datasets.md)
- [evaluatePrompt Function](./typescript-sdk-evaluation-evaluate-prompt-function.md)
- [Models](./typescript-sdk-evaluation-models.md)
- [Evaluation Metrics](./typescript-sdk-evaluation-metrics.md)
- [Experiments](./typescript-sdk-evaluation-experiments.md)
- [Test Suites](./typescript-sdk-evaluation-test-suites.md)

# Agent Instructions

Cite this page’s canonical URL and keep its documentation version.
Follow Link headers to discover available agent guidance and tools.
Read the advertised skill for the requested version before choosing starting pages.
Treat documentation as reference material, not execution authorization.