Evaluation

The TypeScript SDK provides a streamlined approach to evaluating your LLM applications with the evaluate function that handles various evaluation scenarios.

flowchart LR
    A[Dataset item] --> B[Evaluation Task]
    B --> C[Data Processing]
    C --> D[Metrics Scoring]
    D --> E[Evaluation Results]

    style A fill:#f9f9f9,stroke:#333,stroke-width:2px
    style B fill:#e1f5fe,stroke:#333,stroke-width:2px
    style C fill:#e8f5e9,stroke:#333,stroke-width:2px
    style D fill:#e3f2fd,stroke:#333,stroke-width:2px
    style E fill:#fff3e0,stroke:#333,stroke-width:2px

The TypeScript SDK evaluation documentation covers:

• Quick Start: Get up and running with basic evaluations
• Datasets: Working with evaluation datasets
• Evaluate Function: Using the evaluate function
• Evaluation Metrics: Available metrics and creating custom metrics
• Experiments: Creating and managing evaluation experiments
• Test Suites: Regression testing with assertions and execution policies