Feedback Definitions
Feedback definitions allow you to create custom labels to systematically collect and analyze structured feedback on your LLM outputs.
Using Feedback Definitions
Section titled “Using Feedback Definitions”Once created, feedback definitions appear in two places:
- Trace and thread sidebars — Add scores when reviewing individual items.
- Annotation queues — Pick which definitions annotators see when you create a queue.
Creating a Feedback Definition
Section titled “Creating a Feedback Definition”
- Click Create new feedback definition
- Enter a Name for your feedback definition
- Select the Type: Categorical or Numerical
- Define the Values (for Categorical) or Range (for Numerical)
Common Feedback Types
Section titled “Common Feedback Types”| Feedback Type | Type | Values |
|---|---|---|
| Thumbs Up / Down | Categorical | thumbs up, thumbs down |
| Usefulness | Categorical | Useful, Neutral, Not useful |
| Hallucination | Categorical | Yes, No |
| Correct | Categorical | Good, Bad |