Build an LLM evaluation dataset from real business tasks

Start with the decision the evaluation will support

OpenAI’s evals guide describes defining an evaluation, running it against test data and analyzing the results. The mechanism can repeat tests, but the organization still needs to decide what a correct result means. Write the deployment question first: for example, whether a proposed routing configuration is suitable for a defined shared-inbox workload.

Identify the input population, intended output and consequences of error. Keep the model, prompt, tools and review path part of the evaluated configuration. Changing one of them may change the result even when the dataset stays the same.

Write the expected outcome before running the model

For each case, retain the input, source identity, expected fields or decision and the reason that outcome is supported. Use authorized sample data or appropriately prepared examples. Have the operational owner check the answer key before testing. A label produced by another model is a proposal for that review, not an independent ground truth.

Define the outcomes your team accepts, including unresolved cases. A reviewed dataset makes those expectations repeatable when the model, prompt or workflow changes.

Evidence	Meaning
Cases	Ordinary and exceptional inputs
Rubric	Observable accepted outcome
Run	Versioned configuration
Review	Errors and repair effort

Separate cases used to tune the workflow from a held-out set used to assess it. Repeatedly adapting the prompt to every visible example can produce a configuration that knows the test rather than the underlying task. Add newly discovered failures deliberately, preserving a record of when the dataset changed so results remain interpretable.

Work through an inbox-routing rubric

Consider a shared inbox that routes messages to purchasing, finance or a clarification queue. A supplier quotation belongs with purchasing. An invoice concerning a known order belongs with finance under this team’s chosen process. A message containing both a price revision and an invoice needs a documented multi-topic rule; guessing a single category can hide part of the work.

Example inputs and expected treatment
Input	Expected treatment	Important error
Quotation with a known request ID	Purchasing route and correct request link.	Correct queue, wrong purchase.
Invoice with an unknown order	Finance review; unresolved order link.	Invented order identity.
Two topics in one message	Preserve both under the written routing rule.	One topic silently omitted.
Insufficient context	Clarification queue.	Confident but unsupported routing.

Score routing, record identity and preservation of uncertainty separately. One aggregate percentage can hide a configuration that classifies ordinary quotations well but attaches exceptions to the wrong purchase. Use those errors to set review requirements and deployment acceptance.

Use checks appropriate to the output

Exact checks suit stable IDs, allowed categories and arithmetic relationships. A written rubric is more useful for an explanation whose wording can vary while its meaning stays correct. Ask reviewers to judge the supported conclusion and missing conditions, rather than stylistic similarity to a single reference paragraph. Keep disputed cases available for adjudication.

Run the same inputs against the compared configurations.
Record failures, unresolved cases and repair effort.
Inspect high-consequence errors individually.
Repeat a sample where variable behavior affects the decision.
Preserve the configuration, dataset and rubric versions.

Interpret results with their limits

A small set can reveal obvious defects but cannot establish how every future input will behave. Report the case composition and the important omissions alongside the result. Do not claim a universal accuracy level from a convenient sample. A new supplier format, language or workflow population can require a new evaluation slice.

Include time and cost to reach an accepted result, including human correction and unresolved exceptions. Bring the reviewed cases and these measures to a workflow assessment with Trion.

Maintain the dataset after deployment

Turn recurring mistakes into reviewed cases, then check proposed fixes against earlier cases as well. A correction to one category can regress another. Assign ownership for the answer key, sensitive sample handling and release acceptance.