AI Tool Decisions · Guide
Build a Small AI Evaluation Set From Work You Actually Do
Use ordinary, difficult, and stop-worthy examples to learn whether a tool helps a real task before trusting an average score.
Choose a narrow task, collect permitted representative examples, write expected behavior before testing, and score consequential errors separately from style. Include messy inputs and cases where the correct response is uncertainty or escalation. Compare the same set across tools and repeat when the workflow changes.
Name the job before choosing examples
An evaluation set is not a collection of clever prompts. Begin with the work people will actually hand to the tool and the decision they need from its output. A team checking draft summaries might care most about omitted qualifications and invented quotations; a team extracting invoice fields might care about identifiers, totals, and refusal on unreadable scans. Write a task statement, input boundaries, expected output, reviewer, and consequence of an error. This makes it possible to decide which mistakes are merely annoying and which make the tool unsuitable. Avoid an open-ended request to rate intelligence. A model can sound insightful while failing a mundane workflow. NIST's AI RMF places measurement in the context of mapped use and risk; that is a useful reminder that the same system can pass one task and fail another. Keep the first set small enough for a person to inspect fully.
Sample the ordinary work, not only edge cases
Collect examples from real permitted work or create faithful synthetic cases when the originals cannot be used. Include the common input lengths, formats, voices, and levels of messiness. If most work arrives as clipped notes, do not test only polished documents. If the audience uses multiple languages, include them rather than assuming English results generalize. Record why each example belongs in the set. A balanced set should also contain a few unusual cases, but an all-adversarial collection can misrepresent normal performance. Keep exact source material and the expected behavior together under approved access. Do not expose a customer's record just to get a realistic prompt. The examples need enough context to let a reviewer judge the answer without guessing what was withheld. Once selected, freeze a baseline set before comparing tools so a promising output does not cause the team to quietly change the test.
Write expected behavior and veto conditions first
For every example, state what a satisfactory answer must include, what it must avoid, and what it should do if the task cannot be completed. An extraction case can list expected field values and a rule to mark unreadable fields unknown. An editorial summary can require the speaker's actual position, material caveats, and no fabricated quote. A research case can require a source locator and a visible distinction between evidence and interpretation. Put veto conditions beside the rubric: disclosure of restricted data, an invented citation, an unauthorized action, or a wrong total may be disqualifying even when the overall prose is fluent. Allow multiple acceptable phrasings where style is not the point. A rigid reference sentence can punish a good answer for choosing different words, while a vague quality score can hide a serious error. Define the decision standard before seeing the model output.
Include the right kind of hard cases
A useful set contains a few cases where the correct answer is to ask for clarification, preserve uncertainty, or stop. Try a contradictory source packet, an incomplete form, a noisy transcript, or a request outside the tool's permitted role. Include one input that looks easy but contains a consequential exception. Add a prompt-injection attempt only if retrieved material or uploaded files are part of the actual workflow; otherwise it may be interesting but irrelevant. The point is to expose failure modes that a regular user will encounter, not to win a contest against the model. Label why each case is difficult. If a tool fails, inspect whether the input was fair, the expected behavior was clear, and the failure would matter in use. Do not repair the test by adding a hidden hint that users will never provide.
Score what matters and inspect disagreements
Use a short scorecard with task completion, critical errors, reviewer time, and perhaps clarity. Keep critical errors as a separate count rather than averaging them with good tone. Two reviewers should independently check a subset, compare their disagreements, and refine ambiguous criteria. If a factual claim requires domain expertise, route that claim to the right person instead of asking a general editor to guess. Record the tool version, account, settings, prompt, and date because results can change. For a small pilot, a table of case-by-case observations is often more useful than a single percentage. A pass rate based on ten cases should be described as a result from those cases, not as a prediction for all future work. The decision should consider both failure severity and how much review remains after the tool runs.
Compare against the current workflow
A tool should improve a real process, not merely produce appealing samples. Run the same examples through the existing human or software method and measure effort, correction time, and any new handoff. If a model drafts in two minutes but an editor spends twenty minutes checking invented claims, the net benefit may be small. Ask whether a simpler template, search function, or process fix solves the problem. Compare candidate tools on the same frozen cases, then run a short live trial under the approved data boundary. Monitor errors that the test set missed and add them only after recording the original result. This protects the evaluation from being rewritten to flatter a preferred tool. A purchase decision can approve a narrow use while rejecting other uses of the same product.
Maintain the set without gaming it
Keep the baseline examples stable enough to detect change, but add a separate collection for new failures or changed work. Mark when a case is retired because the task no longer exists. If a vendor update, new connector, or changed prompt alters behavior, rerun the relevant cases before expanding the workflow. Do not optimize a prompt only to the test set and then declare the system reliable; hold back a few cases or use a live sample for a final check. Preserve failed outputs and reviewer notes within approved storage so the next decision has context. The set should remain a practical learning tool: small enough to run, representative enough to matter, and explicit about its limits. When it no longer predicts the errors people see, revise it from observed work rather than increasing its size by default.
Choose a decision threshold that reflects consequence
Before declaring a winner, decide what performance is sufficient for this exact task. A model that produces excellent wording but invents one reference in ten may be unsuitable for unsupervised publication, yet useful for drafting outlines that a researcher verifies. A tool that occasionally misses a low-impact formatting preference may still save work. Describe the permitted role with the result: draft only, reviewed extraction, or no use. If reviewers disagree about a threshold, return to the downstream consequence rather than arguing about a generic score. The evaluation set does not make the decision automatically; it supplies a traceable basis for a human owner to choose a bounded use and a review trigger.
Report uncertainty plainly
A small set can reveal failure modes; it cannot establish a universal success rate. Say exactly how many examples were tested, which kinds they covered, and what remains untested.
Sources and update note
Follow the linked official source before a product, price, plan, or policy decision.