AI Workflows · Guide
Write a Stop Rule for an AI Data-Cleanup Pilot
A cleanup experiment needs a point where uncertainty stops the operation, not a prompt to keep guessing.
AI can help identify duplicate entries, normalize labels, and suggest fixes for messy records. A small demonstration is easy to celebrate: the output table looks cleaner. The hard question is what happens when two records might represent the same person, a field conflicts with a trusted source, or the system cannot tell whether an earlier write succeeded. A stop rule answers those questions before the tool is allowed to change live data. It turns uncertainty into a visible review path rather than a quiet correction. Start with a bounded pilot on a copy or a read-only preview. Choose one dataset and one type of change, such as suggesting standardized category labels. Do not bundle deduplication, address updates, and account merging into one “cleanup” task. Each has different evidence and consequences. Write what a correct result looks like for the chosen change, who owns the authoritative value, and whether the change can be undone. A tool that cannot explain why it changed a value should not be the sole actor for that value.
Name the authority for each field
Make a compact field map: field name, meaning, trusted source, permitted transformation, and reviewer. Some fields can be normalized mechanically, such as trimming surrounding whitespace. Others require interpretation, such as deciding whether two company names are the same entity. An AI suggestion can be useful for the second group, but the business owner must define what evidence is sufficient. If the source of truth is another system, the pilot should read that source before proposing a write. A clean-looking table is not proof that the values are right. Distinguish formatting from factual changes. Changing “United States” to an approved country code may preserve meaning if the mapping is defined. Changing an unknown address to a likely address is a new factual claim. A model may treat both as cleanup, but the stop rule should not. Mark fields that affect payments, permissions, legal identity, or customer communications as requiring stronger review or exclusion from the pilot. The choice should follow the actual consequence of an error, not the convenience of the available model. For each candidate change, ask the assistant to show old value, proposed value, source evidence, transformation rule, and uncertainty. If it cannot locate a source or explain the mapping, the proposal belongs in a review queue. Do not let an empty field become a guessed value merely because complete rows score better. A field can be left unchanged while the team asks for better data. That is an acceptable result, not a pilot failure.
Write the stop conditions in plain language
A useful stop rule can be read by an operator under time pressure. For example: “Stop before writing if two candidate records share a name but not a stable identifier; if a proposed value conflicts with the designated source; if a required approval is absent; or if the result of an earlier write is unknown.” Add the action for each condition: show the affected record, preserve the original value, route to the named owner, and record the reason. A vague rule such as “stop on low confidence” does not tell people what to inspect. Include a batch-level threshold. If an offline sample reveals a cluster of wrong mappings, pause the whole pilot rather than asking reviewers to fix each row while the same defect continues. Choose the threshold based on the field's importance and the review team's capacity. The first small sample may justify a qualitative rule: one wrong account ID stops the trial; several minor category disagreements trigger a mapping review. Record the chosen threshold before testing so the team does not move it after seeing an attractive success rate. Define what to do when a result is unknown. A timeout after a write request does not prove that nothing happened. The tool should inspect its operation record and the external system's current state before considering another attempt. If those checks cannot establish the outcome, preserve the case and ask an authorized person to resolve it. Repeating a write to “be safe” can create duplicate records or undo a correct change. This rule applies even in a small pilot.
Test the stop rule offline
Build a set of cases that contains clean mappings, duplicate-looking records with different identifiers, conflicting source values, missing fields, and an interrupted write simulation. Have a reviewer state the expected outcome for each case. A pass is not always a corrected value; it may be “no change, needs owner.” Ask the AI system to propose changes in preview mode and compare every proposed edit with the expected outcome. Record false changes and missed stops separately. Include negative controls. A record that looks inconsistent but is valid should remain untouched. A familiar name that belongs to two people should not be merged. A category that is unusual but allowed should not be forced to a more common label. These examples test whether the pilot respects the data's meaning rather than optimizing for visual neatness. If the model confidently changes a negative control, narrow the transformation or improve the authority map before trying more examples. Have someone who did not design the rule run the review. Can they identify the stop condition from the displayed evidence? Do they know who owns the decision? Does the interface show the original value after a proposal is rejected? If the rule exists only in a planning document but the operator cannot apply it in the workflow, it is not ready. The pilot should make the safe path easy to take, including a clear way to leave a record unchanged.
Keep the pilot narrow when it works
A successful offline check can justify a supervised test with explicit authorization, a small number of records, and readback after each change. It does not authorize a full dataset rewrite. Keep a record of before and after values, stable record identity, reviewer decision, operation result, and any correction. The team should be able to tell whether a change was proposed, approved, attempted, and confirmed. Those are distinct states. A report that says “100 rows cleaned” without this distinction can conceal uncertain outcomes. Review the error patterns after the pilot. If most stops arise from a missing source field, fix the source process. If category disagreements reflect an unclear business definition, ask the owner to define it. If a model repeatedly confuses two entities, restrict the candidate set or keep that field manual. The purpose of a stop rule is not to make AI look conservative; it is to protect the meaning of records while revealing where the process needs work. A good outcome may be a small tool that suggests safe formatting changes and flags the rest. That is still progress. It gives people a cleaner queue and better evidence without pretending every ambiguity can be automated. Write the stop conditions before a live edit, test them on difficult examples, and preserve the original record whenever the evidence is insufficient.
Sources and update note
Follow the linked official source before a product, price, plan, or policy decision.