Rulebook & Rule Creation Best Practices

Rule creation is an iterative process

A rule should not be considered finished simply because it reads well. The real validation comes from running it against representative artwork and reviewing how the systems interprets the instruction. 

Here is a practical guide for creating, testing, and improving high-quality CheckAI rules. 

1. Build the rule around one inspection objective 

Start with the simplest clear rule that expresses the requirement. Test it on representative files, review how the AI interpreted it, and refine only where real ambiguity, false positives, or missed findings appear. 

Before writing anything, define the single question the rule should answer. A rule is easier to understand, test, tune, and maintain when it has one clear purpose. 

If those are genuinely different inspection objectives, split them into separate rules. Related rules can still live under the same section of the rulebook. 

2. Use the short rule + Additional Guidance structure

The visible rule sentence should summarize what is being checked. The Additional Guidance should contain the detail needed to interpret that check correctly. 

Example 

Short rule: Check that nutrient claims comply with the applicable regulatory thresholds. 

Additional Guidance: Explain when a statement is considered a nutrient claim, which nutrition values should support it, how the applicable threshold is selected, and any legitimate exceptions such as negligible declarations. 

3. Using Reference Files V.S. Artwork Only checks

CheckAI rules generally fall into two categories: 

3.1 For Artwork-only / Compliance rules: make the trigger and requirement clear

Many compliance rules are naturally conditional. The rule should make it clear when the requirement applies and what must be true when it does. 

Useful pattern: IF condition → THEN requirement. 

Generic example 

If aspartame is declared in the ingredients list, verify that the required phenylalanine warning is present. 

You do not need a long applicability framework if the condition can be stated naturally. Add explicit “Pass if not applicable” guidance only when the model could otherwise mistake a legitimate absence for missing information. 

Artwork-only / Compliance rule 

TRIGGER → REQUIREMENT → INTERPRETATION → EXCEPTIONS 

TRIGGER: When does the requirement apply? 
REQUIREMENT: What must be true on the artwork? 
INTERPRETATION: How should the requirement be evaluated? 
EXCEPTIONS: What should not trigger a failure? 

3.2 For Reference Comparison rules: make the comparison unambiguous 

A comparison rule should clearly establish what reference content is expected to correspond to what artwork content. How specific you need to be depends on the structure of the reference file. 

• If the reference file has dedicated categories or fields, name the relevant one when necessary. 

• If nearby metadata or another reference category could be mistaken for expected artwork content, explicitly exclude it. 

• If the artwork extraction may group neighboring text together, clarify that only the intended element should be compared. 

• If the reference element is optional, define what should happen when it is not provided. 

Generic example 

Short rule: Check that the Ingredients / Composition on the artwork matches the corresponding content in the reference file. 

Useful guidance: Use the dedicated Ingredients / Composition content as the source of truth. Compare the corresponding artwork ingredient content. Do not treat adjacent warning or marketing text as part of the ingredient declaration solely because it is grouped nearby. 

Reference Comparison rule 

WHAT → SOURCE → MATCH → EXCEPTIONS 

WHAT: What element is being checked? 
SOURCE: What reference content should be used? 
MATCH: What does it mean for the artwork to match? 
EXCEPTIONS: What harmless differences or missing reference content are acceptable? 

4. Define what matters and what does not 

The word “match” can mean different things depending on the rule. Decide what variation is harmless and what variation is substantive. 

Tolerance should be rule-specific. A spelling rule should not broadly accept semantic similarity; a layout-neutral text comparison should not fail because a line wrapped differently. 

5. Add exclusions only when there is a plausible wrong interpretation 

Negative instructions are powerful when the AI has a realistic alternative interpretation. They should solve a known ambiguity, not become boilerplate added to every rule. 

• “Do not treat front-of-pack nutrition as a full nutrition table.” 

• “Do not infer country of origin from a company address.” 

• “Do not treat branding alone as the legal denomination.” 

• “Do not evaluate barcodes or numeric identifiers under this product-name consistency rule.” 

6. Avoid common authoring traps

• Do not combine unrelated checks into one large “check everything” rule. 

• Do not use vague instructions such as “check carefully” or “use context” when the intended scope can be stated directly. 

• Do not add every possible exception pre-emptively; let testing reveal real ambiguities. 

• Do not compensate for extraction defects with increasingly complicated rule text. 

• Do not create overlapping rules that repeatedly flag the same defect unless the separate findings are genuinely useful. 

• Do not make the short rule so generic that it no longer represents the actual logic in the Additional Guidance. 

Troubleshooting and diagnosing rulebooks

Important 

Do not rewrite a rule after every isolated odd result. First understand why the result occurred. A downstream false positive may be caused by extraction or grouping rather than the rule itself. 

When a rule produces an unexpected result, classify the problem before changing the prompt.