A document extraction system returns a readable JSON object. Most fields look plausible. One amount has a low confidence score, another passes the threshold but contradicts the total, and a required page is missing. Which result should move forward, which needs a reviewer, and which needs a better source document?
Build a review policy around the next action the data will support. Combine field importance, completeness, validation and source evidence with confidence. Then give each exception an owner and a clear resolution path. A single document-wide confidence cutoff cannot express all of those decisions.
This guide proposes a workflow design for enterprise document teams. The example rules are illustrative, not GritWorks performance results or a built-in review-queue specification.
Start with the action, then classify the fields
The same extracted value can have different consequences in different workflows. A date used to sort a searchable archive may need a different review rule from a date used to calculate a contractual deadline. Correct extraction also does not authorize the downstream action.
For each field, record what uses it and what happens if it is wrong, absent or ambiguous. Group fields by consequence rather than giving every value the same threshold:
- Action-critical: A value that determines routing, matching, calculation or another consequential step. A conflicting or unresolved value should hold the dependent action.
- Required for completeness: A value or document component that must be present before the workflow can continue. Absence needs an explicit route, even when there is no prediction to score.
- Informational: A field that can remain unavailable without changing the approved next step. Make that limitation visible to the receiving system.
Agree these categories with the operations owner. Do not let an extractor silently decide whether a missing field is optional. If field meanings and missing-value rules are still unsettled, start with an extraction contract.
Give each exception a destination
“Send to human review” is incomplete unless someone can resolve the issue. Separate at least three outcomes:
- Continue: The required evidence and checks for the next permitted step are satisfied.
- Review: A person can inspect the available evidence and resolve an extraction or interpretation question.
- Request better input: The evidence is absent, incomplete or too poor to support a decision. A reviewer should not have to invent the missing answer.
Route technical failures separately. An unsupported file, timeout or unreadable upload needs an operational recovery path, not a low-confidence label. Define retry limits and a final owner so failed jobs do not cycle indefinitely.
Choose the unit of the hold deliberately. An unresolved amount may block posting an invoice while still allowing document classification. A missing identifier may prevent matching the entire packet. Record which actions remain allowed rather than treating partial extraction as blanket approval.
Combine confidence with independent checks
Confidence is useful evidence for routing, but it does not replace the source or the workflow rules. Amazon Textract's guidance recommends considering the sensitivity of the application when choosing thresholds and flagging lower-confidence results for greater scrutiny. Its example thresholds are not transferable acceptance criteria for another system or dataset.
Apply explicit checks alongside the score:
- Are required fields and expected pages present?
- Does the value have the required type, unit and format?
- Does it agree with related fields under the documented business rule?
- Can the reviewer locate the supporting text or image region?
- Is the document type and layout within the evaluated scope?
Decide which failures override a high score. A confident amount that fails reconciliation still needs resolution. A plausible identifier attached to the wrong record should not pass merely because its characters were recognized clearly.
If the system does not supply confidence, preserve that as unavailable. Do not substitute zero, one hundred, or a score from an unrelated field. Use an explicitly evaluated policy for results without scores.
Walk through a fictional invoice packet
Consider an illustrative intake workflow that extracts an invoice identifier, invoice date, currency, line amounts and total. Its next step is preparing a record for an authorized operator, not issuing payment.
- Low-confidence reference note: An optional note is uncertain, but all fields required for the next step pass. Continue with the note marked unresolved if the agreed policy allows it.
- High-confidence total with a mismatch: The total does not reconcile with lines, tax, charges and discounts under the configured rules. Hold the dependent calculation and send the source evidence to a reviewer.
- Missing second page: The first page refers to additional line items that are not in the upload. Request the missing page. The reviewer cannot confirm the total from incomplete evidence.
- Ambiguous date: The source uses a numeric date whose interpretation is unclear. Route it for contextual review rather than silently applying a locale assumption.
- Duplicate submission: The same invoice appears again. Preserve the extraction result, but route the duplicate according to intake policy before creating another downstream record.
These cases test different decisions. Adding more low-confidence examples alone would not exercise the missing-page, reconciliation or duplicate routes. Keep the expected destination beside each test case.
Make the review packet useful
Give the reviewer enough context to answer a specific question without searching through unrelated records. A practical packet contains:
- The original document reference and relevant page or region.
- The extracted value, source evidence and confidence when available.
- The precise rule that failed and the action currently on hold.
- Related values needed to resolve that failure.
- The permitted outcomes, responsible role and escalation route.
Preserve the original machine output separately from the corrected value. Record the reviewer, decision, time and reason. If the source itself is ambiguous, allow an unresolved outcome and escalation. Forcing a reviewer to select a value can turn uncertainty into apparently clean data.
Review access must remain appropriate to the material. Show the evidence needed for the assigned task, with the applicable access controls and retention rules. A review queue is another place sensitive content can accumulate.
Tune the policy without hiding the workload
Choose candidate thresholds on labeled development or validation examples, then evaluate the fixed policy on a separate set. Include routine documents and the failure cases that should trigger a hold. Keep deliberately difficult stress tests distinguishable from the representative mix.
Google Document AI's evaluation guidance explains that raising a confidence threshold generally increases precision while reducing recall. A threshold that optimizes an extraction metric does not automatically optimize a review operation. The cost of an escaped error and the cost of an unnecessary review may be very different.
Alongside extraction quality, track:
- Incorrect results that continued without review, by critical field.
- The share and count routed to each destination.
- Reviewer corrections, unresolved cases and repeat submissions.
- Time waiting, time to resolve and backlog age.
- Outcomes by document family, layout and input condition.
Audit a policy-defined sample of automatically continued results as well as reviewed ones. Otherwise, the easiest errors to count will be those the queue already caught. Do not lower a safety-critical threshold simply to clear a backlog. Adjust capacity, narrow the permitted automated scope, or improve the input and extraction process.
Test the handoff as well as the extractor
Run end-to-end cases that exercise ownership and recovery: a reviewer is unavailable, the source is replaced, a corrected value fails another check, or a downstream delivery is retried. Confirm that the current decision is preserved and repeated delivery does not create duplicate work.
Version the routing rules with the extraction configuration. After changing the model, prompt, schema, preprocessing or document mix, rerun the relevant cases and reassess thresholds. Retain known failures as regression cases while reserving fresh examples for evaluation. The field-level OCR evaluation guide covers the underlying scoring method in more detail.
Where GritWorks fits
GritTransform extracts structured JSON from PDFs and images, supports source-based review, and offers optional per-field confidence scores and source references. GritRender creates controlled document variations with known values for testing extraction and downstream behavior. Use these capabilities to evaluate the evidence your policy needs. Queue orchestration, ownership and release rules remain part of the workflow you design and validate.
Start with one document type, one downstream action and a small set of exceptions with explicit expected outcomes. Discuss that workflow with GritWorks. Bring fictional examples initially, and agree an appropriate channel before sharing sensitive enterprise documents.
