Validate ML Kit OCR Reading Order for Complex Pages

Direct answer

ML Kit Text Recognition V2 returns a hierarchy, not just one string: a Text has TextBlocks, blocks contain Lines, lines contain Elements, and elements may contain Symbols. The API exposes recognized text and geometry for these levels, including axis-aligned bounding boxes and four corner points. Keep the raw result, the library's order, and any application-created order as separate values. A field used for a downstream action should be traceable to the original OCR element or line, not only to a sorted string. The API documents language-aware reading order: Text.getText concatenates blocks, and for Latin it describes top-to-bottom within TextBlock and left-to-right within Text.Line. Honor that as API output, not proof that every two-column, table, footnote, or form layout matches a downstream user's intended projection. Test the actual page class and configured script; neither claim that all OCR lists are unordered nor that they always match the task. Geometry helps describe the recognizer's regions but does not supply reading logic by itself. TextBlock, Line, and Element expose bounding boxes and corner points; corner points are clockwise starting at the top-left, yet perspective can make them non-rectangular and some can lie outside the image. An axis-aligned rectangle can overlap other columns or table cells. Do not sort every region by its top-left y then x and call that natural reading order: row interleaving, spanning headings, vertical writing, right-to-left scripts, and captions can make that heuristic wrong. Preserve the original points and the chosen image coordinate basis. Define a task-specific order only after naming the document contract. For a two-column page, a product might read the left column top to bottom and then the right column; another task might require row-wise ordering across columns. A form may need label-value pairs rather than one linear transcript. Keep the recognizer's text intact, produce a separate projection with its own stable region IDs and rule version, and allow review when geometry cannot establish an unambiguous order. A derived order must not overwrite the original recognition or invent missing punctuation and cells. Build fixtures with human-reviewed references: single-column paragraphs, clear two-column pages, headings spanning columns, tables, marginal notes, skewed or perspective text, empty regions, and mixed scripts that the selected recognizer actually supports. Compare the raw Text.getText output and each TextBlock/Line/Element against expected region membership and intended task order. Store source image identity, model/library package, selected script options, returned hierarchy, corner points, and the application projection separately. If a script is not listed for the selected API or its output is not verified, record unsupported or unknown rather than inferring right-to-left behavior from geometry. The Android ML Kit guide requires API 23 or higher for Text Recognition V2 and offers bundled or unbundled model delivery. Pin the dependency and recognizer options because the installed package and supported scripts affect the actual path. Do not fabricate a confidence value at the block level or use a zero/missing optional signal as a substitute for line or element evidence. This method evaluates ordering and association, not OCR accuracy, language coverage beyond the configured recognizer, or universal reading behavior. Acceptance depends on task. Keep raw text and region IDs beside any reordered projection; when columns or labels are ambiguous, ask for review rather than silently rearranging. Version the projection and the fixture that justifies it. This is a generic Android ML Kit method, not a Cove integration claim. Cove's current multimodal OCR field does not establish an ML Kit recognizer or multi-column ordering path. Pin the recognizer, script options, and projection version.

A two-column notice is read in an order that breaks its task

An Android document tool recognizes a page with a heading spanning two columns, body paragraphs, a boxed table, and a footnote. The OCR text is legible, but an automatic field action consumes regions in a sequence that interleaves columns and attaches a value to the wrong label.

Decision

Preserve ML Kit's raw hierarchy and documented language order, then validate a separate, versioned projection for the exact page task using region geometry and reviewed fixtures.

Expected outcome

The user can inspect the source region behind each projected item; ambiguous layouts remain in raw order or require review instead of silently changing meaning.

Evidence to collect

The claim becomes reviewable only when the following evidence is attached to the same artifact and test run:

• Text and region-geometry fixtures
• Versioned reading-order projection
• Ambiguous-layout review outcomes

Task quality needs a versioned corpus and acceptance rules. Fluent sample output is not an accuracy result.

Implementation workflow

1. Pin the recognizer and script contract. Record ML Kit Text Recognition V2 dependency, Android API level, bundled or unbundled delivery, selected script options, input path, and image transform version. Test only language and layout behavior supported by the exact configured recognizer.
2. Preserve the complete raw hierarchy. Store Text.getText plus each returned TextBlock, Line, Element, their original list positions, text, bounding box, and corner points. Keep this raw result immutable; do not replace it with a sorted concatenation or inferred form value.
3. Build region geometry without assuming rectangles. Use documented corner points and bounding boxes where available. The APIs can return null when nothing is found; preserve that as unavailable geometry rather than inventing a zero rectangle or dereferencing missing points. Retain out-of-image points and mark overlaps, spanning headings and perspective as evidence, not automatic order.
4. Define the task's reading-order rule. Write a versioned rule for the target document: column-wise, row-wise, label/value, or human-reviewed sequence. The rule consumes raw region IDs and produces a separate projection. Do not generalize Latin Text.getText ordering to unsupported scripts or every multi-column page.
5. Compare against reviewed layout fixtures. Include single and multiple columns, spanning headings, tables, side notes, skew, partial text, and supported mixed scripts. Compare raw API order, region geometry, and expected task order; keep recognition errors separate from projection errors.
6. Handle uncertainty before actions. When regions overlap or a label/value pairing is unclear, expose the raw text and ask for correction before save, search, or automated action. Log fixture and rule IDs, not private page content, and never silently overwrite the source recognition.

Keep each transition observable. A failure should identify the stage, artifact, runtime, and recovery action without logging private user content.

Acceptance criteria

• The exact ML Kit dependency, Android API level, delivery option, and supported script configuration are recorded; unsupported scripts or outputs remain unknown.
• Raw Text, TextBlock, Line, Element, and geometry output is retained separately from any app-generated order or field projection.
• Reading-order tests state which API guarantee they use and validate task-specific layouts; no generic y/x sort is treated as universal human reading order.
• Ambiguous or overlapping regions are reviewable, and the result preserves provenance to source region IDs without claiming OCR accuracy or Cove integration.

Decision table

Failure patterns to prevent

• Sorting all boxes by y/x as universal reading order
• Replacing raw OCR with a projection
• Claiming unsupported script or Cove integration

Also prevent silent fallback, unpinned artifacts, missing cancellation, and conclusions that combine unlike configurations. Store unsuccessful runs alongside successful ones.

Minimum reproducibility record

Release checklist

• ☐ The primary query is answered without an unsupported number.
• ☐ Every artifact and runtime is pinned.
• ☐ The representative task and failure policy are explicit.
• ☐ Lifecycle, cancellation, cleanup, and fallback are tested.
• ☐ User-content and network boundaries are documented.
• ☐ Result wording applies only to the recorded configuration.
• ☐ The page links to raw method or evidence when results are added.

Sources and related evidence

• ML Kit text recognition
• Google ML Kit: Text Recognition V2 on Android, hierarchy and script options
• ML Kit Text API: text blocks and reading order
• ML Kit Text.TextBlock API: lines, bounding boxes, and corner points
• ML Kit Text.Line API: line order and elements
• ML Kit Text.Element API: element geometry

Chinese deployment and troubleshooting content is organized in the .奇连 AI 端侧专题.

Frequently asked questions