Direct answer
A phone showing Arabic menus does not prove that an offline AI model can translate Arabic, and a short phrase does not establish long-text behavior. A useful test must cover both language directions, an actual Arabic source corpus, a checked target reference, long passages, paragraph boundaries, mixed digits and punctuation, and the app’s right-to-left presentation at the user’s chosen text size. Memory and context limits can cause truncation or chunk-boundary errors even when each short sentence looks plausible. Android’s layout direction and font scaling are UI behaviors; they do not guarantee that model output, selection, punctuation, or translation meaning is correct. This page does not claim that Cove or another named app supports Arabic. It describes how to evaluate a particular local model and Android build: pin the artifact, run a versioned long-text fixture offline, preserve the original and translated text, test large system font settings, and report every incomplete or malformed result.
Scenario: reading a long Arabic notice on a phone
A user photographs or pastes a multi-paragraph Arabic notice and enlarges the interface text for readability. The passage contains right-to-left text, a street name, times, digits, punctuation, and an important condition. They need to read the translation comfortably without losing the source or the relation between paragraphs.
Decision
Test language direction, text processing, and visual layout separately. Decide which numbers, names, conditions, and paragraph relationships must survive translation. Include large font settings and a narrow screen before deciding the app is suitable.
Expected outcome
The reviewer can identify whether failure came from unsupported Arabic input, truncation, translation changes, broken RTL presentation, or an unreadable layout. The conclusion applies to the exact fixture, app version, model, and font setting tested.
Evidence to collect
The claim becomes reviewable only when the following evidence is attached to the same artifact and test run:
- Arabic source and reviewed target fixtures
- RTL and font-scale screenshots
- Chunk, completion, failure, and recovery trace
Task quality needs a versioned corpus and acceptance rules. Fluent sample output is not an accuracy result.
Implementation workflow
- Confirm the requested language route rather than the UI locale. Record the app interface locale, source and target language, and translation direction as separate settings. Test Arabic-to-target and target-to-Arabic independently. A localized menu only proves that strings exist in the UI; it does not prove that the underlying model accepts Arabic or returns a reliable translation for the selected task.
- Build a long Arabic fixture with reviewed references. Use a consent-safe passage with several paragraphs and have a fluent Arabic reader verify the source and target references. Include names, dates, digits, punctuation, quoted text, and terms whose meaning depends on the surrounding sentence. Keep a short baseline and a longer fixture so truncation and context behavior can be distinguished from basic language handling.
- Record how the app handles long input. Pin the exact model artifact and runtime, then document whether the app rejects, truncates, or splits a passage. If it chunks text, record the boundary rule and how it carries context across paragraphs. Compare the produced output with the full reference, not just the first visible screen, and preserve any omitted or duplicated segment.
- Test right-to-left layout with accessibility text scaling. Run the same fixture with the device’s large-text setting and a narrow display. Check reading order, alignment, cursor movement, selection handles, punctuation, mixed Latin digits, copy, and export. Android’s RTL support and font scaling need to be exercised in the actual view hierarchy; a localized screenshot or a string direction flag is not a complete layout test.
- Score meaning and presentation separately. Have a qualified reviewer inspect omitted clauses, changed numbers, names, negation, and paragraph relationships. Separately note clipping, reordering, overlap, and text that becomes inaccessible at larger font settings. If OCR is part of the workflow, save recognized Arabic text before translation so recognition errors are not counted as translation errors.
- Verify the complete task offline and report its limits. Complete setup first, disable connectivity, and repeat the same long-text task through input, translation, correction, and export. Record failures, cancellation, memory limits, and recovery without inventing universal thresholds. State clearly which direction, input type, screen scale, and passage length were evaluated and what remains unknown.
Keep each transition observable. A failure should identify the stage, artifact, runtime, and recovery action without logging private user content.
Acceptance criteria
- UI locale, Arabic source or target language, and translation direction are recorded separately.
- A reviewed long-text fixture checks paragraph boundaries, names, numbers, negation, and punctuation.
- RTL order, selection, copy/export, and large font behavior are tested on the actual screen layout.
- The full task is repeated offline and the result is bounded to the tested model, app, fixture, and display setting.
Decision table
| Dimension | Choice or evidence | How to interpret it |
|---|---|---|
| Language selection | Verify source and target direction with Arabic fixtures | A translated interface is not evidence of Arabic model input or output quality. |
| Long input | Compare short baseline and multi-paragraph fixture with chunk trace | Reveals context truncation, repeated boundaries, or omitted paragraphs. |
| RTL and large text | Test reading order and interactions at system font scale | A technically correct translation can still be unusable if layout clips or reverses content. |
| Offline result | Disable connectivity after setup and complete input-to-export | Separates model execution from any hidden network-dependent stage. |
Failure patterns to prevent
- Assuming locale support proves translation quality
- Testing short phrases instead of long passages
- Ignoring mixed digits, punctuation, and layout direction
Also prevent silent fallback, unpinned artifacts, missing cancellation, and conclusions that combine unlike configurations. Store unsuccessful runs alongside successful ones.
Minimum reproducibility record
| Layer | Record |
|---|---|
| Device | Manufacturer, model, chipset, RAM class, operating-system build |
| Software | Application version and git commit |
| Model | Family, variant, revision, format, file length, hash, quantization |
| Runtime | Name, revision, requested backend, observed backend evidence |
| Workload | Fixture revision, input hash, prompt hash, output policy |
| Outcome | Completed, failed, cancelled, fallback, and privacy-safe diagnostics |
Release checklist
- ☐ The primary query is answered without an unsupported number.
- ☐ Every artifact and runtime is pinned.
- ☐ The representative task and failure policy are explicit.
- ☐ Lifecycle, cancellation, cleanup, and fallback are tested.
- ☐ User-content and network boundaries are documented.
- ☐ Result wording applies only to the recorded configuration.
- ☐ The page links to raw method or evidence when results are added.
Sources and related evidence
- On-device AI architecture overview
- Cove benchmark method
- Android manifest support for right-to-left layouts
- Android Compose guidance for scalable content and font scaling
- Android system font scaling and accessibility behavior in Android 14
Chinese deployment and troubleshooting content is organized in the .奇连 AI 端侧专题.
Frequently asked questions
Does an Arabic option in Android prove the AI can translate Arabic offline?
No. Android or the app may localize its interface while the model’s input and output language coverage remains different. Test both translation directions with Arabic examples, an installed artifact, network disabled, and a reviewed reference.
Why test Arabic with large text and long passages?
Arabic rendering is right-to-left, long input can reach model context or memory limits, and larger font settings can reveal clipped or reordered output. These are separate risks; a short text sample in the default font will not expose them.
Should Arabic OCR be scored as part of translation accuracy?
Keep it separate when possible. Save the recognized source text, compare it with a human-checked transcription, and then score the translation. Otherwise a missed word from the image can be mistaken for a translation-model error.
Can a benchmark on one Android phone prove Arabic support for every device?
No. A result is limited to the exact artifact, runtime, OS, input fixture, language direction, and layout tested. Other devices and app builds need their own evidence, especially when memory behavior or text layout changes.