Direct answer
A local AI result cache is a reuse layer for derived output, not the permanent record of what the user asked or approved. Its job is to avoid repeating work only when the current request is equivalent under an explicit policy. A cache hit says that a stored output matched the key and freshness rules your application defined. It does not prove that the output is correct, safe, or still appropriate for the user. Keep durable conversation history and provenance in their own product records, and let the cache be disposable. A useful key includes every input that can change meaning: canonical source content or a stable source revision, task identifier, language direction, relevant settings, model artifact revision, tokenizer or template revision, prompt version, parser/schema version, and output format. Do not place raw sensitive text in a visible key or log. A keyed digest can reduce exposure, but it still needs a deliberate threat model and collision handling. If the application cannot identify an input revision reliably, it should avoid reusing the result or ask the user rather than treating a partial key as equivalent. Separate identity from freshness. The key answers whether the request belongs to the same equivalence class; an expiry or invalidation policy answers whether a previously stored result may still be reused now. User edits to the source, changed model or prompt, parser fixes, changed task settings, deleted source records, and revoked consent are explicit invalidation events. Time-to-live can bound reuse, but a short age alone cannot repair a key that omitted a meaning-changing input. If the task depends on rapidly changing context, a cache may be inappropriate even when the input text is identical. Cache only completed outputs that passed the application’s structural validation and task-specific acceptance checks. Do not cache a timeout, parser failure, cancelled generation, partial stream, or generic error as if it were a successful answer. If a task permits user review, preserve that distinction: an unreviewed generated result must not satisfy a workflow that requires a user-confirmed result. A cache entry may store a status that requires review, but a hit cannot silently upgrade it to approved. Treat cached results as derived user data. Decide where they are stored, whether they are backed up, how long they remain, how they are deleted with the source record, and whether export includes them. Minimize stored fields and avoid copying source text into diagnostic metadata. A local-only inference path does not prove that every other application feature has no network traffic, and an in-app cache does not automatically inherit the privacy behavior of an HTTP cache. Review storage and network paths separately. Concurrent requests need a defined policy too. If two identical requests arrive together, the coordinator may share one in-flight operation, queue one behind the other, or allow separate work; it should not persist duplicate entries or cross-wire results from different keys. Record the key version and cache decision in safe diagnostics. A cache schema change should invalidate or migrate entries deliberately, not reinterpret them under new semantics. Design bounded cleanup and observable miss reasons. A user should be able to distinguish expired, invalidated, evicted, unavailable, and never-cached states where that affects the workflow. Capacity eviction is a storage policy, not a claim that the result was obsolete. Keep an entry only as long as it can serve its defined purpose, and ensure source deletion or user reset removes associated derived results. Tests should cover key changes, invalidation triggers, expiry, failures, concurrency, and deletion without publishing unsupported accuracy or performance claims.
A user edits text and repeats a local summary request
A notes application offers a local summary action. The user runs it once, edits two sentences, changes the requested output style, and taps the action again. The team wants to reuse completed work where safe, but an old summary must not appear as if it describes the edited note or the new output contract.
Decision
Include source revision and all meaning-changing task configuration in the reuse key, invalidate entries when the note changes, and show whether a returned result is cached and still requires review.
Expected outcome
The second request cannot reuse the summary for the earlier note revision. Unchanged equivalent requests may reuse a valid entry under the stated expiry policy, without claiming the output is accurate merely because it was cached.
Evidence to collect
The claim becomes reviewable only when the following evidence is attached to the same artifact and test run:
- Cache key schema
- Source and artifact references
- Deletion and rebuild tests
A device badge must come from an exact device, OS, artifact, runtime, backend, and workload record, not parameter-count intuition.
Implementation workflow
The cache key schema, the source and artifact references, and the deletion and rebuild tests form a single comparative record. These evidence items should be documented together so that a rebuilt entry can be matched against the original on the same key, source, artifact, and prompt basis, and any mismatch in the recorded values can be identified directly from that comparison without extending the scope. (62 words)
- Define what counts as the same request. List source identity and revision, task, language direction, settings, model artifact, prompt/template, parser/schema, output format, and any user context that changes meaning. If a necessary factor is unknown, disable reuse for that case.
- Build a privacy-conscious, versioned key. Serialize the selected fields deterministically and include a key-schema version. Avoid raw source text in keys or logs; if using a digest, specify its input encoding, collision response, and whether the digest itself could reveal a guessable input.
- Specify freshness and invalidation separately. Choose an expiry policy appropriate to the task and enumerate explicit invalidation events such as source edit, model or prompt update, parser change, source deletion, or consent revocation. Do not use expiry as a substitute for missing key fields.
- Persist only eligible completed outcomes. Write an entry only after generation finishes and the output passes structural and task acceptance checks. Keep cancelled, partial, timed-out, and malformed outputs out of the success cache; preserve a review-required state when a human decision is part of the workflow.
- Coordinate duplicate requests and cleanup. Choose whether equivalent in-flight calls are coalesced, queued, or independent, and ensure each result remains associated with its exact key. Bound storage, remove entries on source deletion or user reset, and classify eviction separately from invalidation.
- Test reuse boundaries and explain the result. Use fixtures that vary one key component at a time, then exercise expiry, edits, model/prompt/parser changes, failed output, concurrency, deletion, and restart. In the interface, explain a cache reuse decision when it affects user trust without implying a correctness guarantee.
Acceptance criteria
- The cache key includes every documented input that can change task meaning and has an explicit schema version.
- Expired, invalidated, failed, partial, and review-required outcomes cannot be silently reused as approved successful results.
- Source deletion, user reset, storage eviction, and concurrent identical requests have distinct, tested behavior; diagnostics do not expose raw input content.
Decision table
| Dimension | Choice or evidence | How to interpret it |
|---|---|---|
| Durable result history | Retain past outputs with their model and prompt provenance | Answers what was previously produced and when; it does not decide whether that output can satisfy a new request. |
| Result cache | Reuse an output when a complete key and freshness policy match | Optimizes repeated equivalent work; entries may be expired, invalidated, evicted, or deleted and do not establish correctness. |
| Regenerate every request | Always run the selected task again | Avoids stale reuse but repeats work; it still needs failure, review, privacy, and result-history handling. |
Failure patterns to prevent
- Caching without provenance
- Keeping results after source deletion
Caching without provenance, and keeping results after source deletion, undermine the integrity of the recorded evidence and must be avoided. Release wording describing this fact card should rely solely on what has been documented, and the cache entries referenced under the privacy and coverage fact-card boundaries should be described only through the recorded schema, references, and tests. (57 words)
Minimum reproducibility record
| Layer | Record |
|---|---|
| Device | Manufacturer, model, chipset, RAM class, operating-system build |
| Software | Application version and git commit |
| Model | Family, variant, revision, format, file length, hash, quantization |
| Runtime | Name, revision, requested backend, observed backend evidence |
| Workload | Fixture revision, input hash, prompt hash, output policy |
| Outcome | Completed, failed, cancelled, fallback, and privacy-safe diagnostics |
Release checklist
- ☐ The primary query is answered without an unsupported number.
- ☐ Every artifact and runtime is pinned.
- ☐ The representative task and failure policy are explicit.
- ☐ Lifecycle, cancellation, cleanup, and fallback are tested.
- ☐ User-content and network boundaries are documented.
- ☐ Result wording applies only to the recorded configuration.
- ☐ The page links to raw method or evidence when results are added.
Sources and related evidence
- Android Developers: Build an offline-first app and separate data sources
- IETF RFC 9111: HTTP cache keys, freshness, validation, and invalidation concepts
Chinese deployment and troubleshooting content is organized in the .奇连 AI 端侧专题.
Frequently asked questions
Does a cache hit mean the generated answer is correct?
No. It means the stored entry matched the application’s key and freshness policy. Correctness still depends on the task, output validation, and any required user review; the hit alone adds no quality evidence.
Is a time-to-live enough to make a cache key safe?
No. Expiry bounds how long an entry may be reused, but it cannot compensate for a key that omitted source revision, prompt, model, parser, language direction, or another meaning-changing input.
Should a failed or cancelled generation be cached to avoid retry loops?
Do not store it as a successful result. Keep a separate failure record if useful, with a classified reason and retry policy, while ensuring a later valid request can run and produce its own entry.