Direct answer
Local AI request replay centers on running privacy-safe fixtures under pinned artifact, prompt, parser, and runtime identity so that each call mirrors the original setup. Anything outside that fixed quartet must remain unsupposed.
Evidence to collect
The claim becomes reviewable only when the following evidence is attached to the same artifact and test run:
- Fixture revision
- Configuration manifest
- Ordered output and failure records
A release claim is valid only for the recorded configuration and acceptance set. Generalization requires additional records.
Implementation workflow
Each run is then documented alongside the fixture revision and the configuration manifest, capturing ordered output and failure records. Comparing these captures across runs makes drift visible without altering the fixed identity.
- State the decision the test must support.
- Pin device, software, artifact, runtime, and backend.
- Version the workload and expected properties.
- Separate cold, representative, and sustained phases.
- Keep ordered raw records including failure.
- Publish method and evidence before conclusions.
Failure patterns to prevent
- Replaying private user content
- Changing multiple variables together
Two pitfalls merit attention: replaying private user content, and changing multiple variables together. Release wording must stay grounded in recorded evidence rather than extend beyond it, anchored to the BENCHMARK-01 and PRIVACY-01 boundaries.
Minimum reproducibility record
| Layer | Record |
|---|---|
| Device | Manufacturer, model, chipset, RAM class, operating-system build |
| Software | Application version and git commit |
| Model | Family, variant, revision, format, file length, hash, quantization |
| Runtime | Name, revision, requested backend, observed backend evidence |
| Workload | Fixture revision, input hash, prompt hash, output policy |
| Outcome | Completed, failed, cancelled, fallback, and privacy-safe diagnostics |
Release checklist
- ☐ The primary query is answered without an unsupported number.
- ☐ Every artifact and runtime is pinned.
- ☐ The representative task and failure policy are explicit.
- ☐ Lifecycle, cancellation, cleanup, and fallback are tested.
- ☐ User-content and network boundaries are documented.
- ☐ Result wording applies only to the recorded configuration.
- ☐ The page links to raw method or evidence when results are added.
Sources and related evidence
Chinese deployment and troubleshooting content is organized in the .奇连 AI 端侧专题.