Direct answer
A prompt-template mismatch is a serialization problem: the application may be sending valid text in a conversation structure that the selected model generation was not trained to interpret. Inspect the exact rendered request, not only the prompt-building code or a UI transcript. Check role boundaries, message order, end-of-turn markers, open assistant prefixes, system instructions, and any tool or modality tokens. The expected format is model-family and generation specific. Google’s Gemma 4 prompt-formatting documentation defines its control tokens and notes their tokenizer specificity; earlier Gemma generations use a different format. Never copy a template solely because two artifacts share a family name. Keep tokenizer identity and template content as separate but linked inputs. A correct tokenizer can still receive the wrong role structure; a familiar-looking template can be paired with an artifact whose tokenizer differs. Record artifact and tokenizer provenance, template revision, exact rendered bytes, runtime, and app build. If the integration does not expose token IDs, avoid claiming that the runtime tokenized a particular string in a particular way. Focus on the observable serialized request and the model-specific source documentation. Build a fixture with one-turn and multi-turn examples. Include a control prompt, a prior assistant response, a follow-up user message, and a fixed expected behavior such as whether the model should continue an open assistant prefix or start a new turn. Compare the output before and after a template change while keeping artifact, runtime, decode settings, and parser unchanged. Where a format includes special control markers, assert the exact spelling, position, and turn ending required by the matching generation docs. A readable transcript in a debug log may hide serialized markers or may be reconstructed from UI state; capture the exact representation actually submitted. Diagnose by stage. If the rendered request omits a role or places a turn-ending marker incorrectly, fix the serializer. If the request matches the documented template but output still differs, investigate artifact identity, supported task, runtime behavior, and decoding separately. If raw output is correct but the app displays an incomplete conversation, inspect parser and persistence logic. Do not describe a template correction as a model capability expansion or measured quality improvement without broader results. Treat each template revision as versioned compatibility data. Bind it to the intended artifact and tokenizer identities; add fixed regression fixtures before changing it; and review all public language or modality variants that use the same builder. Unsupported template fields should be rejected or omitted according to the official format rather than silently appended. The project has no complete device benchmark dataset, so the outcome here is reproducible request formatting and task behavior—not performance or universal conversational quality.
A new model answers the prompt but ignores the user role boundary
The engine initializes, but multi-turn replies include old assistant text or fail to start a fresh turn. The application copied a prompt builder from another model generation and has not saved the serialized request sent to the runtime.
Decision
Capture the final serialized conversation, compare it with the exact model-generation documentation, and change only the template while preserving other inputs.
Expected outcome
The team can prove whether role/control-token formatting changed and add a fixture that guards the corrected request structure.
Evidence to collect
The claim becomes reviewable only when the following evidence is attached to the same artifact and test run:
- Template and artifact revisions
- Rendered prompt hashes
- Known conversation outcomes
A release claim is valid only for the recorded configuration and acceptance set. Generalization requires additional records.
Implementation workflow
- Pin the artifact and format source. Record model artifact and tokenizer provenance, model generation, runtime, app build, and the official prompt-format document used. Do not infer that another model-family template is interchangeable.
- Capture the exact serialized request. Save a synthetic one-turn and multi-turn request after all prompt-builder transformations. Include role markers, message order, control tokens, turn endings, and open assistant prefix; redact private conversation content.
- Assert model-specific boundaries. Compare role and control-marker spelling and placement against the correct generation guide. Add assertions for each message boundary and final prefix, not just substring checks on the user-visible prompt.
- Isolate template from tokenizer and decoding. Keep artifact/tokenizer, runtime, stop configuration, sampling, and parser fixed while changing the template. If token IDs are not exposed, state that limitation and use serialized text evidence.
- Replay regression conversations. Run the same single-turn, multi-turn, and continuation fixtures before and after the change. Compare raw response, finalization state, parsed conversation, and persisted history as separate outputs.
Keep each transition observable. A failure should identify the stage, artifact, runtime, and recovery action without logging private user content.
Acceptance criteria
- The exact submitted conversation is captured and bound to artifact, tokenizer, template, runtime, and app-build identity.
- Role markers, control tokens, and turn endings are checked against the matching model-generation documentation.
- Template changes are isolated from tokenizer, decoding, parser, and runtime changes in regression runs.
- No universal model quality or performance claim is inferred from correcting prompt serialization.
Decision table
| Dimension | Choice or evidence | How to interpret it |
|---|---|---|
| Template mismatch | Wrong role/control-token/turn serialization | Model may interpret instruction and history incorrectly even when tokenizer and artifact are valid. |
| Tokenizer mismatch | Text is mapped using the wrong token vocabulary or asset | Identity pairing differs; correcting role markers alone cannot establish tokenizer provenance. |
| Parser or history bug | Raw response is correct but app state is stale or incomplete | Investigate after runtime output, not by changing the model template. |
Failure patterns to prevent
- Copying templates across families
- Editing prompts without regression tests
Also prevent silent fallback, unpinned artifacts, missing cancellation, and conclusions that combine unlike configurations. Store unsuccessful runs alongside successful ones.
Minimum reproducibility record
| Layer | Record |
|---|---|
| Device | Manufacturer, model, chipset, RAM class, operating-system build |
| Software | Application version and git commit |
| Model | Family, variant, revision, format, file length, hash, quantization |
| Runtime | Name, revision, requested backend, observed backend evidence |
| Workload | Fixture revision, input hash, prompt hash, output policy |
| Outcome | Completed, failed, cancelled, fallback, and privacy-safe diagnostics |
Release checklist
- ☐ The primary query is answered without an unsupported number.
- ☐ Every artifact and runtime is pinned.
- ☐ The representative task and failure policy are explicit.
- ☐ Lifecycle, cancellation, cleanup, and fallback are tested.
- ☐ User-content and network boundaries are documented.
- ☐ Result wording applies only to the recorded configuration.
- ☐ The page links to raw method or evidence when results are added.
Sources and related evidence
- Cove benchmark method
- Official Gemma 4 prompt formatting
- Official Gemma inference prompt formatting examples
Chinese deployment and troubleshooting content is organized in the .奇连 AI 端侧专题.
Frequently asked questions
Can I reuse one Gemma prompt template for every Gemma generation?
Do not assume so. Official prompt-format documentation can differ by generation, and control tokens may be tokenizer-specific. Verify the exact artifact generation and use its matching source.
Should I change the tokenizer when a model ignores instructions?
Not without evidence. First capture the exact serialized prompt and compare role boundaries and turn markers. Tokenizer identity is a separate check, and simultaneous changes make the failure harder to isolate.
Is a readable conversation log enough to prove the submitted format?
Not always. A debug transcript may omit control markers or reconstruct UI state. Capture the exact serialized representation at the runtime call boundary using synthetic input.