Bridge Protobuf Binary and ProtoJSON Without Hiding Loss

Direct answer

Protocol Buffers binary and ProtoJSON are different wire representations with different compatibility properties. Proto3 generated messages can preserve unknown binary fields when they are parsed and reserialized through APIs that retain the message’s unknown-field state. Converting a message to JSON does not preserve unknown fields in the same way; JSON parsing and printing use field names and JSON mappings, and a JSON round trip can reject or discard information that a newer binary producer added. Copying known fields one at a time into a new message can also drop unknown binary fields. State the exact conversion pipeline instead of saying that “protobuf preserves fields” without naming the format and APIs. ProtoJSON is a defined mapping, not a generic pretty-printer for arbitrary maps. Protobuf field names commonly become lowerCamelCase JSON keys, while parsers accept documented field-name forms; enums normally use symbolic names; bytes become base64 strings; and 64-bit integer families are emitted as decimal strings by default to avoid precision loss in binary64-oriented JSON consumers. Well-known types have special representations. A consumer that converts every value into a loosely typed JSON object can lose these distinctions or produce key collisions. Keep 64-bit values exact across each language boundary. JSON’s common number model cannot exactly represent every integer above 2^53. ProtoJSON therefore emits `int64`, `uint64`, `sint64`, and fixed-width 64-bit integers as decimal strings by default. Some parsers accept quoted or unquoted forms, but a generic JavaScript consumer may parse a bare number imprecisely. Test values around 2^53, signed extrema, unsigned extrema, negative values where allowed, and exponent spellings; do not round through `Double` or a JavaScript `Number` and then write the message back. Bytes and strings are not interchangeable. A protobuf `bytes` field is represented as base64 text in ProtoJSON, while a `string` is Unicode text with UTF-8 validity requirements in protobuf data. If an application decodes base64 into a string “for readability,” arbitrary bytes may be corrupted. Preserve a typed byte field or an explicitly labeled base64 form. Likewise, JSON `null` handling is a presence rule: ProtoJSON treats null as an unset field for ordinary fields, while wrappers and some well-known types have specific mappings. Explicit presence and default values can matter, so test both the message descriptor and generated API. Unknown enum values and unknown fields require version-aware tests. Proto3 enum openness, language runtime behavior, and any imported proto2 closed enums can affect how numeric values are represented or retained. ProtoJSON normally emits enum names, though supported options may emit numbers; a client with an older schema can reject a new symbolic name. Binary-compatible schema evolution does not imply JSON-compatible field renaming or enum deletion. Reserve removed field numbers and names according to the schema rules, and test old/new writer-reader pairs with the exact generated code. A usable bridge needs a declared purpose. If JSON is only for diagnostics, make the output explicitly lossy and never feed it back as an authoritative backup. If JSON is a required interchange contract, version it, pin the `.proto` descriptors and protobuf runtime, name the JSON printer/parser options, and define unknown-field behavior. If binary forward compatibility is required, retain the original binary bytes or use a message-oriented merge/copy path documented to preserve unknown fields. Do not convert binary to JSON and back while claiming the original message survived unchanged. Android builds may use different generated runtimes and APIs. Full protobuf Java, lite runtime, generated message APIs, JSON format utilities, and custom serializers do not expose identical reflection or unknown-field surfaces. Do not recommend `JsonFormat` as available in every Android artifact without checking the resolved dependency and method. Record the protoc/compiler version, generated-code mode, protobuf runtime coordinates, Android min API, parser/printer options, and actual method used. Compile and exercise the exact production configuration; documentation for one runtime is not a compile result for another. Build fixtures from a versioned `.proto` schema. Include a message written by a newer schema with an unknown field, round-trip it through binary-only parse/serialize and then through the proposed JSON bridge, and inspect what survives. Add 64-bit boundary values, bytes with non-UTF-8 octets, enum names and unknown numbers, absent versus defaulted fields, explicit null, maps with keys that could collide after string conversion, and well-known types such as Timestamp, Any, Duration, and FieldMask when used. Compare semantic message fields and wire bytes separately; binary serialization is not a canonical byte format unless a protocol defines one. Report a matrix of each path: binary parse to binary serialize, generated-message copy, binary to ProtoJSON, ProtoJSON to message, generic JSON object conversion, and any custom normalization. For each path state unknown-field retention, numeric exactness, presence behavior, enum handling, and error outcome. A syntactically valid JSON result does not prove round-trip equivalence, authorization, or semantic correctness. This guide is an interoperability method, not a claim that Cove stores protobuf records or implements any particular Android protobuf runtime.

An older Android client edits a message from a newer producer

A binary protobuf message contains a newly added field that the installed client’s schema does not know. The app turns the message into JSON for review, then reconstructs a message from visible known fields and saves it. The new field may disappear even though the binary parser originally accepted the message.

Decision

Choose whether the JSON view is diagnostic or authoritative interchange. If unknown binary fields must survive, retain original bytes or use a tested message-preserving path; never round-trip through ProtoJSON and promise no loss.

Expected outcome

The export/import report identifies exact retained and discarded information for every conversion path and pins the schema and runtime used.

Evidence to collect

The claim becomes reviewable only when the following evidence is attached to the same artifact and test run:

• Pinned .proto/protoc/runtime identity
• Old/new unknown-field and enum fixtures
• 64-bit, bytes, presence, and JSON conversion results

Task quality needs a versioned corpus and acceptance rules. Fluent sample output is not an accuracy result.

Implementation workflow

1. Map the actual generated runtime path. Record `.proto` revision, protoc version, full or lite generated mode, resolved Android runtime, parser/printer options, and each conversion API. Compile and invoke the exact methods instead of inferring that an API exists from general protobuf documentation.
2. Decide which representation is authoritative. Use binary preservation for forward-compatible opaque interchange when the selected message API retains unknown fields. Mark human-readable JSON as diagnostic unless the product has a versioned ProtoJSON contract and accepts its defined loss boundaries.
3. Preserve exact type mappings. Keep 64-bit integers in the specified decimal-string mapping through languages that cannot exactly represent all integers as JSON numbers. Preserve bytes as bytes/base64, enum policy, explicit presence, null semantics, map-key types, and well-known type mappings.
4. Exercise old/new schema transitions. Build fixtures where a newer writer adds unknown fields and enum values, an older reader consumes them, and each binary and JSON conversion is attempted. Compare known fields, unknown-field state where exposed, errors, numeric values, and original bytes independently.
5. Label lossy paths and report limitations. Publish the path-by-path result: binary parse/serialize, message copy, ProtoJSON print/parse, generic JSON conversion, and custom normalization. Never claim semantic equality or canonical bytes from a readable JSON rendering alone.

Keep each transition observable. A failure should identify the stage, artifact, runtime, and recovery action without logging private user content.

Acceptance criteria

• Unknown-field preservation is stated only for a tested binary message path; ProtoJSON conversion and field-by-field copies are tested as potentially lossy.
• The exact Android protobuf runtime, generated-code mode, compiler, conversion API and options compile and execute in the declared build.
• Fixture results preserve exact 64-bit values, distinguish bytes from text, cover enum and presence/null behavior, and identify well-known-type and map-key conversion limits.
• Old/new schema compatibility is reported separately for binary and JSON, including field-name and enum-value changes.
• Lossy diagnostic JSON cannot be mistaken for an authoritative backup or a guarantee of message semantic equivalence.

Decision table

Failure patterns to prevent

• Calling ProtoJSON lossless binary preservation
• Rounding 64-bit values through generic JSON numbers
• Assuming full-runtime reflection APIs exist in Android lite builds

Also prevent silent fallback, unpinned artifacts, missing cancellation, and conclusions that combine unlike configurations. Store unsuccessful runs alongside successful ones.

Minimum reproducibility record

Release checklist

• ☐ The primary query is answered without an unsupported number.
• ☐ Every artifact and runtime is pinned.
• ☐ The representative task and failure policy are explicit.
• ☐ Lifecycle, cancellation, cleanup, and fallback are tested.
• ☐ User-content and network boundaries are documented.
• ☐ Result wording applies only to the recorded configuration.
• ☐ The page links to raw method or evidence when results are added.

Sources and related evidence

• Android privacy and security
• Protocol Buffers — Proto3 Language Guide and unknown fields
• Protocol Buffers — ProtoJSON Format
• Protocol Buffers — Field Presence Application Note
• Protocol Buffers — Serialization Is Not Canonical

Chinese deployment and troubleshooting content is organized in the .奇连 AI 端侧专题.

Frequently asked questions