LiteRT-LM vs llama.cpp on Android: Choose the Runtime First

LiteRT-LM and llama.cpp are different Android deployment paths, not interchangeable loaders for the same file. LiteRT-LM belongs to Google’s LiteRT stack and uses its own model packaging and APIs. llama.cpp is a C/C++ inference project centered on GGUF. Choose the runtime before committing to a model artifact, because a .litertlm package does not become a GGUF model by changing its extension, and a GGUF file is not a LiteRT-LM package.

The runtime decision comes before the UI

Teams often start with a model file they found online and then search for an Android library that can open it. That reverses the durable order. A production decision should begin with:

1. the task the app must complete;
2. the Android devices and system versions it must support;
3. the runtime and execution paths the team can test;
4. the artifact format that runtime accepts;
5. the model variant available in that format.

The runtime affects packaging, native dependencies, acceleration, memory behavior, logging, cancellation, and the release process. It is part of the product architecture, not a utility hidden behind one function call.

Model packaging: .litertlm and GGUF are separate ecosystems

LiteRT-LM is developed publicly by Google AI Edge and uses its own packaging and runtime APIs. The package represents a deployment artifact prepared for the LiteRT-LM toolchain. Its exact contents and size depend on the model and conversion path.

llama.cpp is an open-source C/C++ inference project whose primary model format is GGUF. GGUF stores model tensors and metadata for the llama.cpp ecosystem and related tools.

This format split has practical consequences:

• model availability may differ between the two ecosystems;
• conversion and quantization tools differ;
• metadata and tokenizer packaging differ;
• the same model family name does not prove identical weights or output;
• an evaluation result for one artifact cannot automatically validate another.

When comparing outputs, record the full artifact identity. “Gemma on Android” is not enough. State the model variant, revision, quantization, format, runtime revision, and prompt configuration.

Acceleration: compare supported paths on your devices

LiteRT evolved from TensorFlow Lite into Google’s cross-platform on-device AI framework. Its official description covers CPU, GPU, and NPU execution across supported platforms. That establishes the intended acceleration surface, not a guarantee that every model operation will use every accelerator on every Android phone.

Google’s Gemma run guide lists LiteRT-LM as an open-source framework for Android and iOS deployment and describes CPU, GPU, and NPU paths. The correct implementation question is therefore “which path did this exact artifact use on this exact device?” rather than “does this phone have an NPU?”

llama.cpp controls execution through its own C/C++ runtime and the backends exposed by the project. Its portability is tied to that runtime and GGUF ecosystem. Before treating a backend as part of your Android support promise, reproduce the build and run it on the devices you intend to support.

Never choose between the runtimes from a headline performance number. Use the same task set, equivalent model variants where possible, the same output limits, and a recorded thermal state. If artifacts differ, present the comparison as a product-path comparison rather than a pure runtime benchmark.

Android integration boundary

Both runtimes should sit behind an application-owned inference interface. The rest of the app should not know how a LiteRT-LM session or llama.cpp context is represented.

interface LocalInferenceEngine {
    suspend fun load(): LoadResult
    suspend fun generate(request: GenerationRequest): GenerationResult
    suspend fun cancel()
    suspend fun close()
}

This boundary makes several changes survivable:

• replacing one model package with another;
• adding a CPU fallback;
• changing cancellation behavior;
• testing the UI with a fake engine;
• collecting comparable lifecycle measurements;
• evaluating a second runtime without rewriting product screens.

The engine implementation should own serialization of requests, model lifecycle, concurrency, cancellation, and error normalization. A ViewModel should own user-visible state. A Composable should not load native model state during recomposition.

Portability means more than “runs on several platforms”

LiteRT-LM and llama.cpp both address more than one environment, but they carry different artifacts and APIs. Portability should be evaluated at four layers:

If a team ships Android first but expects desktop or iOS later, it should prototype the second platform before freezing its format decision. A theoretically portable repository does not eliminate platform-specific packaging, lifecycle, permissions, or UI work.

Debugging and release maintenance

For either runtime, preserve enough information to reproduce a failure:

• model family and exact artifact revision;
• format and quantization;
• artifact length and hash when available;
• runtime revision and build flags;
• Android version and chipset;
• selected execution path;
• input that triggered the failure, after privacy-safe redaction;
• lifecycle state, such as first load, resume, or repeated generation.

Cove’s current Gemma 4 E2B configuration illustrates artifact-level traceability: the LiteRT-LM download is pinned to a Hugging Face revision and records an expected length of 2,583,085,056 bytes. That fact helps reproduce delivery and storage behavior. It does not establish a llama.cpp comparison or a universal Gemma package size.

Maintain a small acceptance suite that runs whenever the artifact, runtime, build configuration, or prompt changes. Compilation proves that APIs line up; it does not prove output quality, sustained memory behavior, cancellation, or device compatibility.

Selection checklist

Choose LiteRT-LM for evaluation when:

• the target model is available or supportably convertible for the LiteRT-LM path;
• the team wants to evaluate Google’s LiteRT execution stack;
• Android and iOS are the primary deployment targets;
• the team can test the documented acceleration paths on target devices.

Choose llama.cpp for evaluation when:

• the required model is maintained in a suitable GGUF form;
• the team wants a C/C++ runtime under its own build and release control;
• sharing a GGUF-oriented workflow with desktop environments matters;
• the team is prepared to own the native integration and backend verification.

Evaluate both when format availability or device behavior remains uncertain. Do not ship both merely to avoid making a decision: two runtimes increase binary surface, testing combinations, model storage, and lifecycle complexity.

A fair proof-of-concept

Use one narrowly defined task and create two minimal engine implementations behind the same interface. Record artifact identity, load outcome, representative output, peak memory, cancellation, repeated use, and the actual execution path. Treat any missing field as an unanswered question.

For memory methodology, continue with on-device AI RAM requirements. For architecture trade-offs, see on-device AI vs cloud AI. Chinese readers can use the implementation-focused LiteRT-LM Android tutorial.

Last reviewed: 2026-09-16.