On-Device AI Thermal Throttling: Test Sustained Performance

On-device AI thermal throttling is a sustained-work problem. The first successful request tells you that a model can run; it does not tell you whether response time, execution path, memory, or failure rate changes after repeated work. Use a fixed workload, preserve every run in sequence, record available thermal context, and publish methodology before conclusions.

Define “sustained” for the product

A camera translator, meeting transcriber, document assistant, and local chat app stress hardware differently. Define the session users actually create:

• number and spacing of requests;
• input modality and size;
• maximum output policy;
• whether the model stays loaded;
• idle time between actions;
• screen, camera, and audio activity;
• expected session duration;
• allowed fallback behavior.

Do not import a stress-test loop that has no relationship to the feature. A synthetic loop can reveal a limit, but it cannot by itself describe user experience.

Freeze the workload and environment

The repeated input, prompt, output limit, preprocessing, artifact, runtime, backend, application build, and device software must remain fixed. Record battery state, charging state, ambient context when available, screen behavior, case or enclosure assumptions, and other active workloads.

If the operating system changes execution path during the sequence, mark that transition. A later run completed on a fallback path is not directly comparable with the first run on the preferred path.

Use a release build. Debug instrumentation is useful for diagnosis but can change timing and allocation behavior. If you use both, label them as different experiments.

Preserve the run sequence

A single successful inference does not establish sustained performance. Repeat a fixed workload while recording elapsed time, memory, device temperature or available thermal signals, execution path, and failures. Preserve the raw order rather than reporting only an average.

A sequence can reveal:

• slower initialization after resource pressure;
• gradual response-time change;
• backend fallback;
• allocation failure;
• cancellation delay;
• process termination;
• recovery after an idle interval.

An average can hide all of these. Publish the raw series or a summary that shows first, middle, last, worst, and failure runs with the complete method.

Thermal signal and performance are different measurements

Android publishes thermal status guidance and APIs. A benchmark should name which platform signal it collected instead of converting one device’s observation into a universal temperature threshold.

Temperature or a platform thermal status is context. Completion time and failure are outcomes. Do not infer one from the other without recording both. A warm device may still meet the product target; a cool-looking device may have already switched execution path.

Avoid claiming a universal temperature threshold. Device sensors, reporting, enclosures, cooling design, and platform policies differ. The correct release decision is whether the tested device and workload remain within the product’s defined acceptance criteria.

Memory belongs in the same timeline

Peak memory includes weights, runtime buffers, KV cache, input tensors, operating-system headroom, and allocator behavior. Repeated inference can change the working set even when the artifact never changes.

Record memory at model load, each request boundary, the worst observed point, cancellation, cleanup, and recovery. If the process is restarted, end the sequence and begin a new one. Combining pre-restart and post-restart runs hides lifecycle behavior.

For multimodal features, keep camera or audio workload fixed. Otherwise changes in input tensor size can be mistaken for thermal effects.

Include failure and fallback

The sequence is not complete if failed runs are discarded. Record initialization errors, invalid output, backend fallback, cancellation failure, allocation failure, process death, and user-visible timeout.

Define whether a fallback is acceptable. A CPU fallback may preserve functionality, but it may no longer satisfy the intended interaction. The benchmark should report “completed through fallback” separately from “completed on the preferred path.”

Also test what happens after the device rests. Recovery time may matter for workflows that occur in bursts rather than continuously.

Test record schema

{
  "device": {},
  "os_build": "",
  "app_build": "",
  "artifact": {},
  "runtime": {},
  "backend_evidence": "",
  "workload_revision": "",
  "power_state": "",
  "run_index": 0,
  "elapsed": null,
  "memory": {},
  "thermal_context": {},
  "outcome": "",
  "failure_category": null
}

The schema intentionally avoids prescribing one temperature or timing API. Store what the platform exposes and describe how it was collected.

Comparison table

Cove’s current publication boundary

The Cove repository does not yet contain a complete real-device benchmark dataset with all required fields. Therefore Cove can publish this methodology now but cannot publish sustained-performance, temperature, or device-support conclusions as measured product facts.

Closed testing should produce the raw records. Once available, device pages can be generated from those records instead of being written from model size or chipset marketing.

Release checklist

• Define the real product session.
• Freeze workload, output, artifact, runtime, and build.
• Record power and ambient context.
• Confirm execution path evidence.
• Preserve every run in order.
• Record memory in the same timeline.
• Keep failures and fallbacks in the dataset.
• Test recovery after idle.
• Repeat on the minimum intended device tier.
• Publish collection method with any claim.
• Do not turn one device result into a universal compatibility statement.

Use the complete benchmark method for artifact and fixture design, RAM requirements for memory, and LiteRT-LM vs llama.cpp for runtime-path comparison.

Last reviewed: 2026-09-16.