On-device data and privacy boundaries
Map raw inputs, derived features, retained state and telemetry across trust boundaries rather than treating local inference as an automatic privacy guarantee.
On this page
The question for today
What you’ll understand
- Distinguish local processing from retention, disclosure and access control.
- Calculate storage and telemetry volume under explicit retention policies.
- Explain why sampling, embeddings and smaller logs do not by themselves establish anonymity or differential privacy.
Keeping inference on a device can remove a particular raw-data transfer to a server. It does not decide what the application stores, logs, backs up, or exposes through its outputs. A useful privacy analysis follows data through the entire system, including derived features, crash reports, identifiers, state and model updates. State the adversary and access assumptions before claiming that a local pipeline protects a user’s information.
Draw the data flow and every copy
List the raw sensor input, temporary buffers, extracted features, recurrent state, predictions, diagnostic logs, and network messages. For each item, record who can access it, how long it persists, and what deletes it. Data held only in application RAM has a different exposure from a persistent file, but either can matter under a compromised device or diagnostic tooling. A diagram that stops at the inference call misses copies created by preprocessing, profiling, crash collection, and downstream application code.
An operating system can also transfer data outside the inference process. Android’s Auto Backup documentation is a concrete example: inclusion and exclusion rules depend on platform and target version, and cloud backup and device-to-device transfer have distinct controls. For apps targeting API 31 and above, one manifest flag does not uniformly disable every transfer on every manufacturer’s device. Verify the deployed backup rules and restore behavior rather than assuming that private application storage means data never leaves the device.
Derived representations can still carry sensitive information
A feature vector can preserve identity, activity, location, or reconstructable input details even when it is shorter than the raw signal. Original representation-inversion experiments demonstrate that some deep features retain substantial image information under their tested settings. That finding does not say every embedding is invertible, but it defeats an automatic inference from transformed to anonymous. Evaluate the representation, available auxiliary information, attacker access, and the specific information the product intends to protect.
Reducing telemetry sampling lowers expected record count, not necessarily identification risk for the retained records. Persistent identifiers, precise timestamps and rare events can make a few records highly informative. Ordinary random subsampling by itself is not a differential-privacy guarantee; any such claim needs the full mechanism, adjacency relation and privacy accounting. Separate operational need from convenient debugging: collecting less sensitive aggregate information with short retention can reduce exposure, but its adequacy still depends on the stated threat model and actual implementation.
Validate the shipped data policy rather than the architecture label
Inspect storage, network traces, backup and restore paths, and crash artifacts under normal and failure conditions. Define access control, key handling where encryption is used, retention and deletion, and support procedures for diagnostics. Encryption does not remove authorized endpoint access or prevent the application from intentionally logging plaintext. Include model-state resets and user/session boundaries in the tests, because locally retained history can cross those boundaries even when the device never contacts a model server.
The probability lab illustrates telemetry selection only. Reset, set p to 0.1 and n to 10, predict expected selected count one and probability of no selected record approximately 0.3487, then Step. Compare a fresh Reset with both p at 0.1 and n at 30 before Step: mean three and no-selection probability approximately 0.0424. These exact independent-binomial masses contain no personal information, adversary, backup system, or privacy analysis.
Work it through
A small feature stream can create a larger retained dataset
A mono application processes 16 kHz audio with two-byte samples and keeps only the most recent 60 seconds of raw audio. It also logs 20 float32 feature values ten times per second for 24 hours. Ignore metadata, compression and filesystem overhead.
Raw retained volume is 16,000 × 1 × 2 × 60 = 1,920,000 bytes, or 1.92 decimal MB. This is a 60-second retention policy.
Feature logging rate is 20 × 4 × 10 = 800 bytes/s. Over 86,400 seconds it retains 69,120,000 bytes, or 69.12 MB.
The retained feature dataset is 69.12/1.92 = 36 times larger under these different retention periods. Neither this size ratio nor replacing raw audio with features establishes anonymity; inspect content, identifiers, access and retention purpose.
Retention policy and data semantics matter independently of where inference executes or how compact one feature record looks.
The living notebook
Explore the model
Interactive lab · probability
Telemetry sampling counts without a privacy guarantee
Reset, set p = 0.1 and n = 10, predict mean selected records 1 and P(K=0) ≈ 0.3487, then Step. Compare a fresh Reset with both p = 0.1 and n = 30 before Step: mean 3 and P(K=0) ≈ 0.0424. This exact independent-selection model has no attacker, identifying content, or differential-privacy mechanism.
- Mean · np
- 5
- Variance · np(1−p)
- 2.5
- Total probability
- 1
These are exact binomial probabilities for independent trials with one fixed success probability. Playback tours the distribution; it does not resample outcomes. Parameter changes pause playback and recompute the model at the current elapsed time. Reset restores both parameters and returns time to zero. Playback starts paused. Adjust either parameter, then choose Play or Step.
Look forCollecting fewer records and proving a privacy property require different evidence.
Pause & reason
Check your understanding
Your turn
Try it yourself
A mono device retains 30 seconds of 8 kHz two-byte audio and one hour of feature records arriving at 100 bytes/s. Compute the two volumes and explain whether the smaller one is automatically less identifying.
Take a moment to make a prediction or work it out on paper. Open a hint when you need one.
Give me a hint
Use separate retention durations and decimal MB. Record volume is not a measure of what an attacker can infer.
Compare with the solution
Raw retention is 8,000 × 2 × 30 = 480,000 bytes, or 0.48 MB. Feature retention is 100 × 3,600 = 360,000 bytes, or 0.36 MB. The smaller feature dataset is not automatically less identifying: it might preserve stable identity features or precise event times, while access and auxiliary information also affect exposure. Evaluate those facts alongside a tested retention and transfer policy.
Keep the idea
Take it with you
- Local inference changes particular data flows rather than guaranteeing privacy.
- Derived features and operational artifacts can retain sensitive information.
- Test actual retention, telemetry, backups, access and session boundaries.
Read further
Original sources and deeper explanations to follow at your own pace.
Optional progress, saved in this browser when available. Revisit any lesson at any time.
Lesson not marked complete.