advanced27 min studyLesson 967 in Engineering

On-device data and privacy boundaries

Map raw inputs, derived features, retained state and telemetry across trust boundaries rather than treating local inference as an automatic privacy guarantee.

On this page

What you’ll understand

  • Distinguish local processing from retention, disclosure and access control.
  • Calculate storage and telemetry volume under explicit retention policies.
  • Explain why sampling, embeddings and smaller logs do not by themselves establish anonymity or differential privacy.

Keeping inference on a device can remove a particular raw-data transfer to a server. It does not decide what the application stores, logs, backs up, or exposes through its outputs. A useful privacy analysis follows data through the entire system, including derived features, crash reports, identifiers, state and model updates. State the adversary and access assumptions before claiming that a local pipeline protects a user’s information.

Draw the data flow and every copy

List the raw sensor input, temporary buffers, extracted features, recurrent state, predictions, diagnostic logs, and network messages. For each item, record who can access it, how long it persists, and what deletes it. Data held only in application RAM has a different exposure from a persistent file, but either can matter under a compromised device or diagnostic tooling. A diagram that stops at the inference call misses copies created by preprocessing, profiling, crash collection, and downstream application code.

An operating system can also transfer data outside the inference process. Android’s Auto Backup documentation is a concrete example: inclusion and exclusion rules depend on platform and target version, and cloud backup and device-to-device transfer have distinct controls. For apps targeting API 31 and above, one manifest flag does not uniformly disable every transfer on every manufacturer’s device. Verify the deployed backup rules and restore behavior rather than assuming that private application storage means data never leaves the device.

Vraw=fscbT,Vfeatures=rdbfTfV_{\mathrm{raw}}=f_scbT,\qquad V_{\mathrm{features}}=rdb_fT_f
Here raw sampling rate f_s is samples/s/channel, c is channel count, b is bytes/sample, and retention T is seconds. Feature records arrive at r records/s, contain d elements of b_f bytes, and are retained for T_f seconds. Both volumes use bytes and exclude metadata, compression, replication and storage overhead.↔ Scroll wide equations horizontally.

Derived representations can still carry sensitive information

A feature vector can preserve identity, activity, location, or reconstructable input details even when it is shorter than the raw signal. Original representation-inversion experiments demonstrate that some deep features retain substantial image information under their tested settings. That finding does not say every embedding is invertible, but it defeats an automatic inference from transformed to anonymous. Evaluate the representation, available auxiliary information, attacker access, and the specific information the product intends to protect.

Reducing telemetry sampling lowers expected record count, not necessarily identification risk for the retained records. Persistent identifiers, precise timestamps and rare events can make a few records highly informative. Ordinary random subsampling by itself is not a differential-privacy guarantee; any such claim needs the full mechanism, adjacency relation and privacy accounting. Separate operational need from convenient debugging: collecting less sensitive aggregate information with short retention can reduce exposure, but its adequacy still depends on the stated threat model and actual implementation.

E[K]=mρ,P(K≥1)=1−(1−ρ)m\mathbb E[K]=m\rho,\qquad \mathbb P(K\ge1)=1-(1-\rho)^m
Assume m independent telemetry-selection decisions with common retention probability rho in [0,1]. K counts selected records, so its expectation has record units. These are binomial collection statistics, not a privacy budget, identification probability, leakage bound, or claim about correlated real logging.↔ Scroll wide equations horizontally.

Validate the shipped data policy rather than the architecture label

Inspect storage, network traces, backup and restore paths, and crash artifacts under normal and failure conditions. Define access control, key handling where encryption is used, retention and deletion, and support procedures for diagnostics. Encryption does not remove authorized endpoint access or prevent the application from intentionally logging plaintext. Include model-state resets and user/session boundaries in the tests, because locally retained history can cross those boundaries even when the device never contacts a model server.

The probability lab illustrates telemetry selection only. Reset, set p to 0.1 and n to 10, predict expected selected count one and probability of no selected record approximately 0.3487, then Step. Compare a fresh Reset with both p at 0.1 and n at 30 before Step: mean three and no-selection probability approximately 0.0424. These exact independent-binomial masses contain no personal information, adversary, backup system, or privacy analysis.

A small feature stream can create a larger retained dataset

A mono application processes 16 kHz audio with two-byte samples and keeps only the most recent 60 seconds of raw audio. It also logs 20 float32 feature values ten times per second for 24 hours. Ignore metadata, compression and filesystem overhead.

  1. Raw retained volume is 16,000 × 1 × 2 × 60 = 1,920,000 bytes, or 1.92 decimal MB. This is a 60-second retention policy.

  2. Feature logging rate is 20 × 4 × 10 = 800 bytes/s. Over 86,400 seconds it retains 69,120,000 bytes, or 69.12 MB.

  3. The retained feature dataset is 69.12/1.92 = 36 times larger under these different retention periods. Neither this size ratio nor replacing raw audio with features establishes anonymity; inspect content, identifiers, access and retention purpose.

The result

Retention policy and data semantics matter independently of where inference executes or how compact one feature record looks.

Explore the model

Interactive lab

Interactive lab · probability

Telemetry sampling counts without a privacy guarantee

Reset, set p = 0.1 and n = 10, predict mean selected records 1 and P(K=0) ≈ 0.3487, then Step. Compare a fresh Reset with both p = 0.1 and n = 30 before Step: mean 3 and P(K=0) ≈ 0.0424. This exact independent-selection model has no attacker, identifying content, or differential-privacy mechanism.

Telemetry sampling counts without a privacy guaranteeThese are exact binomial probabilities for independent trials with one fixed success probability. Playback tours the distribution; it does not resample outcomes. Current readings: Mean · np: 5; Variance · np(1−p): 2.5; Total probability: 1. All parameters are adjustable with the labeled controls below.03581000.070.150.220.3Successes kP(X = k)k = 0 · P = 9.77e-4
Exact probabilityHighlighted outcomeMean np
Mean · np
5
Variance · np(1−p)
2.5
Total probability
1
t = 0 s

These are exact binomial probabilities for independent trials with one fixed success probability. Playback tours the distribution; it does not resample outcomes. Parameter changes pause playback and recompute the model at the current elapsed time. Reset restores both parameters and returns time to zero. Playback starts paused. Adjust either parameter, then choose Play or Step.

Look forCollecting fewer records and proving a privacy property require different evidence.

Check your understanding

An application performs inference locally and uploads only embeddings. Which privacy conclusion follows from that architecture alone?

Try it yourself

A mono device retains 30 seconds of 8 kHz two-byte audio and one hour of feature records arriving at 100 bytes/s. Compute the two volumes and explain whether the smaller one is automatically less identifying.

Take a moment to make a prediction or work it out on paper. Open a hint when you need one.

Give me a hint

Use separate retention durations and decimal MB. Record volume is not a measure of what an attacker can infer.

Compare with the solution

Raw retention is 8,000 × 2 × 30 = 480,000 bytes, or 0.48 MB. Feature retention is 100 × 3,600 = 360,000 bytes, or 0.36 MB. The smaller feature dataset is not automatically less identifying: it might preserve stable identity features or precise event times, while access and auxiliary information also affect exposure. Evaluate those facts alongside a tested retention and transfer policy.

Take it with you

  • Local inference changes particular data flows rather than guaranteeing privacy.
  • Derived features and operational artifacts can retain sensitive information.
  • Test actual retention, telemetry, backups, access and session boundaries.

Read further

Original sources and deeper explanations to follow at your own pace.

Optional progress, saved in this browser when available. Revisit any lesson at any time.

Lesson not marked complete.