Privacy and membership inference

Membership inference asks whether a candidate record or person was part of a training dataset. Reconstruction attempts to recover data, while attribute inference estimates a hidden characteristic. These tasks can overlap in consequences but have different success criteria. Specify whether the observer sees labels, confidence scores, loss values, or internal parameters, and whether they possess the candidate record or auxiliary data. Interface restrictions change the assessment, not the underlying definition.

Original membership-inference research demonstrates privacy risks in studied models, and later work examines connections with overfitting. Better generalization can reduce some signals but does not constitute a universal privacy guarantee. Conversely, a weak attack result does not establish safety against stronger observers. Compare to a baseline that already knows the candidate population and available auxiliary facts, so the assessment measures information attributable to model access.

A balanced test set is convenient for comparing methods, but real membership prevalence may be very low. A small false-positive rate applied to many nonmembers can produce more false accusations than correct identifications. Report true-positive rate at relevant false-positive levels, uncertainty, and positive predictive value under explicit prevalence assumptions. Do not interpret a detector score as an individual's membership probability unless calibration and population assumptions support that use.

Members and nonmembers should come from comparable populations and collection periods. Otherwise an observer may distinguish time, geography, or preprocessing rather than training participation. Separate validation used to choose an attack threshold from final assessment data. Stratify where justified, because vulnerable subgroups can be hidden by averages, and use synthetic or appropriately authorized records so the privacy assessment itself does not create an unnecessary disclosure.

Limit data collection and retention, protect training records and logs, enforce access policy, and expose only output detail needed for the product. Rate controls can constrain repeated querying but do not by themselves provide mathematical privacy. Differential privacy can offer a formal guarantee under a stated neighboring-dataset definition and correct accounting; it addresses a different evidentiary question from whether one empirical observer happened to succeed.

Maintain privacy assessments through fine-tuning, distillation, adapter releases, and changes in output detail. Keep results tied to the exact model and access regime. A membership test should not be used as definitive evidence that a specific person's record was included: false positives and distribution mismatch matter. When reporting a potential leak, preserve minimal authorized evidence and involve the people responsible for data handling and incident response.