Statistical organizations collect data via survey forms and other methods. The microdata are valuable for modeling and analysis. To produce a public-use file, the organizations mask the data in a manner that may prevent re-identification of data associated
Modeling and Quality of Masked Microdata
William E. Winkler, william.e.winkler@census.gov
U.S. Bureau of the Census, Room 3000-4, Washington, DC 20233-9100
other quantities that must be preserved for an analysis to be accurate.
Statistical organizations collect data via survey forms and In producing a public-use file Z, we assume that the other methods. The microdata are valuable for modeling and original, non-public data X are of high quality and then are analysis. To produce a public-use file, the organizations systematically distorted (i.e., masked) so that we cannot re-mask the data in a manner that may prevent re-identification identify information with individuals. By high quality, we Abstract
of data associated with individual entities. The public-use microdata may allow one or two sets of analyses that approximately reproduce analyses that could be performed on the original microdata. This paper describes a general method of creating models of data that is related to methods of creating appropriate aggregates of data that are needed for sufficient statistics in general classes of models (Moore and Lee 1998, DuMouchel et al. 2000, Owen 2003). If the aggregates can be approximately reproduced, then the masked microdata may allow one or more analyses that correspond to analyses on the original, non-public microdata. It will typically not yield data suitable for general analyses.
Keywords: Data Mining; Likelihood, Loglinear,
Multivariate
1. Introduction
Statistical agencies collect data via surveys and other sources that are used in producing aggregate statistics that are used in publications. Increasingly, there is demand for public-use data files that can be used for a variety of analyses that are not covered by the published aggregates. There are two issues. The first is that the public-use data should be suitable for several analyses that approximately reproduce analyses that are performed on the original, non-public data. The strengths and the limitations of the public-use data should be described in detail. The second is that the privacy of data associated with individual entities should be maintained. In this paper, we will be primarily concerned about the former issue but will indicate situations where re-identification of information with individuals may be straightforward.
There are two ways in which an analysis of microdata might be described. In the first, more difficult, method, we might do detailed modeling of the distributional characteristics of the data that are used for evaluation and prediction. In the second method, which might be considered as a special case of the first method, we might specify a number of aggregates that are needed for an analysis. For instance, we might specify the particular moments of continuous variables that are needed for analysis. In a regression, we might specify that the first two moments need to be preserved with very high accuracy. In loglinear modeling we might specify the marginal totals or mean that the list of entities is complete, unduplicated, and representative of the desired population and that values associated with the individual fields correspond to some underlying reality. For instance, the income of an individual in a record might correspond accurately to the actual income of the individual. The marital status might accurately correspond to actual marital status. The date-of-birth would be accurate and could be used for computing age at different dates. In this possibly idealized situation the original data X could be used for several analytic purposes.
Our key concept of quality is that the data are “fit for use”
in the sense of reproducing one or more analyses. The “fit for use” and other definitions of quality such as the Eurostat set (Haworth et al. 2001) are too general and vague to be applied in a straightforward manner. We will be primarily concerned with accuracy and comparability for which we can specify certain aggregates such as the first two moments that are needed in a regression analysis. As an example, we might like the first two moments that are computed on the masked data Z to approximately agree with the first two moments on the original data X. The approximate agreement should be sufficiently close that regression coefficients almost agree and that variance of the regression coefficients either agrees or increases slightly according to which masking procedure has been used.
The goal of the paper is to supply a few ideas for systematic methods that, given a specific analytic need, can be applied to determine the accuracy of an analysis on masked data Z. This can be somewhat straightforward when we show how closely a set of aggregates computed on Z is to the corresponding set of aggregates computed on X. If we
want to show that a regression can be done on an entire set Z and on several subdomains of Z, then the aggregates place substantial restrictions of the specific records in Z. If the set of restraints is sufficiently large in the sense of exactly preserving a large number of aggregates from the original data X, then the masked data Z may need to agree with the original microdata X. If the restraints only need to be preserved within some small epsilon, then portions of some records in Z may necessarily be close to some records in X. As a specific instance, combinations of fields in some records in Z may have values in the tails of certain distributions causing them to be much more easily re-identified.
expressed are those of the authors and not necessarily those of the U.S. Census Bureau. Dat Abstract: Statistical agencies collect data from surveys ...
expressed are those of the author and not necessarily those of the U.S. Census Bureau. Summ_专业资料。Disclaimer: This report is released to inform ...
Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of ...
The views expressed are those of the authors and do not necessarily represent o?cial positions of the Federal Reserve Bank of St. Louis or the Federal...
The views expressed are those of the authors and not necessarily those of the U.S. Census Bureau. Goodness-of-Fit and Badness-of-Fit Diagnostic Tests...
The views expressed herein are those of the authors and not necessarily those of the National Bureau of Economic Research, th...更多>>We thank Bernd ...
The views expressed are those of the individual authors and do not necessarily reflect official positions of the Federal Reserve Bank of St. Louis, the...
The views and interpretations expressed in this paper are those of the author and do not necessarily represent the...更多>>and Jaime Saavedra at GRADE ...
The views expressed are those of the author and not necessarily those of UNESCO. The designations employed and the presentation of material throughout ...
The views expressed in this publication do not necessarily reflect those of the European Central Bank. All rights reserved by the authors. Typeset and ...
热门文档