Bỏ qua

Data De-identification

Warning

The current page still doesn't have a translation for this language.

But you can help translating it: Contributing.

De-identification is the umbrella term for reducing how directly data identifies a person while retaining only the utility a test actually needs. Masking, generalization, and pseudonymization are techniques used to reach that outcome; they are not interchangeable legal guarantees.

  • Masking removes visible characters, for example with Mask or MiddleMask.
  • Generalization reduces precision, for example by retaining only a postal-code prefix.
  • Pseudonymization replaces a value with a stable token. The current Hash converter is stable for the same value, field path, and <setup rngSeed>; a different field path receives a separate token space.
  • Anonymization describes the resulting dataset only when re-identification is no longer reasonably possible. A converter alone cannot establish that property.

Prefer synthetic generation when production-derived values are unnecessary. When a test must retain source shape or selected relationships, define the required utility first and expose no more information than that purpose needs.

A complete CSV example

The source deliberately contains one repeated customer ID. The descriptor proves four different transformations without using real personal data.

data/customers.ent.csv
1
2
3
4
customer_id|full_name|email|postal_code
C-100|Alice Doe|[email protected]|10115
C-100|Alice Doe|[email protected]|10115
C-200|Bob Roe|[email protected]|20095
de-identification.xml
1
2
3
4
5
6
7
8
9
<setup rngSeed="20260730" numProcess="1">
    <generate name="customers" count="3" source="data/customers.ent.csv"
              distribution="ordered" target="CSV">
        <key name="customer_id" script="root.customer_id" converter="Hash('sha256', 'hex')"/>
        <key name="full_name" script="root.full_name" converter="Mask"/>
        <key name="email" script="root.email" converter="MiddleMask(2, 4)"/>
        <key name="postal_code" script="root.postal_code" converter="CutLength(3)"/>
    </generate>
</setup>

The .ent.csv suffix declares an entity-shaped CSV source. target="CSV" creates a downloadable result artifact in the Platform; use the registered LogExporter only when inspecting bounded diagnostic output instead of publishing an artifact.

The result has these observable properties:

  • both occurrences of C-100 receive the same pseudonym within customer_id, while C-200 receives a different one;
  • names retain only their length;
  • email values retain two leading and four trailing characters;
  • postal codes retain only the first three characters.

rngSeed is the run-level key owner for seeded hashing. Rotating it changes the pseudonyms. The current implementation isolates hashes by statement path, so this example proves linkability only inside the customer_id field; it does not claim automatic joins between unrelated fields or models. An unseeded hash remains deterministic but loses keyed separation and is unsuitable as a protection claim for low-entropy values.

Choosing the outcome

Review the complete dataset and threat model, not just the selected converter. Direct identifiers, rare combinations, free text, timestamps, and external linkability can still identify a person after individual fields have been transformed. Preserve a relationship only when the test explicitly requires it, and rotate or separate token domains when it does not.

Hosted Platform workflow

The Platform Database View can create a baseline de-identification descriptor from database metadata. See De-identification workflow in the Platform; the resulting XML remains governed by the Core contract documented here.