Lab 08 · Privacy-Safe Test Data Factory

Nisha needs realistic order data for staging, and the shortcut everyone reaches for — "just anonymize a production dump" — is exactly how PII ends up in a prompt, a screenshot, and a vendor's training corpus. Under India's DPDP Act rules and GDPR alike, test data is personal data until proven otherwise, and "we removed the name column" has never once been proof. In this lab you run a proper pipeline: automated PII discovery with Presidio, an honest audit of what the scanner missed, masking, then a synthetic replacement generated to a validated schema — with the sensitive step kept on a local model. The extract below is fake but shaped like the real thing, planted identifiers and all.

Time, tier & cost

~90 minutes hands-on · Core tier (maps to module M5 and playbooks PB-24/PB-25). Cost: $0 (Presidio, Faker, Pydantic, Ollama are all free and local).

Setup

mkdir -p qa-lab-08 && cd qa-lab-08
python3 -m venv .venv && source .venv/bin/activate
pip install presidio-analyzer presidio-anonymizer faker pydantic
python -m spacy download en_core_web_lg
# optional local model for the sensitive step:
ollama pull llama3.1:8b

prod_extract.csv — the "production" slice. Treat it as radioactive from this line on:

order_id,customer_name,email,phone,ship_address,id_number,notes,amount_inr
9001,Rohit Sharma,rohit.sharma81@gmail.com,+91 98450 12345,"14 MG Road, Indiranagar, Bengaluru 560038",ABCPS1234F,Leave with security,2499.00
9002,Fatima Begum,fatima.b@outlook.com,9912345678,"Flat 3B, Sea Breeze Apts, Bandra West, Mumbai 400050",4321 8765 2109,Call before delivery,760.50
9003,Daniel D'Souza,dan.dsouza@yahoo.in,+91-80-2556-1234,"7 Rest House Rd, Bengaluru 560001",ABCPD9876K,Customer says neighbour Mrs. Iyer (98801 44556) can receive it,15200.00
9004,Priya Venkatesan,priya.v@company.co.in,08041234567,"221 Anna Salai, Chennai 600002",,Refund to card ending 4242 requested on call,439.99
9005,Aarav Mehta,aarav2019@gmail.com,+91 90000 11111,"5 Ridge Rd, Shimla 171001",,Minor account - father Rakesh Mehta rakesh.m@gmail.com approves,899.00
9006,GLOBAL TRADERS PVT LTD,accounts@globaltraders.in,+91 22 4001 2000,"Plot 12, MIDC, Andheri East, Mumbai 400093",27AAACG1234A1Z5,GSTIN on invoice pls,88000.00

schema.py — the Pydantic contract your synthetic data must satisfy:

from pydantic import BaseModel, EmailStr, Field, field_validator
from datetime import date

class Order(BaseModel):
    order_id: int = Field(ge=1)
    customer_name: str = Field(min_length=1, max_length=80)
    email: EmailStr
    phone: str = Field(pattern=r"^\+91[ -]?\d{5}[ -]?\d{5}$")
    ship_city: str
    ship_pin: str = Field(pattern=r"^[1-9]\d{5}$")
    order_date: date
    amount_inr: float = Field(gt=0, le=500000)

    @field_validator("order_date")
    @classmethod
    def not_future(cls, v):
        assert v <= date.today(), "order_date cannot be in the future"
        return v

Create pii-audit.md and data-policy.md empty.

Steps

1. 1Run automated discovery first — and time-box it. Write scan.py: load the CSV, run AnalyzerEngine().analyze(text=..., language="en") over every cell (or row-concatenated text), and print entity type, value, confidence per hit. Dump the results into pii-audit.md under "Scanner found." Presidio will reliably flag names, emails, phones, and locations. Ten minutes, no manual reading yet.
2. 2Now audit the extract by eye against a checklist. Read all six rows the way a DPO would, hunting categories, not strings: direct identifiers, contact data, government identifiers, financial traces, third-party persons, minors, and business-vs-personal data. Add a "Human found" section to pii-audit.md. The seeded extract contains at least five identifier classes the default scanner misses or mislabels — you are not done until your human list is longer than the scanner's.
3. 3Write custom recognizers for the Indian identifiers. Presidio ships no default PAN or Aadhaar-pattern recognizer — add them: a PatternRecognizer for PAN ([A-Z]{5}[0-9]{4}[A-Z]), one for Aadhaar-like 12-digit groups (\d{4}\s\d{4}\s\d{4}), and one for GSTIN (\d{2}[A-Z]{5}\d{4}[A-Z]\d[A-Z]\d). Rerun the scan and record what the upgraded scanner now catches that stock Presidio didn't. This before/after is the calibrated-trust datapoint of the lab.
4. 4Mask with the anonymizer — free text included. Use AnonymizerEngine to replace every detected entity (<NAME>, <PHONE>, <PAN>…) and write masked_extract.csv. Then manually verify the notes column row by row: the third-party neighbour and her phone, the card reference, and the father's email must all be gone. Free text is where masking pipelines die; the notes column is the seeded proof.
5. 5Decide the GSTIN question in writing. Row 9006 is a company: GSTIN and a business address are not personal data — but the extract pattern can't know that, and sole proprietors' GSTINs can identify a person. Write your ruling and rationale in pii-audit.md (defensible either way; undefended = ungraded).
6. 6Generate the synthetic replacement. Write factory.py with Faker (Faker("en_IN")) generating 50 orders validated through Order — every record passes Pydantic or the run fails. Force edge cases into the batch deliberately: names with apostrophes and Unicode (D'Souza, Ñina, a Devanagari name), a leap-day order_date (2024-02-29), boundary pins (110001, 999999), amount at exactly 500000, and an 80-char name. Emit synthetic_orders.csv plus a one-line proof: 50/50 records valid.
7. 7Use the local model for the sensitive-shaped step. With Ollama, prompt llama3.1:8b to generate 10 realistic free-text delivery notes in the style of the masked ones (notes are the field whose realism you can't get from Faker). Rule being drilled: raw-shaped production text goes only to local models; the cloud model may see schemas, masked samples, and code — never the unmasked extract. Log which tool saw which artifact in pii-audit.md.
8. 8Write the one-page policy.data-policy.md, four sections, one page: (1) classification — what counts as PII here, including the free-text and third-party cases you just hit; (2) the decision tree — raw extract → local-only; masked extract → approved cloud tools; synthetic → any tool; (3) the pipeline — scan with custom recognizers → mask → human free-text review → synthetic replacement, with the audit file as evidence; (4) DPDP/GDPR one-liners — purpose limitation applies to test data, and "we masked the name column" is not compliance.

Verify

• •
  pii-audit.md has three layers: stock-scanner findings, human findings, upgraded-scanner findings — with the deltas explicit.
• •
  masked_extract.csv contains zero surviving identifiers, including all free-text plants.
• •
  Custom recognizers for PAN, Aadhaar-pattern, GSTIN exist and demonstrably fire.
• •
  synthetic_orders.csv: 50/50 Pydantic-valid, edge cases present and listed.
• •
  Tool-exposure log shows the unmasked extract touched only local tools.
• •
  data-policy.md fits one page and contains the decision tree.

Self-grading rubric (pass = 7/10):