The fastest way to turn an AI-assisted QA practice into a data-protection incident is pasting a production extract into a chat window. This workflow puts detection and masking in front of every prompt: Presidio finds the PII, a local model handles the sensitive step, and a per-tool decision tree says what may enter which assistant — DPDP Rules and GDPR compliant by construction.
Tier: Core · Workflow area: Test data & environments
This playbook builds the pipeline — discover, mask, verify, then prompt — plus the one-page decision tree that makes the policy usable by the whole team rather than known by one careful engineer.
presidio-analyzer, presidio-anonymizer) and its NLP model.Discover before you believe. Run Presidio's analyzer over a representative extract and measure what lives in it — schemas lie about where PII hides (free-text notes fields are the classic offender):
from presidio_analyzer import AnalyzerEngine
analyzer = AnalyzerEngine()
results = analyzer.analyze(text=row_text, language="en",
entities=["PERSON","EMAIL_ADDRESS","PHONE_NUMBER","CREDIT_CARD",
"IN_AADHAAR","IN_PAN","LOCATION","DATE_TIME","IP_ADDRESS"])
discovery report — orders_extract.csv (10k rows)
EMAIL_ADDRESS 10,012 · PERSON 11,394 · PHONE 9,876 · IN_PAN 214 (!)
IN_AADHAAR 3 (!!) · notes column: 1,102 PERSON hits nobody expected
PAN and Aadhaar in an orders table is a finding to report upstream, not just to mask.
Extend recognizers for your domain identifiers. Out-of-the-box entities miss customer IDs, order references, internal employee codes — anything that re-identifies when joined. Add pattern recognizers (PatternRecognizer with regex + context words) for each, and re-run discovery until the report matches your data-owner's expectations.
Mask with consistency, not just redaction. Test data must stay useful: referential joins and realistic shapes survive when masking is deterministic per value:
from presidio_anonymizer import AnonymizerEngine
# strategy per entity:
# EMAIL -> hash-keyed fake: u_a41f2@masked.example
# PERSON -> Faker name seeded by hash(original) (same person -> same alias)
# PHONE/PAN/AADHAAR -> synthetic valid-format from reserved ranges
# free text -> replace spans, keep surrounding prose
Same input always maps to the same alias, so anita@… appearing in users, orders, and a log line becomes the same masked identity everywhere — joins, dedupe tests, and log-correlation workflows keep working.
Verify the mask with a second, independent pass. Re-run the analyzer over the masked output; the pass condition is numeric:
verification — orders_masked.csv: 0 findings above 0.4 confidence
canary check: 25/25 planted identifiers caught and masked in pipeline test
The canary set — identifiers you deliberately plant in a pipeline test — is your ongoing regression suite for the masker itself. A masking pipeline without its own tests is a policy written in hope.
Route the residual risk through a local model. Some steps need semantic judgment on unmasked data — "which of these free-text notes mention health conditions?" — precisely the data that must not leave. Run those steps on Ollama:
ollama run llama3.1:8b "Classify each note: contains-sensitive-category or clean.
Notes follow..." < notes_batch.txt
Local models are good enough for classification and pre-screening; the powerful external model gets the masked output afterward. This split — local for sensitive, external for masked — is the workflow's load-bearing wall.
Publish the decision tree and make it the default path. One page, three questions, on the wall:
Q1 Is the artifact fully synthetic (factory-generated)? -> any approved tool
Q2 Has it passed the masking pipeline with 0 findings? -> approved external
tools (list: ...), with client tools first on client work
Q3 Neither? -> local Ollama only,
or don't use AI for this step. NO EXCEPTIONS FOR DEADLINES.
Wire convenience toward compliance: a make mask FILE=... target, masked-extract storage the team pulls from by default, and the pipeline running in CI on every new extract. The policy people follow is the one that is easier than the violation.
Masking is for names, numbers, and identifiers — not for guilt. If discovery finds regulated identifiers (Aadhaar, PAN, card numbers) somewhere they should never be stored, masking the extract does not fix the system that leaked them into an orders table. File that as a finding to the data owner; the pipeline just handed you the evidence.
notes, emails in metadata_json, and an Aadhaar number in description. Always run discovery on actual data.This playbook is the restricted-environment playbook — but services-company delivery adds one layer: the client's rules override yours. Confirm which masking standard the client requires (some mandate their own tooling or on-premises processing), run the pipeline inside their environment where required, and put the decision tree into the engagement's runbook with the client's approved-tool list substituted. Disclose the pipeline in the engagement's data-processing documentation — a working masker with canary tests is an asset in the client's audit, and it is much better introduced before their DPO asks.
Privacy-safe AI-assisted QA is an architecture, not a resolution. Detection you can test, masking that preserves utility, a local path for what must not leave, and a three-question tree anyone can follow — build it once, and "can I paste this?" stops being a judgment call forever.