Realistic Fixtures, Factories, and Seed Data From Your Actual Schema

Tier: Foundation · Workflow area: Testing & Quality

AI is excellent at generating data that honors your real schema constraints and deliberately covers the edge distributions, provided you feed it the schema and forbid it from inventing one.

When to use

• •
  Standing up factories (factory_boy, Fishery, FactoryBot) for a new test suite or a module that never had them.
• •
  Seeding a demo or staging environment with plausible, non-production data.
• •
  You keep writing User("a", "b@c.com") inline in tests and the duplication is rotting.
• •
  Load-test or migration-rehearsal datasets that must respect FK graphs and check constraints.

Skip when a handful of hand-written literals is genuinely clearer — a fixture layer for two tests is ceremony.

Prerequisites

• •
  The authoritative schema in machine-readable form: SQL DDL, ORM models, Pydantic/Zod schemas, or proto files.
• •
  A factory/faker library for your stack (factory_boy + Faker, Fishery + faker-js) — all free.
• •
  Known domain constraints that are NOT in the schema (business rules like "premium accounts always have a billing address") written down, because the AI cannot infer them.

The workflow

1. 1.

   Feed the real schema, forbid invention. Paste the DDL or ORM models:

   Copy

   Here are our SQLAlchemy models for User, Subscription, Invoice (below).
   Generate factory_boy factories. Rules:
   - Respect every constraint you can see: nullability, unique email, CHECK
     (amount_minor >= 0), FK from Invoice.subscription_id.
   - Do not add fields that are not in the models. Do not guess enum values —
     use exactly the ones defined.
   - Timestamps: timezone-aware UTC, spread across the last 400 days, never
     all-midnight.
   - Names/emails via Faker with locale mix en_IN + en_US; include names with
     apostrophes, hyphens, and non-ASCII.
   

2. 2.

   Add the business rules the schema cannot express. Follow up: Additional invariants: a Subscription in state 'past_due' must have >=1 unpaid Invoice; trial users have no Invoice rows. Encode these as factory traits/params, not comments. This is the step that separates usable factories from pretty noise.

3. 3.

   Request an edge-case catalog, then bake it in. List 15 edge cases production data would contain for these models (boundary amounts, leap-day birthdays, max-length strings, unicode, concurrent-period subscriptions). Emit a pytest fixture edge_users that returns all of them. Review the list — you will usually add two domain-specific ones it missed and cut two irrelevant ones.

4. 4.

   Verify against the real database. Insert a generated batch into a scratch DB and let the constraints be the judge: INSERT failures mean the factories drifted from the schema. For JSON-schema'd payloads, validate every generated object with the actual validator (Pydantic/Zod/jsonschema), not by eye.

5. 5.

   Wire distribution realism for seed/load data. For volume seeding, ask for a generator script with explicit distributions ("80% of users have 1–3 invoices, 1% have 200+; amounts log-normal around ₹1,200") so aggregate queries and pagination behave like production.

Tool variants

• Copilot / Cursor in-repo: best default — the models are already in context; generate factories file-by-file.
• Chat assistant: fine for the edge-case catalog and distribution design; paste schemas in.
• Claude Code / agent mode: can run the insert-verify loop itself against a local Postgres and fix constraint violations until the batch inserts clean.

Verification

• •
  A generated batch of 1,000 rows inserts into a scratch DB with zero constraint violations.
• •
  Uniqueness under volume: no duplicate-email collisions at your seed size (Faker sequences, not random choice).
• •
  The edge fixture actually breaks something when run against known-weak code paths — that is its job.
• •
  Regenerated data is deterministic when seeded (Faker.seed(42)), so CI failures reproduce.

Pitfalls

• Letting the AI hallucinate the schema. If you describe tables in prose instead of pasting DDL, you get plausible fields that do not exist. Always paste the source of truth.
• Uniformly random data. Real data is skewed; uniform data hides pagination, index, and aggregation bugs. Specify distributions.
• Fixture drift. Schema migrations silently orphan factories. Add a CI test that builds every factory and round-trips it through the DB — it fails the day someone adds a NOT NULL column.
• "Realistic" sliding into "real." The moment someone pastes production rows in as examples, you have a data-protection incident, not a fixture.

Restricted-environment variant

Schemas are usually shareable when table/column names are not client-identifying — but check the NDA; some client schemas are themselves confidential. If so: rename tables/columns to neutral equivalents before prompting, keep a local mapping, and rename back in the generated code. Under DPDP Act 2023, personal data in logs or dumps must never be the seed source — synthetic-only is not just hygiene, it is compliance. Local models via Ollama handle factory generation from pasted DDL well; this is one of the tasks where the frontier gap barely matters.

Success criteria

• •
  Factories exist for every core model, encode the extra-schema business rules, and are used by new tests instead of inline literals.
• •
  The edge-case fixture is in CI and has caught at least one real bug (apostrophes, tz, boundary amounts are the usual first blood).
• •
  Seed script produces a demo environment a PM can screenshot without anyone asking "why is every customer named test1?"
• •
  Zero production-derived values anywhere in the fixture layer.