When real examples are scarce — a new product, a rare intent, a domain where logging is restricted — LLMs can manufacture training pairs and golden Q&A sets from your corpus. Ungated, this produces the classic failure: thousands of fluent, samey, subtly wrong examples that teach the model your generator's quirks.
Tier: Advanced · Workflow area: Data for fine-tuning & evaluation
Gated properly — LLM-as-judge filtering, dedup, grounding checks, and distribution management — synthetic data is a legitimate multiplier, and engineers who automate it well are leveraged (LinkedIn's fastest-growing-roles list has "data annotator" near the top precisely because this work is being industrialized).
Don't synthesize what you can sample: real data beats synthetic at equal count, and synthetic eval data must never be the only judge of a model trained on synthetic training data from the same generator.
Ground generation in real corpus chunks. Free-floating generation invents facts; grounded generation reads a real document chunk and produces Q&A or instruction pairs answerable from that chunk. Iterate over the corpus so coverage follows the corpus, not the model's imagination:
import anthropic
from pydantic import BaseModel
class GeneratedPair(BaseModel):
question: str
answer: str
source_span: str # the sentence(s) the answer relies on
client = anthropic.Anthropic()
for chunk in corpus_chunks: # loop over the API, checkpointed
response = client.messages.parse(
model="claude-opus-4-8",
max_tokens=2048,
thinking={"type": "adaptive"},
messages=[{"role": "user", "content":
"From the passage below, write one question a real user would ask "
"and its answer. The answer must be fully supported by the passage; "
"quote the supporting sentence(s) in source_span. Vary question style: "
"factual, how-to, comparison, edge-case.\n\n"
f"PASSAGE:\n{chunk.text}"}],
output_format=GeneratedPair,
)
save_candidate(chunk.id, response.parsed_output)
Expected output: one grounded pair per chunk with an auditable source_span. Prompt for style variety explicitly — generators left alone converge on one question shape.
Gate 1 — groundedness check. A separate judge call (different prompt, ideally different model config than the generator) verifies each answer against its source chunk: supported / partially / unsupported, with a confidence score. Unsupported pairs are dropped; partials go to a human-review queue. Never let the generator grade its own homework in the same call.
Gate 2 — dedup and diversity. Embed all candidate questions; drop near-dups above ~0.92 similarity, keeping the higher-judged member. Then look at the survivors' spread: cluster the embeddings and count per cluster — fifty clusters of two beats two clusters of fifty. If clusters are lopsided, generate more from the under-covered corpus regions, not more overall.
Gate 3 — distribution checks against reality. Compare synthetic to real data on measurable axes: length distributions, question-type mix, vocabulary richness, label balance. Synthetic text runs longer, politer, and more uniform than real user text; if the histograms diverge hard, adjust generation prompts (add real examples as style anchors) rather than shipping a dataset that smells like a model.
Human-audit a stratified sample. 100 items across clusters and judge-score bands, scored by a person against the same rubric. This calibrates the judge (agreement rate) and catches systematic weirdness no automated gate sees — questions no human would ask, answers that are technically supported but useless.
Label provenance and split by origin. Every record carries origin: synthetic, generator model, prompt version, and source chunk ID. Mixing synthetic into training sets is fine (start with minority shares — synthetic augmenting real, not replacing it); synthetic leaking unlabeled into a golden eval set silently corrupts every number downstream. RAGAS's testset generator plays the same role for RAG evaluation — same gates apply to its output.
Close the loop with outcome metrics. The final gate is empirical: did the augmented training set improve the downstream eval (on real held-out data)? Did the synthetic golden set's rankings agree with a small human-judged ranking? Track this per generation batch, and prune batches that don't pull their weight.
A model fine-tuned on synthetic data must never be evaluated primarily on synthetic data from the same generator — the shared quirks cancel out and every metric flatters. Keep a real-data eval slice, however small, as the final word.
unsupported pair in the shipped set.Synthetic data generated from client documents is a derivative of those documents — treat it under the same classification as the source corpus, not as "fake data we made up". Confirm the client approves the generator provider seeing corpus chunks, run generation inside their environment, and store candidates, judge scores, and the final sets in their storage. Flag the IP question explicitly: some client agreements restrict creating training assets from their content, and that's a legal conversation, not an engineering one. If external APIs are off the table, distilabel plus a self-hosted model covers generation and judging with the same gate structure. Deliver the provenance-labeled dataset, the gate reports, and the human-audit results as one package — the client's ML team should be able to defend every synthetic example's origin.