Release Notes and Pre-Deploy Risk Briefs from Change Data

Tier: Core · Workflow area: CI/CD & release engineering

The notes are low-risk drafting; the risk brief is a genuine stability control, because AI-accelerated delivery ships more change per deploy and the risk review is where that speed gets converted back into safety.

When to use

• •
  Cutting a release that bundles more than a handful of PRs.
• •
  Deploying to an environment with a change-review or CAB step that needs a written summary.
• •
  Post-incident periods where every deploy gets extra scrutiny and you need the scrutiny to be fast.

Skip the risk brief for single-line, feature-flagged, instantly-revertible changes — process theater helps nobody.

Prerequisites

• •
  Clean merge history between two tags (git log v1.41..v1.42 --merges or the GitHub compare API).
• •
  Access to recent incident titles/dates (even a text export works).
• •
  Any capable AI assistant or an LLM API for the scripted version.

The workflow

1. 1.

   Extract the raw change set.

   Copy

   gh pr list --search "merged:>=2026-07-20 base:main" \
     --json number,title,labels,files,author > changes.json
   git diff v1.41..v1.42 --stat > diffstat.txt
   

2. 2.

   Generate release notes with an audience split.

   Copy

   From the attached merged-PR list, write release notes in three sections:
   - For customers: user-visible changes only, plain language, no internal names.
   - For engineers: all changes grouped by service, with PR links.
   - Dependency/infra changes: version bumps, schema migrations, config changes.
   Use only the attached data. If a PR title is too vague to classify, list it
   under "Needs description" instead of guessing what it does.
   

3. 3.

   Generate the risk brief from the same data plus operational context.

   Copy

   Now produce a pre-deploy risk brief for this change set. Context attached:
   diffstat, dependency diff, and the last 90 days of incident titles.
   
   Output:
   1. Riskiest changes (max 5), each with: why, blast radius (single service /
      shared infra / data), and the specific rollback for that change.
   2. Migration/irreversibility flags: schema changes, data backfills,
      one-way config flips.
   3. Any change touching a service named in a recent incident.
   4. Overall risk: LOW / MEDIUM / HIGH with one-sentence justification and
      an explicit confidence level. State what context you did NOT have.
   

4. 4.

   Verify the brief against reality. For each "riskiest change": open the actual diff and confirm the characterization. Check the migration list against the real migrations directory — this is a claim audit, and missed migrations are the classic gap.

5. 5.

   Human sets the final risk rating. The model's LOW/MEDIUM/HIGH is an input. The deploy decision, timing (not at 5 pm Friday), and rollback-readiness call belong to the engineer whose name is on the deploy.

Tool variants

• Scripted LLM API call in the release pipeline: generates both documents automatically on tag creation, posts to the release page and the deploy channel — the sustainable form.
• Claude Code / agentic CLI at release time: can run the gh/git extraction itself and iterate; good while you are still tuning the prompts.
• Change-intelligence features in incident platforms (incident.io, Datadog change correlation): use them as an additional context source for step 3, not a replacement for the brief.

Verification

• Validate: every PR in the notes exists; every "Needs description" item chased down before publishing.
• Plan: risk brief cross-checked against the actual diff for the top-5 changes; migrations list matches the migrations directory exactly.
• Policy: customer-facing notes contain no internal service names, ticket IDs, or security-sensitive details (grep for your internal naming pattern).
• Cost/stability: HIGH-risk deploys get a stated mitigation (canary, off-peak window, feature flag) before proceeding — the brief must change behavior, not just exist.

Pitfalls

• Vague PR titles in, fiction out. "Fix stuff" becomes an invented description. The "Needs description" escape hatch in the prompt exists to stop this — enforce it.
• Risk ratings without context. If you skip attaching incident history, the model rates on diff size alone, which is a weak proxy. Garbage context in, confident garbage out.
• Notes leaking internals. Models default to including everything; the customer section needs the explicit exclusion rule and a human skim.
• Rubber-stamping. If every brief comes back LOW and gets glanced at, the control has decayed. Periodically seed a known-risky change set and check the brief catches it.

Restricted-environment variant

PR titles, service names, and incident history are often client-confidential. Run the scripted version inside the client's environment against their approved AI endpoint (Bedrock/Azure OpenAI in-tenancy), or redact service names to stable aliases before using a public assistant and re-substitute locally. For fully restricted estates, the structured templates in steps 2–3 work as human checklists too — the format is the transferable asset.

Success criteria

• •
  Release notes and risk brief produced in under 15 minutes per release, both from the same extraction.
• •
  Claim audit finds zero invented changes across three consecutive releases.
• •
  At least one deploy decision (timing, canary, extra review) visibly changed by a brief within the first quarter.
• •
  Change-review meetings consume the brief instead of reading raw PR lists.