Release-Engineering Copilot: Notes, Metadata, Rollout Health

Tier: Core · Workflow area: Mobile engineering & release

This playbook builds the copilot around your existing release train — generation is cheap; the discipline is in the source data, the translation gates, and the go/no-go evidence.

When to use

• •
  You cut releases at least monthly and the notes/metadata/health-report toil is measured in hours per release.
• •
  Store listings exist (or should exist) in multiple locales and currently lag the English copy by weeks.
• •
  Staged rollouts happen, but the "is it healthy?" check is one engineer squinting at Crashlytics.

Skip the localization half if you genuinely ship one locale; keep the notes and health halves — they pay at any scale.

Prerequisites

• •
  Merged-PR history with usable titles/labels between release tags (git log v4.11.0..HEAD --merges or the GitHub API). Garbage titles in, garbage notes out — a release-notes label convention fixes this cheaply.
• •
  fastlane (or equivalent) already doing the mechanical delivery: deliver/supply for metadata, so generated text drops into fastlane/metadata/<locale>/ files rather than a web form.
• •
  Access to rollout health sources: Crashlytics/Play vitals, App Store Connect metrics, and your analytics tool's release dimension.
• •
  An AI assistant; a scriptable one (API or terminal agent) if you want this in CI rather than a chat window.

The workflow

1. 1.

   Generate release notes from the real changelog, in two registers. Feed the merged-PR list between tags, tagged by label:

   Copy

   Here are merged PRs between v4.11.0 and v4.12.0 with labels.
   Produce: 1) user-facing release notes - max 500 chars, benefits not
   implementation, no ticket numbers, skip internal/infra PRs entirely;
   2) internal notes for support and QA - every user-visible change,
   known risks, and which flags gate what.
   Do not mention anything not present in the input list.
   

   The last line matters: models pad thin releases with invented polish ("bug fixes and performance improvements" is fine; "faster sync" when nothing touched sync is a support incident).

2. 2.

   Draft store metadata as structured files, not prose in a chat. Have the assistant write directly into the fastlane layout — release_notes.txt per locale, and when the listing itself changes, description.txt/promotional_text.txt — respecting hard limits (Play: 500 chars for what's new; App Store: 4000-char description). Character limits go in the prompt and get validated by a script, not by trust.

3. 3.

   Localize with translation discipline, not vibes. For each target locale:

   Copy

   Translate this release-notes text to de-DE, fr-FR, ja-JP, hi-IN.
   Rules: keep product names and feature names untranslated per this
   glossary; match the register of the existing store listing (pasted);
   stay under 500 characters AFTER translation; flag any line that is
   culturally awkward or unclear rather than translating it literally.
   

   Then gate: machine-translated notes for minor releases can ship after a fluent-speaker skim; listing copy and top-revenue locales get real native review. Track which locales got which gate.

4. 4.

   Assemble the rollout-health brief from numbers you paste, not numbers it remembers. At each stage gate (day 1 at 10%, day 3 at 25%...), collect crash-free users, ANR rate, top new crash clusters, key funnel metric, and store rating delta — then:

   Copy

   Compare v4.12.0 at 10% rollout vs v4.11.0 full rollout on the pasted
   metrics. Output: verdict (expand / hold / halt) with reasoning,
   the three most concerning signals, and what to check before the next
   gate. Flag any metric where the comparison is statistically dubious
   because of small cohort size.
   

   The AI formats and flags; the release owner makes the call and records it. Wire the halt path to fastlane or the Play Developer API so "halt" is one command, not a scramble.

5. 5.

   Template the whole thing into the release ritual. Put steps 1–4 in a runbook (or a fastlane lane calling a script) so the copilot runs identically for every release, and archive each release's notes, metadata diff and health briefs in one folder — that archive is your audit trail when a rollout question comes back a month later.

Tool variants

• Claude/Gemini via API in CI: notes and metadata generation as a pipeline step on tag creation; PR opened against fastlane/metadata/ for human review.
• Terminal agent (Claude Code): interactive mode — it can run the git log, read labels, write locale files and run the char-limit validator in one loop.
• Chat assistant only: paste the PR list and metrics manually; keep the prompts in a team doc so output stays consistent across release owners.
• Play Console / App Store Connect AI features: use where present, but keep your own pipeline as the source of truth so both stores say the same thing.

Verification

• •
  User-facing notes claim only shipped, user-visible changes — checked against the PR list line-by-line by the release owner.
• •
  Char-limit validation passes for every locale file before deliver/supply runs.
• •
  Each localized file passed its designated gate (skim vs native review) and the gate level is recorded.
• •
  Every rollout expansion has an archived health brief with the actual metrics and the human verdict.
• •
  Support/QA confirm the internal notes matched what users then reported — a monthly spot check.

Pitfalls

• Notes generated from commit messages instead of curated PR titles. You will publish "fix ci flake again" to the App Store. Curate at the PR level with labels.
• Invented improvements. The model fills silence with benefits; the "only from input" clause plus review catches it.
• Translation without a glossary. Feature names get translated inconsistently across locales and releases; the glossary file is mandatory, versioned, and grows.
• Health briefs from memory. If you let the model "recall" typical crash rates, it will. Every number in the brief must appear in the pasted input.
• Ratings-blind rollouts. Crash-free rate can hold while reviews crater over a UX change; the ratings delta belongs in the brief.

Restricted-environment variant

PR titles and store copy are usually shareable; metrics dashboards may not be. If policy restricts external AI: run generation through your enterprise tenancy, or split the work — notes and translations via the approved cloud tool (public-facing text anyway), health briefs via a local model or a plain script that just formats the comparison table without narrative.

Success criteria

• •
  Release-note and metadata prep drops from hours to minutes per release, with zero store rejections for limit or policy violations.
• •
  All target locales ship same-day with the release, each through its recorded review gate.
• •
  Every stage-gate decision is archived with evidence; halts happen at 10%, not after full rollout.
• •
  A new release owner can run the entire ritual from the runbook on their first train.