Synthesize Team-Health Signals Without Skipping the Human Conversation

Workflow area: Leadership Ops · Tier: Core · Hands-on time: ~30 minutes

Engineering leaders sit on more team-sentiment text than they can process: quarterly survey comments, retro boards, pulse checks, onboarding feedback. Most of it is skimmed once and archived, which means the team keeps telling you things you keep not hearing. AI synthesis fixes the reading problem. It does not fix the listening problem — a theme extracted from 40 comments is a hypothesis about your team, with sampling bias, anonymity limits, and model over-coherence baked in. The playbook's output is a shortlist of conversations to go have.

When to use — and when not to

• Use it on survey free-text at every cyclethe numeric scores get read; the comments, where the actual information lives, usually do not.
• Use it across sourcesretro notes plus survey comments plus onboarding feedback, synthesized together, surface themes no single source shows.
• Use it when you suspect driftafter a reorg, a departure, a crunch — as a structured check on the "everything seems fine" your calendar view gives you.
• Skip it below an anonymity floorsynthesizing free-text from a 4-person team de-anonymizes it in practice; below roughly 8–10 respondents, read manually and quote nothing.
• Skip it for individual assessment, alwaysmining sentiment data to evaluate or identify a person is a betrayal of the survey's premise and, in most orgs, of explicit promises made when it ran.
• Skip acting on any theme unverifiedthe rule is structural: theme → conversation → then action; never theme → action.

The recipe

1. 1.

   Assemble and strip the corpus. Export survey free-text, retro board contents, and pulse comments for the period. Remove names, emails, and metadata columns that could identify authors (tenure + team + role often triangulates a person — drop to the coarsest grouping that is still useful). This sanitization happens before anything is pasted anywhere.

2. 2.

   Run the theme synthesis.

   Copy

   You are synthesizing team-health signals for an engineering
   leader. Sources: [e.g. "Q2 engagement survey free-text (34
   comments), 6 sprint retro boards, 3 onboarding surveys"].
   All text is anonymized; treat it as such.
   
   [PASTE CORPUS]
   
   Produce:
   1. THEMES — recurring concerns or positives, each with: a
      neutral one-line name; how many distinct comments support it
      (count, don't estimate); 2-3 representative verbatim quotes;
      and a confidence grade — HIGH (many sources, consistent),
      MEDIUM (one source or mixed), LOW (few comments, could be
      one loud voice). 
   2. TENSIONS — places where comments directly contradict each
      other; do not average them into a mushy middle.
   3. ABSENCES — topics notably missing given the period (e.g. an
      incident, a reorg) — silence is a signal too.
   4. NOT-A-THEME — things mentioned once that I might overweight
      because they are vivid.
   
   Rules: quotes verbatim, never composited from fragments. Do not
   diagnose causes. Do not recommend actions. Count honestly — 
   "several" is not a number.
   

3. 3.

   Audit the quotes and the counts. Good output looks like:

   Copy

   THEMES
   - On-call load unsustainable — 11 comments — HIGH
     "third weekend page this month and nothing changes" 
     "I dread the rotation more than deadlines"
   - Review turnaround improving — 5 comments — MEDIUM
   TENSIONS
   - "leadership is transparent" (4) vs "decisions appear from
     nowhere" (6) — likely differs by sub-team; cannot confirm
     from anonymized data.
   

   Spot-check every quote you might ever repeat: find it verbatim in the source. A composited or "cleaned" quote attributed to your team is a fabrication with your name on it. Then sanity-check counts — models miscount; if a theme's number drives a decision, count it yourself.

4. 4.

   Convert themes into conversations, not conclusions. For each HIGH theme, decide where the human verification happens: an open question in the next team meeting, a topic across your 1:1s, a dedicated retro. Bring the theme, not the quotes — "I'm hearing on-call load is wearing people down; what's your experience?" — so nobody plays guess-the-author.

5. 5.

   Close the loop publicly. The fastest way to kill survey participation is synthesis that vanishes. Share back: the top themes, what you verified in conversations, what you are doing, and what you heard but are not acting on and why. Teams forgive "no" far more readily than silence.

Make it yours

• With a DX/Swarmia-class survey platform: let the platform do first-pass theming, then run this synthesis across its output plus your retro notes — the cross-source pass is where the platform's single-source view breaks.
• Small org, no survey program: run a lightweight quarterly pulse (three open questions), and lean harder on the anonymity floor — with 12 respondents, report only themes with 3+ supporting comments and paraphrase rather than quote.
• Org-level leader: synthesize per-team first, then across teams, and resist the flattening — "morale is mixed" across five teams usually means two specific teams are struggling; the roll-up hides exactly what you need to see.

Quality check

• Every quote is verbatim and every count is realspot-check both; these are the two places models silently fabricate.
• Confidence grades survived into your thinkingif you are about to act on a LOW-confidence theme because it is vivid, the NOT-A-THEME section was written for you.
• The anonymity promise held end to endno identifying metadata pasted, no quote repeatable to a room that could source it, no per-person anything.
• At least one human conversation happened per acted-on themecheck yourself honestly: did you verify, or did you go straight from synthesis to fix?
• The team saw the loop closesynthesis the team never hears about trains them to stop writing comments.