Lab 02 · Diagram Pipeline: NL → Diagram-as-Code → Drift Check

Rajesh's team at Meridian Retail approved the support-copilot ingestion design three weeks ago. The build team has since shipped Terraform. Nobody has compared the two. This is the normal state of architecture documentation everywhere — and diagrams-as-code in git, drift-checked against IaC, is the practice that fixes it.

Time & tier

~90 minutes hands-on · Core tier (maps to module M2). Cost: ₹0 — Mermaid, git, and a chat assistant (free tier or local Ollama model); no cloud account, nothing is deployed.

Setup

Install git and any Mermaid preview (VS Code Mermaid extension, or the free mermaid.live editor). Create meridian-diagrams/, run git init, and add these two files.

File 1 — brief.md (the approved design intent):

Meridian Retail — support copilot: ingestion + serving brief (v3, APPROVED)

1. Support agents upload policy/product documents to an S3 bucket.
2. Upload events flow through an SQS queue (buffering, retry, backpressure)
   to an embedding worker (Lambda), which writes chunks and vectors to
   Postgres/pgvector on RDS.
3. Chat traffic: ALB → ECS Fargate service, which retrieves from pgvector
   and calls the LLM provider.
4. A Redis (ElastiCache) response cache sits in front of the LLM call;
   an identical question within 24 hours must not re-hit the model.
5. All compute runs in PRIVATE subnets; nothing but the ALB is
   internet-facing.
6. Single region at launch; multi-region is out of scope for v1.

File 2 — deployed/main.tf (what the build team shipped — an analysis snippet, not meant to apply; supporting network/IAM resources are omitted):

resource "aws_s3_bucket" "docs" {
  bucket = "meridian-support-docs"
}

resource "aws_s3_bucket_notification" "docs_events" {
  bucket = aws_s3_bucket.docs.id
  lambda_function {
    lambda_function_arn = aws_lambda_function.embed_worker.arn
    events              = ["s3:ObjectCreated:*"]
  }
}

resource "aws_lambda_function" "embed_worker" {
  function_name = "meridian-embed-worker"
  runtime       = "python3.12"
  handler       = "worker.handler"
  filename      = "build/embed_worker.zip"
  role          = aws_iam_role.embed_worker.arn
  timeout       = 300
}

resource "aws_db_instance" "kb" {
  identifier                  = "meridian-kb"
  engine                      = "postgres"
  engine_version              = "16.3"
  instance_class              = "db.t3.medium"
  allocated_storage           = 50
  username                    = "copilot"
  manage_master_user_password = true
}

resource "aws_elasticache_cluster" "response_cache" {
  cluster_id      = "meridian-cache"
  engine          = "memcached"
  node_type       = "cache.t3.micro"
  num_cache_nodes = 1
}

resource "aws_ecs_service" "chat_api" {
  name            = "meridian-chat-api"
  cluster         = aws_ecs_cluster.main.id
  task_definition = aws_ecs_task_definition.chat_api.arn
  desired_count   = 2
  launch_type     = "FARGATE"
  network_configuration {
    subnets          = [aws_subnet.public_a.id, aws_subnet.public_b.id]
    assign_public_ip = true
    security_groups  = [aws_security_group.chat_api.id]
  }
}

resource "aws_lb" "chat" {
  name               = "meridian-chat-alb"
  load_balancer_type = "application"
  subnets            = [aws_subnet.public_a.id, aws_subnet.public_b.id]
}

Guided steps

Step 1 — Generate the diagram from the brief (20 min)

Paste brief.md to your assistant and ask for a Mermaid flowchart LR covering both the ingestion path and the serving path, with the network boundary (private vs internet-facing) visible as subgraphs. Iterate until it renders cleanly. AI diagram generators are plausibility machines: they will happily add an API Gateway, a DLQ, or a second AZ that the brief never mentions, because those usually appear in such designs.

✅ Checkpoint: a rendering architecture.mmd showing upload → queue → worker → pgvector, and user → ALB → ECS → pgvector/LLM with a Redis cache.

Step 2 — Verify node-by-node against the brief (15 min)

Print the brief and the diagram side by side. For every node and every edge ask two questions: is it in the brief? and is everything in the brief on the diagram? Delete anything the AI invented; add anything it dropped (the 24-hour cache rule and the private-subnet boundary are the ones models most often lose). Keep a short verification-notes.md listing what you deleted or added.

✅ Checkpoint: verification-notes.md lists at least one invented or dropped element you corrected.

Step 3 — Make it a diagram-of-record in git (10 min)

Commit brief.md, architecture.mmd, and verification-notes.md with the message docs: approved ingestion+serving diagram v1 (verified against brief v3). The commit is the point: a diagram in a wiki screenshot cannot be diffed, reviewed, or drift-checked; a .mmd file in git can be all three.

✅ Checkpoint: git log shows the commit; git diff on a trivial edit shows a readable line-level diff.

Step 4 — Run the AI drift check against the Terraform (30 min)

Now paste architecture.mmd (your verified intent) and deployed/main.tf (the shipped reality) into your assistant with this prompt shape: "Compare this diagram-of-record with this Terraform. List every mismatch — components present in one but not the other, and attributes that contradict the diagram or the brief. Quote the exact Terraform lines as evidence." The Terraform contains exactly three mismatches. Your AI may report more (cosmetic naming noise) or fewer (it loves to assume the S3-to-Lambda wiring "implies" the brief's design). Adjudicate every claim yourself against the two files: confirm real mismatches, strike hallucinated ones, and keep hunting until you hold exactly three with quoted evidence.

✅ Checkpoint: drift-report.md lists exactly 3 confirmed mismatches, each with the offending Terraform lines quoted and a severity call.

Step 5 — Adjudicate and update the record (15 min)

For each mismatch, decide the resolution direction: change the infrastructure to match the intent, or change the diagram because reality is acceptable — and say who must approve which. (Hint: at least one of the three is a security matter where "update the diagram" is the wrong answer.) Commit the drift report and, where you accepted reality, the updated diagram, as a second commit.

✅ Checkpoint: second git commit containing drift-report.md with a resolution direction and approver for each mismatch.

Verification

• Diagram fidelityevery node/edge in architecture.mmd traces to a numbered line of the brief; nothing invented survives.
• Drift countdrift-report.md contains exactly the 3 seeded mismatches (check the answer key only after), each with quoted Terraform evidence.
• No false driftany extra mismatch your AI asserted is listed as struck, with the reason.
• Repo shapetwo commits: the verified diagram-of-record, then the drift report and resolutions.

Completion artifact

The meridian-diagrams/ git repo. Self-grading rubric (pass = 7/9):

• Mismatch capture (0–3)one point per seeded mismatch found with evidence.
• False-positive control (0–2)no unconfirmed AI claims in the final report = 2.
• Diagram verification (0–2)verification notes show real deletions/additions against the brief = 2.
• Resolution judgment (0–2)each mismatch has a direction and an approver; the security-relevant one is resolved toward intent = 2.

Client-safe variant

Client IaC frequently cannot enter public AI tools. The pipeline is unchanged: Mermaid, git, and diffing are all local. For the AI steps, use the client-approved tenancy or a local model (ollama run llama3.1 handles diagram generation and file comparison acceptably), and before prompting replace bucket names, account-specific ARNs, and CIDRs with placeholders — drift checking compares structure, so redaction costs nothing.

Stuck?

• •
  Mermaid won't render: subgraph names with spaces need quotes, and --> arrows must not have a space between dashes.
• •
  The AI's diagram keeps adding components: append "Include ONLY elements stated in the brief; if unsure, leave it out" to the prompt.
• •
  The drift check returns ten mismatches: most are naming or cosmetic; keep only differences that change behavior, topology, or exposure.
• •
  The drift check returns one mismatch: feed it the brief as well as the diagram, and ask it to check queueing, caching, and networking explicitly.

Answer key (don't read before attempting)

1. 1Missing SQS queuethe brief requires upload events to flow through an SQS buffer; the Terraform wires aws_s3_bucket_notification directly to the Lambda. Bulk uploads will throttle the worker; events can be lost during deploys. Resolve toward intent.
2. 2Wrong cache enginethe brief specifies Redis (ElastiCache); the Terraform provisions engine = "memcached". The 24-hour TTL-based response-cache semantics the brief assumes need to be re-validated; treat as resolve-toward-intent unless the build team documents equivalence.
3. 3Public exposure of computethe brief requires all compute in private subnets with only the ALB internet-facing; the ECS service runs in aws_subnet.public_a/b with assign_public_ip = true. This is the security mismatch: updating the diagram to match reality is the wrong answer.