One-Off Scripts, Regex, jq and SQL — with Mandatory Verification

Tier: Foundation · Workflow area: Feature development

AI also confidently writes ones that are subtly wrong, and "subtly wrong data-fix SQL" is a career event. The playbook is one habit: generate, then verify on a sample, then run — never generate-and-run.

When to use

• •
  Any single-use script, query, or incantation: jq/awk/sed, regex, ad-hoc SQL, pandas snippets, curl loops, git surgery.
• •
  Tools you use too rarely to hold in your head (jq syntax, awk field logic, git filetree archaeology).

Skip AI for one-liners you already know cold — typing beats prompting at that size.

Prerequisites

• •
  Any chat assistant; no repo access needed.
• •
  A safe sample of the real input (3-5 representative records/lines, sanitized).
• •
  For anything destructive: a dry-run path (SELECT before UPDATE, --dry-run, output to a new file).

The workflow

1. 1.

   Show real input, state exact output. The number-one quality lever is a genuine sample:

   Copy

   Write a jq filter. Input (one of ~5,000 similar objects):
   {"id":"ord_91","status":"failed","attempts":[{"ts":"2026-07-20T10:11:02Z",
   "error":"card_declined"},{"ts":"2026-07-21T09:02:44Z","error":"timeout"}]}
   
   Output: CSV lines `id,last_error,last_attempt_ts` — the error and ts of
   the LATEST attempt only, ordered by ts. Objects with empty attempts
   should output `id,,`. Watch out: attempts are not guaranteed sorted.
   

2. 2.

   Ask for the trap list. One extra sentence catches most edge-case bugs: "List the input variations that would break this filter." For regex: "Show three strings that match and three near-misses that don't." You are making the model generate its own test cases.

3. 3.

   Run on the sample first. Paste the sample through the actual command locally. Check the edge case (empty attempts) by hand. For regex, test the near-misses. Ten seconds of sample-run catches the classic failures: unsorted arrays, greedy quantifiers, locale-dependent sort, null vs missing key.

4. 4.

   For destructive operations, demand the dry-run form first. Data-fix SQL example:

   Copy

   Postgres. Fix: orders stuck in status='processing' older than 7 days whose
   payment succeeded should become 'paid'. Tables: orders(id, status,
   updated_at), payments(order_id, state).
   Give me: (1) a SELECT that returns exactly the rows the UPDATE would touch,
   with a count; (2) the UPDATE wrapped in an explicit transaction with the
   same WHERE; (3) what could make (1) and (2) diverge between runs.
   

   Run the SELECT, eyeball the count against your expectation ("should be ~40, not 40,000"), then run the UPDATE inside BEGIN/verify/COMMIT.

5. 5.

   Sanity-check the totals. After the real run: row counts in vs out, wc -l, a spot-check of two known records. If the number surprises you, the script is wrong until proven otherwise.

Tool variants

• Chat assistant (any vendor): the default; keep a running "shell snippets" conversation so context (your OS, shell, jq version) persists.
• Terminal agent: can run and iterate the command itself against the sample — great for gnarly multi-step pipelines; keep it away from production credentials.
• IDE inline completion: fine for pandas/py one-offs inside a scratch notebook where you will see output immediately anyway.

Verification

• •
  Command ran against a sample and produced hand-verified output, including at least one edge case, before touching full data.
• •
  Destructive ops: SELECT-preview count matched expectations; UPDATE ran in a transaction; post-run count matched the preview.
• •
  Regex: tested against near-miss strings, not only happy matches.
• •
  Output totals reconcile with input totals (or the difference is explained).

Pitfalls

• Generate-and-run. The whole failure class. The habit is a checklist of one item: sample first.
• Describing input instead of showing it. "JSON with nested attempts" yields a filter for the model's imagined JSON. Paste the real shape.
• Version drift. jq 1.6 vs 1.7, GNU vs BSD sed/awk on macOS, MySQL vs Postgres date math — state your versions or expect incantations for someone else's machine.
• Trusting rm, git push --force, DROP suggestions. Anything irreversible gets typed by you, deliberately, never pasted from a chat.

Restricted-environment variant

The scripts are rarely sensitive; the data is. Paste structure, not payloads: replace real values in your sample with fakes of the same shape ("email":"user1@example.com", realistic-length IDs) — the model only needs shape and types. Production data with personal information (names, phones, addresses) must not enter hosted tools; under India's DPDP Act treat customer fields in dumps as regulated data. Local models handle this playbook well — jq/regex/SQL generation is comfortably within an 8B-coder's range, so a fully offline Ollama workflow is a real option for client-side work.

Success criteria

• •
  Task done in under 10 minutes that would have cost 30+ of manual docs-diving.
• •
  Zero incidents from ad-hoc scripts: every destructive run was previewed and transaction-wrapped.
• •
  You caught at least one model mistake at the sample stage — proof the verification habit is live, not ceremonial.
• •
  Your prompt included real sample input without leaking real customer data.