Every professional now touches the API layer, because anything AI does repeatedly and unattended runs through one. This module covers the anatomy of a call, schema-enforced JSON contracts, tool calling, the failure modes that arrive inside a 200, and the cost arithmetic.
Somewhere between "I use ChatGPT to draft emails" and "I ship AI features" sits a layer every experienced IT professional now touches: the API. It doesn't matter whether your title says developer. QA engineers wire LLM calls into test pipelines. Sysadmins script ticket triage. PMs read API logs to debug an eval failure. Security engineers review what a prompt actually sends. The chat window is where you met the model; the API is where your organization actually uses it — and employer job postings have caught up to that fact.
An LLM behind an API is a metered, probabilistic, seconds-slow remote service that returns text unless you constrain it. The two skills that turn it from a toy into infrastructure are (1) making the call correctly and (2) forcing the output into a JSON contract your code can trust.
Our Phase 1 analysis of 2025–26 job postings found ten expectations that appear across all fourteen IT role families — not just software engineering. Number four on that universal list: LLM APIs plus structured outputs and function calling. It shows up in QA listings ("LLM tooling in CI/CD," "prompt engineering for test automation"), in DevOps listings ("we expect you to use LLMs heavily — runbooks, alert configs, postmortems"), in data engineering listings ("structured outputs" named verbatim alongside vector databases), and in PM interview loops that ask candidates to reason about token costs and failure modes.
The reason is mechanical: anything AI does repeatedly and unattended runs through an API, not a chat tab. A chat session has a human in the loop who tolerates a rambling answer. A nightly job classifying 5,000 support tickets has a json.loads() call that does not. The moment AI moves from "assistant I talk to" to "component in a workflow," three things you never thought about in chat become your problem: the request shape, the output contract, and the bill.
You don't need to become a machine-learning engineer. You need to be able to read an API call, predict what it costs, recognize why it failed, and specify the JSON you want back. That's this module.
Every major provider — Anthropic, OpenAI, Google — converges on the same request shape. Learn it once and you can read all three.
A request carries a messages array: alternating turns tagged user and assistant. The model's job is to generate the next assistant turn. Alongside the array rides a system prompt — persistent instructions that frame the whole conversation ("You are a release-notes writer. Be factual and concise."). System instructions carry more weight than user content, which is why production apps put role, rules, and output format there rather than repeating them in every user message.
The critical mental-model shift from chat: the API is stateless. There is no server-side conversation object. If you want turn five to remember turn one, you re-send turns one through four in the messages array on every call. "Memory" in every chat product you've used is just client code re-sending history — and paying for those tokens each time.
Models don't see words; they see tokens — subword chunks averaging roughly ¾ of an English word (code and non-English text tokenize less efficiently). Three numbers govern every call:
max_tokens parameter you set. Hitting the cap truncates the response mid-thought (you'll see a stop reason like max_tokens) — it's a ceiling, not a target.Every response also returns a usage object reporting exactly how many input and output tokens the call consumed. Log it from day one; it is the raw material for the cost section below.
Both examples do the same job — summarize a pasted incident report in three bullets — so you can see how little differs between vendors.
import anthropic # pip install anthropic; reads ANTHROPIC_API_KEY from env
client = anthropic.Anthropic()
response = client.messages.create(
model="claude-sonnet-5",
max_tokens=1024,
system="You summarize incident reports for engineers. Max 3 bullets.",
messages=[
{"role": "user", "content": f"Summarize this incident:\n{report_text}"}
],
)
text = next(b.text for b in response.content if b.type == "text")
print(text)
print(response.usage.input_tokens, response.usage.output_tokens)
from openai import OpenAI # pip install openai; reads OPENAI_API_KEY from env
client = OpenAI()
response = client.responses.create(
model="gpt-5.6",
instructions="You summarize incident reports for engineers. Max 3 bullets.",
input=f"Summarize this incident:\n{report_text}",
)
print(response.output_text)
Same idea, different spelling: a model ID, an instruction channel, the content, and a text result. Google's Gemini follows the same shape with the google-genai SDK (client.interactions.create(model="gemini-3.6-flash", input=...)). Once you can read one vendor's call, you can read them all — which is exactly why this course stays vendor-neutral.
Tutorials age fast in this space. As of mid-2026, current Anthropic model IDs are plain aliases like claude-sonnet-5, claude-haiku-4-5, and claude-opus-4-8 (no date suffixes), and OpenAI's primary surface is the Responses API — if a blog post shows date-stamped model strings or 2023-era parameters, check the official docs before copying it.
Free-form text is fine when a human reads the answer. Automation needs a contract: if your script expects {"intent": "billing", "urgent": true} and the model returns "This looks like a billing issue, and it seems urgent!", your pipeline is broken in a way that only surfaces at 2 a.m.
The naive fix — "please respond in JSON" in the prompt — works often, which is worse than never: it fails just rarely enough to reach production. The real fix is schema-enforced structured output, where the API itself guarantees the response conforms to a JSON Schema you supply. All three major vendors now offer it; the parameter names differ, the idea doesn't.
Anthropic — pass the schema under output_config.format on messages.create, or use the SDK's messages.parse() helper with a Pydantic model:
from pydantic import BaseModel
class Ticket(BaseModel):
intent: str # e.g. "billing", "support", "sales"
urgent: bool
summary: str
response = client.messages.parse(
model="claude-sonnet-5",
max_tokens=512,
messages=[{"role": "user", "content": f"Classify this email:\n{email_body}"}],
output_format=Ticket,
)
ticket = response.parsed_output # a validated Ticket instance
The raw-schema form is output_config={"format": {"type": "json_schema", "schema": {...}}} — the response text is then guaranteed-valid JSON for that schema.
OpenAI — the same capability is text.format with "type": "json_schema" on the Responses API (or response_format on Chat Completions), plus a strict: true flag; the Python SDK's .parse() helpers accept a Pydantic class directly, exactly like the Anthropic example above.
Gemini — pass response_format with "mime_type": "application/json" and a schema (Pydantic's model_json_schema() output drops straight in) on the interaction call.
Know the mapping — Anthropic: output_config.format (type json_schema); OpenAI: text.format / response_format with strict: true; Gemini: response_format with an application/json MIME type and schema. Assessments test whether you can name where the schema goes for each vendor.
Structured output is only as good as the schema. Habits that pay off:
"intent": {"enum": ["billing", "support", "sales", "spam"]} turns a fuzzy classification into a closed decision the rest of your code can switch on.required and set additionalProperties: false. Optional fields plus open objects reintroduce the ambiguity you were trying to remove.description fields. The model reads them — a one-line description per property is prompt engineering that lives inside the contract.Structured outputs constrain what the model says. Tool calling (also "function calling") lets the model ask your code to do things. You declare tools — each with a name, a description, and a JSON Schema for its arguments — and instead of answering directly, the model can respond with a structured request: "call get_invoice with {"invoice_id": "INV-4412"}." Your code executes the function, sends the result back as a new message, and the model continues — possibly calling more tools — until it produces a final answer.
Three things to internalize at literacy level:
On tool definitions, set the strict/validated mode your vendor offers (Anthropic: strict: true with additionalProperties: false and a required list) so tool arguments are guaranteed to match your schema — and always treat the parsed input as an object; never regex the serialized JSON string, since escaping varies across model versions.
Older material presents temperature as the master creativity dial: 0 for "deterministic," 1 for "creative." Two honest corrections:
temperature/top_p/top_k with a 400 error; you steer behavior with prompting and an effort setting (how hard the model thinks) instead.OpenAI and Gemini still expose temperature on most models, and it remains useful there for genuinely creative tasks — but the direction of travel is clear: design your systems to tolerate output variation via schemas and validation, not to suppress it via sampling parameters.
Determinism was never on the menu. If your workflow breaks when wording varies, the fix is a tighter contract (schema, enum, tool call) — not a lower temperature.
An LLM API fails like any remote service, plus a few ways of its own:
retry-after header; the official SDKs retry 429s and 5xx errors automatically with exponential backoff (Anthropic's default is 2 retries — configurable). For batch workloads, smooth your request rate rather than blasting and praying.stop_reason: "max_tokens") and a refusal (stop_reason: "refusal" on current Claude models) both arrive inside an HTTP 200. Check the stop reason on every call; a 200 does not mean "usable answer."Never blanket-retry 4xx errors, and never let a retry loop wrap a side-effecting tool call that isn't idempotent — a payment API called from inside a retried LLM turn will fire twice. Retries are for failures that heal themselves: 429, 5xx, and network errors.
LLM APIs bill per token, input and output priced separately — and output tokens typically cost about five times more than input. That asymmetry drives most cost intuition: a long prompt with a short answer is cheap; a short prompt demanding a 3,000-word report is not.
Anthropic's ladder (as of July 2026) illustrates the tiering pattern every vendor follows:
| Model | Input / 1M tokens | Output / 1M tokens | Role |
|---|---|---|---|
| Claude Opus 4.8 | $5.00 | $25.00 | Hardest reasoning, agentic work |
| Claude Sonnet 5 | $3.00 | $15.00 | Workhorse — strong quality per dollar |
| Claude Haiku 4.5 | $1.00 | $5.00 | High-volume simple tasks |
OpenAI and Google publish equivalent ladders (flagship / workhorse / mini-flash tiers) on their pricing pages — the shape is identical even when the numbers move. Four levers cut real-world spend more than model choice alone:
max_tokens sensibly, and ask for terse formats. Remember the stateless design: conversation history you re-send is input you re-buy every turn.The habit that makes all four possible: log usage on every call from day one. You cannot tier, cache, or trim what you haven't measured.
Do the arithmetic before you build. A workflow of 10,000 calls/day averaging 2,000 input + 300 output tokens is 20M input + 3M output tokens daily — about $105/day on a Sonnet-class model versus about $35/day on a Haiku-class one. Two minutes of multiplication beats a surprise invoice.
You do not need a corporate account to build the muscle:
gemini-3.6-flash from Python at no cost within rate limits. The lowest-friction way to run every exercise below.enum, one boolean, all fields required, additionalProperties: false — and use your vendor's structured-output mechanism to extract records. Validate each result in code (parse it, check the enum values). Add a repair step: on validation failure, retry once with the invalid output and the error message appended to the prompt.max_tokens=20 and confirm the truncation stop reason, (b) use a bogus model ID and observe the 4xx, (c) paste an input large enough to matter and compare the reported usage/cost against a small one.output_config.format, OpenAI text.format/response_format, Gemini response_format + schema) are the contract.usage from day one.output_config.format, schema limitations