The AI-Ready Database Estate — Vectors in Your Engine

The estate argument is simple. In 2025–26 every major engine shipped native vector search: pgvector default-enabled on every managed Postgres, SQL Server 2025 with a native VECTOR type and DiskANN indexes, Oracle 23ai's AI Vector Search included at no extra charge, MongoDB Atlas with hybrid search GA even for on-prem Community deployments. That means the org's first semantic-search feature does not need a new platform, a data movement pipeline, or a vector-database vendor — it needs the DBA to enable and operate a capability inside the engine they already run. Demos are everywhere; operations is the gap, and operations is your job description.

One boundary, stated up front: corpus pipelines, chunking strategy, and warehouse RAG engineering belong to the data-engineering track. This module owns what happens inside the engine — enablement, indexing, lifecycle, and capacity — on data that already lives there.

The scenario — the index build that ate the maintenance window

Shruti is the senior DBA at a Hyderabad B2B marketplace: PostgreSQL 16 on RDS, a products table at 14 million rows. The app team wants "AI search" and has a Pinecone quote in hand; she counters that the data is already in Postgres and pgvector will do it without moving anything. She is right — and then the first production step goes wrong in a way no tutorial covers.

Same concepts, different knobs — the engine map

Learn the concepts once; port them per engine. The transferable core:

• A vector column typefixed-dimension float arrays as first-class data, with half-precision variants where storage bites.
• An approximate-nearest-neighbor indexgraph-based (HNSW or DiskANN): fast, memory-hungry to build, and approximate by design.
• A distance functioncosine, dot product, Euclidean; chosen to match the embedding model, not by taste.
• A recall-versus-latency trade-offtunable at build time and query time; this is where the DBA's measurement habit earns its keep.
• An external embedding modelthe one piece that never lives in the engine, and therefore the dependency you must wire, secure, and budget.

Vendor claims ride along with the features — MongoDB's "+30% retrieval quality" from native reranking is a vendor number; Oracle's positioning, Microsoft's benchmarks likewise. The features are real and verifiable on your own data; the percentages are marketing until you reproduce them.

The DBA concerns nobody demos

This list is the module's core. Every one of these hit Shruti or will hit you, and none appear in quickstarts:

• Index build memory and timegraph indexes want the working set in memory during build; undersized build memory turns hours into days. Size it, schedule it as a change, and know the rebuild story before you need one.
• Replica behaviorindex builds and bulk embedding loads are WAL/redo-heavy; replicas lag exactly like any bulk operation. Vector work goes through the same replica-lag checkpoints as a Module 2 backfill.
• Backup and storage growthembeddings are big: dimensions × 4 bytes × rows, plus index overhead. 14M rows × 1,536 dims ≈ 90 GB before the index. Forecast it, and evaluate half-precision types (halfvec) before defaulting to full float.
• Query plans when vector search meets WHEREthe hard case is hybrid: nearest-neighbor plus relational filters (category = 'pumps' AND in_stock). Pre-filter versus post-filter changes both correctness (post-filtering can return fewer than k results) and performance; pgvector 0.8's planner work targets exactly this. Read these plans like any other plan — this is why the estate needs a DBA at all.
• Recall is a tunable, not a givenapproximate indexes trade recall for speed via build parameters (m, ef_construction) and query-time parameters (ef_search). "It returns results" is not "it returns the right results" — recall@k against a brute-force exact baseline is the measurement, on your data, not the README's.
• The external-model dependencyembedding generation calls something outside the engine: an API, an Ollama box, a platform service. That dependency has latency, cost, quotas, an availability story, and (for SQL Server's REST endpoint) a security review. The database now has an upstream — treat it like one.

The embedding lifecycle — vectors go stale

The demo embeds once. Production data changes, and a vector computed from last month's product description is a wrong answer waiting to surface. Embedding lifecycle is an ops responsibility with four parts:

1. 1Refresh on change. Row updates must queue re-embedding — trigger-driven queue tables, CDC consumers, or engine-native automation (Atlas auto-embeddings, SQL Server external-model calls, Oracle DBMS_VECTOR pipelines). Synchronous embedding inside the write path adds an external API call to your OLTP latency — almost always wrong; queue it.
2. 2Detect drift. Whatever the refresh mechanism, it will miss rows — bulk loads that bypassed triggers, queue failures, restored backups. A reconciliation query comparing text-change timestamps against embedding-refresh timestamps finds text-vector drift; run it on a schedule, alert on the count.
3. 3Plan the model upgrade. Embeddings from different models are not comparable — upgrading the model means re-embedding everything. That is a full-table operation with real cost (rows × tokens × price, or GPU-hours), a batching plan, and a mid-migration answer: dual columns, or a cutover window where search quality dips. Write this runbook before the first model deprecation notice, not after.
4. 4Budget cost and latency. Embedding calls are metered. A hot table re-embedding thousands of rows an hour has a monthly bill and a queue-depth SLO; both belong in the same capacity review as storage and IOPS.

Shipping the win — semantic search on data you already have

This is the career argument made concrete: a working "search that understands meaning" feature over an existing products, tickets, or documents table — shipped engine-side by the DBA, with recall measurements and a capacity forecast attached — is the most legible possible proof that the database team is where AI capability lives. It reuses everything this track has taught: the change gates from Module 2 for the index build, the measurement-first discipline from Module 1 for recall tuning, and it sets up Module 5, where access to this new capability gets governed.

Try it now

1. 1Do the storage math. For your most search-worthy table: rows × embedding dimensions × 4 bytes, plus ~30% index overhead. Write down the number, the halvec/half-precision alternative, and what it does to your backup window.
2. 2Stand up the local proof. Local Postgres + pgvector, 1,000 rows of real-shaped text, embeddings from a local model (Ollama), an HNSW index, five test queries. Note the index build time and memory — at toy scale, as calibration.
3. 3Draft the drift query. For that same table, write the reconciliation query you would run in production: rows where the text's updated timestamp is newer than its embedding's — the query that catches what the refresh pipeline misses.

Check yourself: The app team says vector search results "seem worse lately" — no errors, no slow queries. List your first three checks, in order, and what each would prove.

This module anchors LAB-07 (pgvector semantic search in a live app schema) — embeddings, HNSW tuning with recall@k versus latency measured across parameter sets, and a 10x storage forecast — and LAB-08 (embedding lifecycle ops inside OLTP) with its trigger-driven refresh queue and seeded drift hunt. Do it at work this week → pb-28 (pgvector like an operator), pb-29 (engine-side semantic search), pb-30 (embedding lifecycle), pb-31 (vector index tuning), and pb-32 (porting the mental model to SQL Server 2025 and Oracle 23ai).