Reviewing AI-Written SQL, and What Appreciates in the AI Era

Correct syntax, wrong meaning

Omni's April 2026 analysis of text-to-SQL failures put 81.2% of them in the schema and semantic bucket, not the syntax bucket. That statistic describes a specific and uncomfortable experience: the query runs, the numbers look like money, the chart renders, and every figure is wrong.

Here is the query an assistant produced when asked for "monthly revenue" against the schema from the previous lesson.

-- What the assistant wrote. Every line parses. Every number is wrong.
SELECT strftime('%Y-%m', o.order_dt) AS month,
       SUM(o.gross_usd) AS revenue
FROM orders o
JOIN order_items i ON i.order_id = o.order_id
GROUP BY month
ORDER BY month;

Four defects, none of them a syntax error.

1. The join fans out. order_items has one row per line item, so joining it multiplies each order row by its item count. An order of three items contributes its gross three times. This is the single most common way a generated aggregate is silently inflated, and it is invisible unless you know the child table's cardinality.
2. The same join silently drops orders. It is an inner join, so any order with no line item at all vanishes from the result. Fan-out inflates the months full of multi-item orders and the dropped parents deflate the months full of item-less ones, which is why you cannot assume a fanned-out number is simply too big. It can land on either side of the truth, and in this data one month lands on each side.
3. The required filter is missing. The contract says status = 'complete'. Cancelled and pending orders are being counted as revenue.
4. It reports gross, not net. The contract defines net_revenue as SUM(gross_usd - refund_usd). Refunds never leave the number.

Defects 1 and 2 are the reason a review is a query rather than an opinion. Two errors pointing in opposite directions can cancel to something that looks plausible, so the only way to know the size of the damage is to compute both numbers and subtract.

Notice that the join was not even needed. Nothing in the request required a line item. The generator reached for a related table because the schema said the tables were related, which is exactly the guess a semantic model's predefined join paths exist to remove.

The four checks

Reviewing generated SQL is a repeatable procedure, and it is the durable half of this skill because it works on a colleague's query, on your own at 6pm, and on anything a model writes next year.

Order of evaluation

1. Grainone row per what?
2. Filtersis every required predicate present?
3. Fan-outdoes a join multiply the measure?
4. Contractdoes it match the metric definition?

The fan-out check has a fast field test: count the rows before and after the join. If the row count grew, any SUM of a parent-table column grew with it. If it shrank, parents without a matching child were dropped and the same SUM is now too small. When you genuinely need both the parent measure and the child detail, aggregate the child first in a CTE and join one row to one row, or use a correlated subquery. Do not aggregate across a fanned-out result and hope.

The strongest form of a review is not an opinion, it is a query. If you can produce the months where the suspect number and the contract number disagree, and the size of each gap, the conversation is over in one message. That is the closing exercise of this course.

What commoditizes, and what appreciates

Writing SQL is being automated in public. dbt Copilot reached GA in March 2025; Databricks shipped Assistant, Genie Code, and Lakeflow through spring 2026, with a cited benchmark moving coding-agent success from 32.1% to 77.1% once Genie Code was in the loop. Treating "I can write a window function" as your differentiator is a losing bet.

What appreciates is everything the generator cannot do because it does not know your business: modeling and grain decisions, data quality, cost governance, and review judgment. The adoption data says the market agrees. Only about 5.2% of organizations run AI agents in production. 72% want AI-assisted coding, but only 24% trust AI to manage pipelines. Gartner expects 60% of agentic analytics projects that rely on MCP alone without a semantic layer to fail by 2028.

Read the gap between 72 and 24 carefully, because it is a job description. Closing it requires reasoning traces, auditable logs, human-in-the-loop checkpoints, and eval harnesses. That is more engineering rigor, not less, and it is rigor of a kind this course has been teaching since the first idempotent pipeline.

One honest closing note about this whole level. None of the AI-era material is intern-loop core yet. A 2,817-report corpus of mainstream data-engineering interviews contains zero RAG or vector questions, and fundamentals still gate the hire. But 2026 job descriptions ask for it, AI-native startups test it, and it all compounds on the fundamentals rather than replacing them.

Common mistake: trusting a query because it executed and returned plausible numbers. Execution proves the syntax, nothing more. The three defects above all survive execution, and two of them survive a code review that only reads the query without knowing the contract.

Interview nuance: "here is a query our AI assistant wrote, review it" is entering loops at AI-forward companies, and the fan-out plus missing-filter combination is the archetypal planted bug. Say the checks out loud as you read: grain, filters, fan-out, contract. Then offer to prove it with a diff query rather than asserting it, because that is what a senior reviewer actually does.

On a real platform this differs. Here you diff two CTEs in SQLite. In production the same reasoning is a reconciliation test in dbt or a data-diff job comparing a candidate model against the governed metric, wired into CI so a query that disagrees with the contract fails the pull request instead of reaching a dashboard. The query you are about to write is that test, minus the scheduler.