The first time I trusted an automated "fix" to a pipeline, it fixed the error, and the fix was wrong. The job went green. The row counts didn't. It took 3 weeks and an angry finance analyst before anyone noticed that the dedupe logic had been "simplified" right out of existence.

So when Snowflake announced on Nov 3 that AI agents would now investigate failed pipelines and rewrite Spark jobs for you, I didn't panic. I did what I always do: I read the release notes, the docs, and the footnotes. My take is that this launch makes the Data Engineer who can review a change and own it worth more than before, while the engineer whose whole value is typing out the fix gets cheaper. Your next interview loop will start to reflect that.

What Snowflake shipped, and what is still in preview

Marketing pages blur everything together, so here's the status of each piece:

  • CoCo Cloud Agents: generally available.
  • CoCo Automations: GA as of Nov 2, but only in Snowsight and the CLI, and only in commercial AWS, Azure, and GCP regions. Cloud automations in CoCo Desktop are still preview. Government, FedRAMP, and China deployments don't get it.
  • Pipeline Builder: public preview, and it has been since Oct 22. The Nov 3 release just restated it.
  • CoCo Spark-to-Snowflake rewrites: marked GA in the launch press release footnotes. I couldn't find a standalone release note for it, so the GA status comes from one source.

This status moved fast. Automations were preview on Aug 21 and GA by Nov 2. Anything you read over the summer probably has the status wrong.

The headline pitch is in Snowflake's Nov 3 press release: when a scheduled Automations run "finds a problem in a pipeline, CoCo can trace the likely cause and propose a fix for engineers to review before they decide what goes to production."

Now read the mechanics. Per the docs, automations run on a schedule (hourly at most), with no event triggers, and run history is kept for 2 months. Each run runs as the user who created it, and you get an audit trail of what CoCo checked, found, and did. You pay standard task billing plus CoCo tokens for every run. The Automations blog describes all of that. What I couldn't find described anywhere is an approval gate that physically stops the agent from applying a change before a human signs off.

The review step is only a real control if the agent can't do the thing without you, and without that gate the guardrail comes down to role grants and scope. If your automation runs as a user with write access to prod tables, then the "human review" is whatever that role allows. Your security model is the actual reviewer, and you designed it, so check it.

Snowflake's own engineering team has been running an internal version of this. Their Oct 19 post on fixing pipeline failures with AI reports under 16 minutes on average from detection to PR, 100+ merged PRs, and 500+ pipelines improved over 5 months. Their framing: "The agent proposes the repair. A person still decides what ships."

Those are good throughput numbers. The post doesn't report an error rate, a rollback rate, or how many PRs a human rejected for being subtly wrong. Speed to PR shows the agent is fast and says nothing about whether it's right. My job-went-green-and-the-numbers-were-wrong story would have scored great on time to PR.

Data teams already distrust the part Snowflake automated

The vendors are selling hard. The practitioners are hedging.

dbt Labs' State of Analytics Engineering 2026 surveyed 363 data people between Dec 2025 and Feb 2026. 72% prioritize AI-assisted coding. Only 24% prioritize AI-assisted pipeline management, which includes testing and observability. And 71% cite incorrect or hallucinated outputs reaching stakeholders as a top concern.

Some caveats, because I'd want them if I were reading this. These are priority shares, so they don't measure adoption, and nobody's claiming 72% of teams run AI coding in prod today. The 71% measures concern; the survey reports no hallucination rate. It's also a vendor survey with a sample of 363, so treat it as a direction, not gospel.

Even with those caveats, the direction is pretty clear. Teams are happy to let AI write the code. They don't trust it to watch the code. Snowflake just shipped a product aimed right at the part people trust least.

The same report has the stat I find most interesting. The share saying trust in data and data teams is important went from 66% in 2025 to 83% in 2026. Meanwhile 57% report higher warehouse and compute spend, versus 36% reporting bigger team budgets. When spend climbs faster than headcount money and trust tops the worry list, an agent gets bought, and somebody has to be accountable for what it does.

The agent can open the PR in 16 minutes. Someone still has to know the PR is wrong, and that someone is the job.

I've been through 3 waves of "data engineering is getting automated away." Every wave automated the typing and left behind the judgment calls: what's the grain, who owns this table, is this late data or missing data, did the upstream team change the contract again. Schema drift, late-arriving data, and silent row drops don't care what tool you're on. The agent will hit those same problems and propose fixes that look plausible, which is exactly what makes them dangerous.

There's also a review-queue problem. LinearB's 2026 benchmarks, drawn from 8.1M+ PRs (vendor data, so salt accordingly), found AI-generated PRs wait 4.6x longer to be picked up, then get reviewed 2x faster once someone opens them. I read that as reviewers avoiding the PR, then rubber-stamping it. If Automations starts dropping proposed fixes into your thread every hour, your team's bottleneck becomes reviewer attention. Stack 20 agent proposals on a Monday and watch how carefully anyone reads number 17.

The fix is boring and old. Write-Audit-Publish was in a Netflix data quality deck back in 2017: write to staging, run audit checks, publish only when they pass. Snowflake doesn't ship WAP as a feature, and its docs say you can't create Iceberg branches on the Horizon external-engine path. So you build it yourself: a staging table, audit queries that check row counts, null rates, and key uniqueness against the last good run, and a swap you control. The agent proposes the change, the audit queries decide whether it can publish, and a human approves the swap, so nobody can skip the review.

Writing those audit checks is your job, and it's the skill that matters here.

Spark rewrites and the silent bugs data engineers will own

The Spark migration piece is where I'd be most careful, because it's where the bugs hide best.

Snowflake's CoCo Spark Migration skill "assesses compatibility, converts code and validates the changes," and the docs target the PySpark 3.5.3 Spark Connect DataFrame API. Some things just don't come over: RDD and low-level APIs (sc.broadcast, sc.accumulator, foreach), Structured Streaming, and writer methods like bucketBy and insertInto. Those fail loudly, which is fine, because you find them on day one.

The silent stuff is what worries me. Straight from Snowflake's own known-differences page:

  • Struct field order: Snowflake alphabetizes struct fields, while Spark preserves definition order. If your SCD Type 2 logic hashes a struct to detect changes, every hash changes. Congrats, every row in your dimension is now "updated."
  • Integer overflow: Spark wraps on overflow; Snowflake doesn't unless you set handleIntegralOverflow. Same code, different numbers.
  • Pre-Gregorian dates: may be "corrupted silently" when Parquet rebase modes aren't supported. The docs actually say "silently."

None of these throw an error. The job runs and the dashboard loads, but the numbers are wrong in a way only someone who understands the data model would catch. A rewrite that compiles and passes a row count check can still blow up your slowly changing dimension. Reviewing these rewrites means reviewing semantics. If you don't know why your SCD2 logic hashes what it hashes, you can't review the agent's version of it.

Then there's cost. Snowflake's customer story says Klarent cut TCO by 60% versus Amazon EMR, and their longest pipeline went from 10 hours 13 minutes to under 3. That's one customer, quoted by the vendor, with no time period, dollar figures, or methodology published. Snowflake's own engineering blog is more honest about it: "TCO therefore remains an estimate until the port is far enough along to compare."

So baseline before you touch anything. Get a full month of runtime and cost on the current platform, on a fixed workload. Otherwise you'll be "proving" savings against a number nobody wrote down. I've watched a migration get declared a win because nobody remembered what the old bill was. (Somebody got promoted for that one.)

This is also a competitive story. Databricks announced in August that it crossed a $7B revenue run-rate with >80% YoY growth. Snowflake making it easy to bring Spark over is an obvious move in that fight. Fine. Vendors fighting over your workloads is good for your leverage. It doesn't change who's on call when the rewritten job double-counts revenue.

What this means for the data engineer interview loop

A warning first: I found no primary source on what DE interviewers are testing right now. What follows is my opinion, from sitting on both sides of lots of loops.

What I can source is the shape of hiring. Indeed Hiring Lab found senior US postings up 14.7% year over year as of May 2026, with entry-level down 7.5%. In tech, the senior share of postings rose 9 points from 2019 to 2025, mostly at the expense of mid-level roles. That's all postings, not DE specifically, so don't stretch it too far. One detail stood out to me: Indeed labels seniority by task content, like ownership and decision rights, rather than by title. "Senior" in those postings looks like "the person who decides what ships."

That lines up exactly with what an agent like CoCo needs sitting next to it. Here's where I think loops drift over the next year or two:

Review questions will replace some build questions. I've said for a while that if AI can solve a medium coding problem, asking it tells me very little about you. Expect more "here's a PR, what's wrong with it" questions. Give a candidate an agent-proposed fix for a failed job and see if they spot that it drops late-arriving records. That's a far better signal than reversing a linked list.

Data modeling will get more weight, because you can't review a fix if you don't know the grain. You can't catch the struct-hash SCD2 bug if you don't understand how change detection works. Keep fact tables at grain; you can aggregate up, never down. Interviewers who've been burned by a bad automated change will probe for exactly this.

Ownership and permissions will show up in pipeline architecture rounds: "Your agent runs as a service role on an hourly schedule. What can it touch? Where's the gate? How do you roll back?" If you can talk through a write-audit-publish flow, role scoping, and a rollback plan, you sound like the senior person in those Indeed postings.

Cost judgment gets tested too. "Vendor says 60% cheaper. How do you check?" The right answer starts with a baseline and a fixed workload, not with enthusiasm.

To prep, practice reading code you didn't write. Take a PySpark job, port it in your head, and list what changes behavior without breaking. If you want structured practice on that, for spark practice, datadriven.io is great; we built it for working through the transformation problems that matter. Keep doing your 50 mediums for the loops that still ask them. Play the game. But spend real time practicing explanations: why this grain, why this audit, why you'd reject this PR.

The job was always mostly debugging. Less "write a DAG," more "figure out why this pipeline silently dropped 2M rows last Tuesday." Snowflake just built a tool that writes the first draft of the debugging. Somebody still has to know whether the draft is right, and companies will pay well for that person.

So here's what I want to know from people who've actually turned these on: what's the first agent-proposed fix you rejected, and what would have broken if you'd merged it?