Your AI Agent Pipeline Wastes Tokens on the Wrong Tiles

The viral Maya pyramid story sounds like a win for AI agents. A team pointed dozens of agents at NASA LiDAR data and found a structure in days. The unglamorous part is why it worked at all: they did not let agents read the whole forest. They triaged tiles first, then spent expensive model calls on candidates.

By the end of this article you'll be able to structure a multi-agent scan over a large raster dataset without burning your token budget on empty ground.

Naive agents scan everything

The obvious approach: tile the LiDAR hillshade into images, ask a vision model "is there a structure here?", repeat for every tile. For a survey area of a few hundred square kilometers at 1-meter resolution, that's millions of tiles.

At even a modest per-call cost, that multiplies into real money and days of wall time. Worse, most tiles contain nothing but trees. Terrain-derived products like hillshade or digital surface model (DSM) difference rasters make features visible, but they don't reduce how many tiles you inspect.

The fix is a funnel. Cheap filters eliminate the boring 99% before any agent sees a pixel.

Build a funnel, not a swarm

A practical pipeline has three stages:

1. Compute terrain derivatives (slope, hillshade, openneas) 2. Score every tile with a cheap local filter 3. Send only high-scoring tiles to vision agents

Stage 2 is where most articles go vague. You don't need a model for it. Simple raster statistics catch most archaeological candidates: unusually flat platforms on slopes, regular geometric edges, local elevation anomalies.

Score tiles with raster statistics

Here's a working scorer using rasterio and numpy. It flags tiles that are flat relative to their surroundings, a signature of platforms and plazas:

import numpy as np import rasterio from rasterio.windows import Window def tile_score(path, row, col, size=512): """Score one tile: high means flat plateau inside rough terrain.""" with rasterio.open(path) as src: win = Window(col, row, size, size) elev = src.read(1, window=win).astype("float32") if np.isnan(elev).all(): return 0.0 grad = np.abs(np.gradient(elev)) interior = grad[size // 4 : -size // 4, size // 4 : -size // 4] border = grad.mean() - interior.mean() # Flat interior + steep surroundings = candidate platform return float(np.clip(interior.mean() * -50 + border, 0, None))

This runs in milliseconds per tile on a laptop. Sort all tiles by score, keep the top percentile, and only now invoke the expensive step.

Point agents only at candidates

With a candidate list, the agent stage becomes affordable. Each agent gets one tile image plus a prompt asking for structured output, not prose:

import json PROMPT = """Classify this hillshade tile. Respond with JSON only: {"structure": "platform"|"terrace"|"none", "confidence": 0.0-1.0, "reason": "one sentence"} """ def classify(tile_png: bytes, client) -> dict: resp = client.messages.create( model="claude-sonnet-4-5", max_tokens=200, messages=[{ "role": "user", "content": [ {"type": "image", "source": { "type": "base64", "media_type": "image/png", "data": tile_png.decode(), }}, {"type": "text", "text": PROMPT}, ], }], ) return json.loads(resp.content[0].text)

Structured JSON makes the next stage mechanical: rank by confidence, cluster nearby hits, and route the top clusters to human review. Many agents in parallel help here, but the parallelism is over a shortlist, not the whole raster.

Where multi-agent setups fail

Three failure modes show up in every large-agent run I've reviewed:

Correlated false positives. Agents agree with each other because they share the same model and prompt. One confident wrong call about a logging terrace gets echoed by ten agents. Break the correlation by varying prompts or using two different models on the shortlist.
Prompt drift across long runs. Agents editing shared state or re-prompting each other accumulate context. Freeze the prompt per stage and version it like code.
Edge-tile artifacts. Features straddle tile boundaries and get split into two unconvincing halves. Score and classify overlapping tiles, then merge by coordinate before trusting any "none" verdict.

That last one matters most. A pyramid straddling four tiles looks like nothing on any one of them. Overlap by 15-20% and dedupe hits.

Validate before you celebrate

A confident agent verdict is a lead, not a discovery. Before claiming anything:

Check the raw DEM (digital elevation model), not just the hillshade, for a consistent closed form.
Compare against existing surveys and satellite imagery for modern explanations: roads, quarries, logging.
Get eyes on the ground or a field archaeologist's review.

The source story earned its headline because humans verified the candidate. Keep that step.

Key takeaways

Never send a full raster to vision agents; score tiles with cheap raster statistics first and spend model calls on the top percentile.
Overlap tiles by 15-20% so features on boundaries survive classification, then merge hits by coordinate.
Run multiple agents on candidates only, and vary prompts or models so their errors aren't correlated.
Treat every agent hit as a lead for human verification, never as a finding.