LLM observability has a data problem hiding inside a measurement problem. Teams collect prompts, completions, model names, latency, and trace identifiers, but token-usage fields are not always present—particularly when responses are streamed, proxied, or generated by self-hosted models. Without reliable usage data, it becomes harder to explain model costs, compare workloads, and identify prompts that consume tokens without improving answer quality.

One traditional solution is to export traces, run tokenization or evaluation in a separate service, and load the results back into the analytics database. That works, but it creates another pipeline to operate and another place where data can become stale.

ClickHouse Cloud's executable user-defined functions (UDFs) offer another approach: run custom code from SQL, close to the data being analyzed. In an October 5, 2026 announcement, ClickHouse made executable UDFs generally available across its AWS, Google Cloud, and Azure offerings. The release adds a Native runtime for compiled languages, broader outbound network access, infrastructure-as-code support, and query-level metrics for UDF execution.

The accompanying example project demonstrates two complementary workloads: a compiled Rust function that counts LLM tokens and a network-enabled Python function that asks an external model to grade a sample of completions.

The architectural shift is useful, but the details matter. Compiled Native UDFs and network-enabled Python UDFs are different execution paths. Teams should not assume that every runtime has network access or that moving code into a database makes its resource consumption free.

Why it matters

LLM observability depends on more than the usage numbers returned by a model provider. Some responses omit token counts, and provider-reported totals may not reveal how much of the input came from system instructions, retrieved documents, user prompts, or generated output.

That breakdown can be valuable. If a retrieval pipeline suddenly doubles the amount of context it supplies, token costs may increase even when the number of model calls remains unchanged. If a system prompt grows over time, the cost may accumulate across thousands of requests.

ClickHouse's example uses a synthetic dataset in which approximately 30% of spans lack usage information. This is a result from the company's demonstration dataset, not a universal measurement of all production LLM traffic. The example then counts tokens from the recorded text and compares its estimates with provider-reported usage where available.

The second problem is quality. Token counts explain part of the bill, but not whether a completion was useful. A separate model can grade a sample of responses, producing structured scores and verdicts that can be aggregated by application, model, or feature.

Putting these computations closer to the trace data allows engineers to connect usage, cost, and quality through familiar SQL queries. It also makes resource accounting more important: CPU-heavy tokenization and network-bound model evaluation have very different performance characteristics.

What ClickHouse announced

The general-availability release introduces several capabilities for executable UDFs in ClickHouse Cloud:

  • Native runtime: Upload compiled binaries for supported languages, including Rust, Go, C++, and JavaScript compiled with Bun.
  • Outbound network access: Network-enabled UDFs can call public endpoints, subject to the configured runtime and service capabilities.
  • Azure availability: Executable UDFs are available across ClickHouse Cloud on AWS, Google Cloud, and Azure.
  • Memory limits and deterministic functions: Configure per-process memory limits and mark functions deterministic when identical inputs are guaranteed to produce identical outputs.
  • Cloud API and Terraform: Manage UDFs through the Cloud API and supported Terraform resources, including per-service version pinning.
  • Execution metrics: ClickHouse 26.6 and later adds UDF-specific profile events and asynchronous metrics that help expose child-process costs alongside query telemetry.

The distinction between runtimes is important. The Native runtime is intended for compiled, self-contained code; the network-enabled example in ClickHouse's article uses Python. The announcement describes outbound network access as generally available, but it does not mean a Native binary can automatically make external API calls. Check the current runtime documentation before choosing an implementation.

This is an excerpt. The full article continues with:

  • Network-enabled UDFs: useful, but not free
  • Per-query metrics make UDF costs visible
  • What the token-counting benchmark tells us
  • What data and AI engineers should do
  • Limitations and open questions
  • Key takeaways

Read the full article on AI Data Legends → — with diagrams, code and sources.

Originally published on AI Data Legends. Each post here is an excerpt; the full articles live at aidatalegends.com.