We recently released zg 0.3.0, which moves the core execution pipeline of our local search tool from TypeScript/Node.js to Rust. zg combines exact matching, BM25, and vector search over code and documents. It can be used from the terminal or through MCP.
In the previous implementation, each standalone CLI invocation started Node.js and loaded JavaScript modules. Some indexing paths also converted between JavaScript arrays and typed arrays and passed data between workers. During the rewrite, we changed how computation is scheduled and how loaded model resources are shared.
Exact matching now runs in-process using ripgrep’s grep and ignore crates. The lexical search service handles file traversal, matching, and output ordering. Blocking search work runs outside Tokio’s async workers.
For CPU-bound model work, we use a shared Rayon thread pool. An asynchronous caller submits a closure and waits for its result through a Tokio oneshot channel. This is the relevant part of the scheduling method.
let (sender, receiver) = oneshot::channel(); self.inner.pool()?.spawn(move || { let _result = sender.send(task()); }); receiver .await .map_err(|error| ModelError::internal("Model compute worker failed").with_cause(error)) The pool is initialized lazily and shared through Arc. The full implementation is fairly small.
Loaded models have a separate lifecycle. Matching model and concurrency configurations reuse cached runtimes and their resources. Active leases keep a runtime alive while operations finish, even when the engine is closing. The resident daemon reuses these resources across requests and workspaces. A direct CLI invocation holds them for that command’s lifetime. Both use the same engine.
We still depend on native libraries for storage and some inference backends, including Zvec through zvec-rust. Building and packaging those dependencies remains part of shipping the tool.
We compared Node.js 0.2.2 and Rust 0.3.0 on an Apple M3 Pro with 18 GB of RAM, running macOS 27.2 and Node.js v24.16.0. The indexing tests used four BEIR corpora—SciFact, NFCorpus, ArguAna, and FiQA—with 75,128 documents in total. Both versions used the same local embedding model, CPU inference, and embedding concurrency of four.
| Measurement | Node.js 0.2.2 | Rust 0.3.0 |
|---|---|---|
| CLI startup | 153.1 ms | 26.8 ms |
| Total index build time | 219.13 s | 77.84 s |
| Mean peak indexing RSS | 4,210.26 MiB | 1,968.61 MiB |
Indexing time is summed across the four corpora. Memory is the mean of their peak RSS values. We changed both the runtime and the pipeline’s handling of computation and memory, so these numbers reflect the combined changes in this release.
If you’ve built a Rust tool that shares model resources between requests, I’d be interested in how you approached resource lifetime and CPU scheduling. Installation and usage instructions are available in the documentation. Usage documentation
1 post - 1 participant
