Cactus Compute argues that tool calling does not need a big brain. Its open Needle 2 model maps commands onto function signatures with just 45M parameters, a sliver of today’s frontier LLMs, because the task requires no world knowledge and no open-ended prose.
Everything fits in one 14MB file, and a whole session runs in about 28MB of RAM. The weights stay at 2-bit CQ2 quantization, locked inside a C++ engine, which means no runtime install and no inference-time download.
Speed claims: 500 tokens per second on a Raspberry Pi 5, 300-700 tokens per second on phones under $200, and 400-1,500 on Meta Quest 3S and Apple Vision Pro.
The engine compiles JSON schemas into a byte-level grammar, so the matcher knows which tokens are legal before logits exist and skips up to 98% of the vocabulary. The arithmetic path stays int8; weights never decompress into RAM.
Pebble already runs Needle locally in its Index 01 app for offline voice actions. The deployment target is the low end of the hardware stack, where accelerators simply do not exist.
