Researchers at UC Berkeley and UT Austin have open-sourced a serving engine called FreeToken that lets developers run frontier-scale models on machines they already own. The idea is to make a personal workstation a realistic host for inference workloads that normally assume a rented cluster.
The engine adapts rather than assuming a fixed setup: it reshuffles computation and model state on the fly to fit whatever CPU, memory, and interconnect bandwidth the machine has in that moment. In the team’s testing, the 753B GLM-5.2 ran on a single workstation card, a 284B model on a gaming desktop, and a 35B model stayed interactive on an 8GB laptop GPU. The release is permissively licensed and easy to try: the code lives on GitHub, installs through Python’s pip under the name freetoken v0.1.2, and bundles a one-click desktop app for Windows and Linux.
The economics are the point. Releases like Kimi-K3, GLM-5.2, and DeepSeek-V4-Flash keep narrowing the gap with closed systems, yet serving costs still fall hardest on individual developers and small shops, especially as agentic workloads multiply the inference calls. The researchers point out that over 100 million consumer machines already carry discrete GPUs – idle capacity that developers pay to rent by the hour instead of using.
Whether a workstation can sustain long agent loops at that scale in practice is still unproven, but FreeToken at least lets small teams sample the frontier without a data center budget.
