We're an independent lab for local AI. Intelligence you rely on should run on hardware you own, stay open to inspection, and be measured honestly. We build the pieces that make that possible — an inference engine, a state format that moves its work exactly, and, next, the machinery that proves an AI system did what it claims.
The request has timed out. An agent is refunding forty euros to a customer. It sent the instruction to the payment provider and received no answer. Now it must choose: send the instruction again, or stop.
One observation. Several worlds. Opposite right answers. A useful eval should expose the conditions its theory of success leaves out—and leave those discoveries useful to the next investigator.
Read moreWe work from the job that needs doing to the hardware that does it: the engine, the state it keeps, the kernels underneath, and the evidence that says whether any of it worked. Say what you measured. Name the machine. Show the receipt.
Measured, or it doesn't ship.
Four open repositories: the engine, the cache format it replays, the dashboard that watches it, and the book that explains how it all works.
We built Muser to show what local inference can be: Muse Glimmer, a 52-layer ~30B model, running well on one Mac — with the option of handing long prompts to a GPU node for prefill. On our exact-token benchmarks it matches or beats a source-pinned llama.cpp at every depth we tested, with or without speculation.
On natural text, speculation helps on some content and not on others — the benchmarks publish both sides.
We built kvpack because a computed KV cache is work worth keeping. It saves inference state and restores exactly the same bytes — after a restart, a crash, or on another machine — and it refuses state that doesn't match the model, tokenizer, and configuration it came from. The refusal is the point: a wrong restore should never be silent.
Cross-engine reuse works when two adapters deliberately share a representation — kvpack doesn't invent one.
The dashboard we wanted while running Muser: fleet health, cache savings, and live sessions, with one rule everywhere — missing data stays visibly missing. No zero-filling, no demo numbers, no gaps quietly connected on a chart.
One Rust binary and one checked-in HTML page — no CDN, no package tree, no frontend build.
Everything we learned building these systems, written down: how one token is generated on Apple Silicon, how the speculative lanes work, how kvpack replays state exactly, and how a GPU node hands prefill to a Mac. Forty chapters, with every measured number cited to a receipt.
vLLM prefills long context in NVFP4 on a GB10 node; Muser decodes on Apple Metal. At 130,815 tokens that is 137.4 s to first token instead of 570.1 s — 4.15×, five counted reps, deterministic output in every one.
vLLM processes long context in NVFP4 on a DGX Spark or another GB10-class machine.
kvpack · TLS + HMAC
mismatch → reject
The Mac verifies identity and integrity, installs the KV state, then Muser continues generation with Metal.
One request, two architectures. The Mac checks identity and integrity before installing anything; a mismatch is rejected, and the rejection keeps its receipt. The demonstrated topology is one GB10-class producer and one Mac consumer — scale-out is what we're working on next. We also tried speculative decoding across the wire: it didn't pay for itself, so we published the negative result and moved on.
One of us lives in Zurich, the other in Buenos Aires; Patagonia is the shared love. The lab is the two of us, building these systems end to end.
Systems researcher and author working across local AI, distributed systems, and developer education. A former RabbitMQ core developer who has also worked at Apple, he co-authored RabbitMQ in Action and now builds reproducible inference systems for Apple Silicon.
Zurich, SwitzerlandAI researcher working across language models, deep learning, and reinforcement learning. His public work spans NLP and sequence tagging, PyTorch experiments, question-answering systems, generative AI, and deep-research agents, connecting research ideas to practical code.
Buenos Aires, ArgentinaThe next pieces of work, and where each one stands today.
today's qualified topology is one Mac and one GB10 node
nodes can join in one command; leaving cleanly is next
unqualified — requests with images fall back to local prefill
how we decide whether an agent system worked — in private development
All of it is open — clone the repos, run the benchmarks, and see for yourself.
