Most “AI companies” are a prompt, a key to someone else's API and a nice deck. We are built differently. In five and a half months we built a complete stack of our own: an inference engine that competes head-to-head with llama.cpp and often beats it; an agent runtime with persistent memory, a planner and task preemption; a measurement harness that decides every night whether the agent got better; and a training loop in which the agent changes its own weights only after statistical proof. All of it in Rust, all of it in production 24/7 on ordinary consumer GPUs. What follows are not promises but numbers from our benchmarks, with pointers to where they live.
We are not users of someone else's AI. We build every layer ourselves — from the kernel that writes to the GPU to the agent that retrains its own weights at night and proves in the morning that it got better.Lines of Rust: agi-core, llamars, agi-web-ui, uide, codex-substrate — with ~1,700 tests.
Faster than llama.cpp: GPU prefill on Qwen3.5-35B, RX 6900 XT (Vulkan), at every context depth.
KV-cache compression (MLA): 30 KiB per token instead of ~270. Huge context on modest hardware.
Instead of 106 s — the reply to a message that interrupts a running task: checkpoint and KV-prefix reuse.
Our own engine,not a wrapper around llama.cpp.
llamars — pure Rust, ~150K lines, Vulkan and Metal without a single line of C.
The path was honest: first bindings to llama.cpp, then a runtime on top of them — and a ceiling. You cannot teach someone else's engine to snapshot a conversation to disk, share a GPU between two generation lanes, or tell the orchestrator its true context capacity. So llamars was born — our own inference engine for GGUF models.
Inside are things that usually only framework authors get to build. On Metal the whole model graph lives on the GPU: one command buffer per token, fused norm+matvec, packed QKV, flash-decoding with KV append, f16 activations end-to-end (ggml still keeps f32). On Vulkan — chained GEMM for prefill, int8 decode, cooperative-matrix kernels in f16/fp8, flash-decode with GQA sharing, q4_0 KV arenas. And an MLA cache for the DeepSeek family: 30 KiB per token instead of 270.
| Model · hardware | llamars | llama.cpp | Δ |
|---|---|---|---|
| Qwen3-8B Q4_K_M · M1 Max, Metal · pp512 / tg128 | 524 / 47.8 | 473 / 45.0 | +10.8% / +6.3% |
| TinyLlama 1.1B · M1 Max, Metal · pp512 / tg128 | 3626 / 239 | 3217 / 232 | +12.7% / +3.2% |
| Qwen3.5-35B-A3B · RX 6900 XT, Vulkan · GPU pp512 | 3135 | 1762 | +78% |
| Qwen2.5-Omni-3B · RX 6900 XT · audio encode, 2.5 s | 150 ms | 364 ms | ×2.4 |
| Qwen3-Embedding-0.6B · embed, 11 tokens | 200 ms | 540–768 ms | ×2.7 |
The engine is multimodal: a 32-layer Whisper or ViT encoder runs in a single command buffer, CosyVoice3 TTS (language model, flow matching and vocoder) is rewritten in Rust end to end, SDXL/Flux/Wan diffusion produces bit-identical images across restarts. And — rare for an inference engine — native LoRA training directly on quantized weights: a 27B model, 4096 context, 52 seconds per step, on one consumer card.
We don't hide where we started. The first version of the engine earned an internal document titled *Brutal Reality Check*: a beautiful backend architecture, zero tokens per second, 44 of 201 tensors loading. It was rewritten bottom-up — from one working matrix multiply to a full GPU graph — and since then every kernel ships only together with a benchmark against llama.cpp on the same hardware.
Memorythat survives a restart.
UIDE — our own store: RocksDB, a home-grown HNSW, a signed operation log.
An agent without memory is a chatbot. We split memory into its own engine, UIDE: persistent KV on RocksDB, embeddings, a compact dependency-free HNSW index (recall@10 ≥ 0.9 against brute force), index snapshots with delta reconciliation instead of rebuilds, and an Ed25519-signed operation log — groundwork for federating memory between instances. Inference is injected into UIDE through a trait: the store knows nothing about where vectors come from.
Goal Graph
A persistent DAG of tasks on top of memory: node digests, per-step token budgets, and a lesson from a failure available on the very next step of the same run.
Three tiers for KV
VRAM → RAM → NVMe with a measured restore cost, no constants in code. An 83,000-token conversation with an image: a turn in 10.5 s instead of ≈56 s of full re-prefill.
Recall under control
Memory benchmark: recall@5 = 0.963, MRR = 0.935, median 966 ms. Class balancing so rare memories don't drown.
A runtimewhere everything is a step.
AGI Core: ~170K lines, ~1,200 tests, 1,227 commits in 5.5 months.
AGI Core is the irreducible kernel of an autonomous agent: a flow engine, a tool registry, memory, an event bus, an inference router and transport. Everything else is plugins and JSON process definitions that can change at runtime. The router holds four backends at once — local GGUF via llamars, OpenAI-compatible APIs, Anthropic and a remote agi-core — and picks per task.
A few decisions you won't find in popular agent frameworks:
- AttentionQueue. A user message preempts a running task at a safe point: checkpoint, KV-prefix reuse, reply — and back to the task. Measured: 12 seconds instead of 106.
- A unified step language. Every step is a graph node with an effect type (pure / world / judgment). Pure regions fuse, world steps are journaled and replayable, webhooks are graph nodes too — no per-channel code.
- Client I/O. The brain on the server, the hands with the user: files, shell and terminal run in the user's own workspace via reverse RPC (Zed through ACP, the browser, a daemon).
- App Modules. The agent assembles mini-apps where the UI is data, not code; they run on web and mobile clients without a rebuild.
- Plugin ABI. Native
.so/.dylibwith C symbols, hot-loaded — a tool can be written in any language.
We measure savings too. One agent-loop benchmark: before optimisation — 5 tasks of 6 and 217,904 tokens; after — 6 of 6 and 88,548 tokens, −59%. That is not “the model got smarter”; the context compiler stopped carrying junk into the prompt.
An agent thatlearns at night — and has to prove it.
agi-bench: 37 tasks, 7 categories, Wilson confidence intervals, every night.
Self-improvement without measurement is self-deception. So the benchmark lives apart from the agent, in a directory the agent's process cannot read; tasks are parameterised by seed so they cannot be memorised; part of the set is held out. Every night on a timer: a run, a comparison with the baseline, a bootstrap of the difference.
00:30 UTC — build a corpus from the agent's “flight recorder” for the day: ~6.6M tokens per two days of work.
Stop the kernel, free the GPU.
Train a LoRA on our own quantisation: a 27B model, ~120 steps, 24 seconds per step.
Run the benchmark with the new adapter (B), then with the old one (A) — 30 tasks × 3 trials.
Compare: Wilson interval, bootstrap of the difference. Verdict: promote / hold / reject.
Bring the kernel up with the winner. Change nothing if there is no proof.
The first full night ended in reject — and it turned out the benchmark had measured memory, not quality: merging the adapter at load time pinned 14 GiB. We fixed the deployment format; the next night gave hold: B 89/90 vs A 88/90 — a difference within noise. No proof — no change of weights. We consider that the most important property of the system, not a weakness.
Codethat can be checked formally.
codex-substrate: a content-addressed graph of programs with a proof kernel.
The next layer is already a prototype: codex-substrate — a code graph where every function is normalised, hashed with BLAKE3 and signed with Ed25519, with Hindley–Milner type inference, a dependently-typed proof kernel, Lean 4 import and compilation to WebAssembly. And inside AGI Core lives a verified decompiler: a tower of modules grown from a single nand atom recovered the whole of libm — 38 of 38 functions to 10⁻¹² on 20,000 probes each, with a binary 31% smaller than the original. This is what will let an agent change its own code by proving correctness rather than hoping for it.
What of thisyour business needs.
We don't advise from slides: every recommendation has been tested on our own hardware in the last six months.
Everything above we built for ourselves — which is exactly why we know where AI projects lose money. Usually in three places: overpaying for third-party APIs where a 35B model on one card would do; agents without memory and without measurement that “sort of work”; and data leaving for someone else's cloud because “there is no other way”. There is.
Audit and architecture
We take apart your AI loop: models, cost per token, latency, memory, tool security. You get a plan with numbers — what to keep on APIs, what to bring in-house, what to rewrite.
Inference on your side
We deploy models on-premise on consumer GPUs or in your cloud: quantisation, a KV cache sized for your context, batching. Measured against your current API — in tokens per second and in money.
Agents and training
We design agents with memory, a planner and a quality gate; we set up a nightly training loop on your data. Or we hand you ready roles — lawyer, accountant, recruiter, assistant, calls, chats.
Formats range from a one-day audit to a turnkey deployment with our runtime inside your perimeter. Write to [email protected] or use the contact form: describe the task and what you already run — we come back with a plan and an estimate. On why agents need a sandbox — see the separate article.
A consultant is worth the rakes he has already stepped on that you haven't bought yet. Behind us are our own engine, our own runtime and our own benchmark — and a folder of post-mortems for each.Products in this article
AI consulting
Stack audit, inference on your side, agents with memory and nightly training — from the people who wrote all of it themselves.
Why security agents need a sandboxAn agent that can 'just run tools' on a security review is a second attacker — unless every call has a grant. AGI Core isolates tool execution; Forge keeps the skill list in UIDE.
Turnkey smart homes on our own OS: 0xd-os and 0xd-uiWe build turnkey smart homes on components we designed ourselves. At the core is 0xd-os — one operating system that runs identically on a flagship tablet and on a device with 512 KB of memory.
Whose Computer Is This? Yours, or…We wrote an OS from scratch in Rust and booted it on a Samsung Galaxy Tab S6. Inside the device live at least eight executing entities — seven of them not ours, and almost every one can kill us. The opening map of a series on who really owns a modern SoC.