Rui / रुई: a hundred agents in 7.4 MB

I believe most agents will soon be started by other agents. A few years (okay weeks) ago, OpenAI pointed about 10,000 agents at the Navier–Stokes Millennium Prize problem and succeeded. I expect runs like that to become routine, with models trained to work alongside hundreds, if not thousands, of other agents.

That makes memory a scarce resource on a machine running agents. Most of the time an agent is just waiting for the model, so what matters is what a waiting agent costs. Today that cost is high: 100 waiting agents take 276 MB in pi and 210 MB in Codex app-server, and Jarred Sumner’s Bun AOT build of Claude Code takes 267 MB with 67 active subagents. At the rate pi and Codex app-server add per agent, 10,000 would need about 12 GB and 18 GB.

In Rui, my agent harness, the same 100 waiting agents cost 7.4 MB, and 10,000 would need about 200 MB.

A swarm pays for memory in three ways

In my testing, about 85% of each turn went to waiting on the model.

So an agent mostly holds a conversation open while the model thinks. A swarm then costs the harness’s starting memory, plus what each waiting agent adds times the number of agents.

The first is keeping each agent’s state in memory while it waits. Harnesses that run every agent in one process, like Codex app-server and pi, add about 1.8 MB and 1.2 MB for each one.

The second is giving every agent its own process. Each process brings its own copy of everything: a heap, threads, a connection to the model. fx does this, at about 4.8 MB per agent. Many harnesses do, because they were built for one person driving one agent in a terminal.

The third is loading a heavy runtime before any agent starts. OpenCode’s server takes 300 MB before its first agent.

Rui avoids all three. Here is how the five harnesses compare as agents are added.

One process runs every agent’s loop

Simplified, an agent is a loop which sends user inputs to an API and gets back outputs, which it shows the user or acts on in the environment. The loop goes on until something interrupts it: the environment, the user, or the model itself. This loop needs little memory. What it needs is the network stack, the storage stack, and small bits of information for coordination.

This is the same loop across every agent, so you don’t need a separate process for each one. One process can run the loop for all of them: it stores the state and runs the network stack.

In Rui that process is the Host. Rui is a compiled Zig program, so there is no runtime to load first, and the Host starts at 5.8 MB. It loads its network stack and SQLite once. Every agent’s request goes over one shared connection, and one event loop watches all of them, so no agent needs its own thread.

A waiting agent’s request is one stream on the shared connection, and everything else about it is rows in SQLite, so memory doesn’t grow with conversation length.

So a turn should cost almost nothing. Follow one agent, swarm/42, through one.

One turn costs a few KB

• Send. You send swarm/42 a message. Rui saves it to SQLite first, so it survives a crash.
• Admit. If one of the 100 slots is free, swarm/42 takes it. If not, it stays a row and waits.
• Request. Rui builds the request from the saved conversation and sends it over the shared connection. This is the only step that costs memory.
• Wait. The model thinks. This is most of the turn, and nothing runs for swarm/42 while it waits.
• Answer. The answer streams in through a small fixed buffer and goes straight to SQLite.
• Bash. If the model asks to run a command, Rui starts it as a separate process. That memory belongs to the command and is freed when it exits.
• Release. The result is saved and the slot is freed. swarm/42 is just rows again until its next message.

The few KB the Request step adds are mostly curl, the library Rui uses to talk HTTP. Everything else was set aside once, when the Host started.

What one process costs you

Running every agent in a single process has a cost too: one shared limit, and one point of failure.

The limit is the number of slots. Say you start Rui with the default 100 slots. A slot holds one piece of running work, either a model call or a Bash command, so up to 100 agents can be working at once. Agent 101 waits as a row until a slot frees up. It has no request built and costs no memory while it waits. (Past the point where the provider rate-limits you, more slots only buy more retries.)

If the Host crashes, every agent stops with it. No saved work is lost, because the Host saves every message and result in SQLite before it acts on them. Start the Host again and the work is still there.

Start a swarm

Rui runs on Apple Silicon Macs and x86-64 Linux, and works with OpenAI’s Codex models today, so you need a ChatGPT account. Install Rui, sign in and pick a model:

curl -fsSL https://github.com/DivyanshGolyan/rui/releases/latest/download/install.sh | sh
export PATH="$HOME/.local/bin:$PATH"

rui auth login codex
rui defaults configure --model gpt-6.1-sol

Then run rui in your project and ask for a swarm:

Split this repo into areas, start one agent per area to look for bugs, and give me one list of what they find.

Those agents will spend most of their time waiting. In Rui, a hundred of them wait in 7.4 MB.

How I measured

Everything ran on one MacBook Pro (M1 Pro, 16 GB). Rui measured between 6.4 and 7.4 MB at 100 agents across two sessions. Each harness talked to a fake model that accepts every request and never answers, so every agent stays waiting on the model with its request open. Rui talked to it over TLS and HTTP/2, the others over plain HTTP, which costs Rui a little extra. Rui’s requests carried one tool, Bash, and the others carried 4 to 15, so part of their memory pays for features Rui doesn’t have. The others were Codex app-server 0.164.0, pi 1.4.0, fx 0.0.14 and OpenCode 1.19.4. I count the physical footprint, the memory macOS charges to each process, summed over every process a harness starts. I don’t use RSS, because it counts shared program code once per process and inflates any harness that runs a process per agent. For 10,000 agents, Rui needs 10,000 slots: an idle Host started with 10,000 slots takes about 100 MB, or about 10 KB per slot, before each agent adds its 7.5 KB. The turn timings come from 15 live sessions on gpt-6-luna, run one at a time, each a small task that calls Bash.