Divyansh दिव्यांश

Infinity vs Rui: 1 GB vs 4.7 MB


divyanshgolyan.com/posts/infinity-vs-rui/

I believe most agents will soon be started by other agents, and they will spend most of their time waiting. Last week AWS released Infinity, a runtime built on the same belief. Their pitch: 75,000 agents should fit in the memory of a Raspberry Pi.

I like to look at claims like this in case I can improve Rui, my agent harness. So I took Infinity’s own benchmark and ran it on Rui.

Where a waiting agent lives

Simplified, a waiting agent is its conversation history and a place in line. Infinity drops everything else between turns and keeps the history in memory. Rui moves the history to disk as well, into SQLite.

Where a waiting agent's history lives, at 10,000 idle agents. Infinity holds every agent's history in memory, 1,075 MB. Rui keeps it in SQLite on disk, 1.3 GB, and uses 4.7 MB of memory. The two memory boxes are to scale; the disk is not.
10,000 idle agents. The two memory boxes are drawn to scale; the disk is not.

1 GB vs 4.7 MB

The benchmark gives every agent 20 turns. Each turn is a user message, a 1.2 KB answer that calls a tool, the tool’s result, and a second answer. Then the agent goes idle. I ran the same turns through Rui on the same machine and measured both with the same metric.

At 10,000 idle agents, Infinity holds 1,075 MB and Rui holds 4.7 MB. Infinity adds 101 KB for every agent, because each agent’s history stays in memory. Rui’s line stays flat, because an idle agent is just rows on disk.

Private memory as idle agents grow from 0 to 10,000, each after 20 turns. After a first step of 110 MB at 500 agents, Infinity rises in a straight line, about 101 KB per agent, to 1,075 MB. Rui stays flat at 4.7 MB; it was measured to 9,500 agents and a dashed end carries its flat line to 10,000.
Past its first 500 agents, Infinity climbs by 101 KB with every idle agent. Rui stays flat.

Extend both lines to Infinity’s own target of 75,000 agents. Infinity needs about 7.6 GB, which matches the 8 GB Raspberry Pi in their pitch. Rui still needs about 4.7 MB of memory, plus about 9.5 GB on disk.

The history still exists in Rui. It takes 1.3 GB on disk, which is far cheaper than memory, and it survives a crash. If Infinity’s local process dies, its 10,000 conversations go with it. (Infinity’s serverless mode keeps history in a database instead.)

What Rui pays for it

Disk is slower than memory, and Rui pays for that (partly) in CPU. A model call costs Rui 12.1 ms of CPU and Infinity 0.13 ms, about 93 times less.

CPU time per model call with 20 turns of history: Infinity 0.13 ms, Rui 12.1 ms. Of Rui's, 40% (4.8 ms) goes to rebuilding the request from SQLite, 30% (3.6 ms) to other SQLite work, 7% (0.8 ms) to fsync and 23% (2.8 ms) to the rest.
CPU time per model call, with 20 turns of history.

Most of that gap is one choice I made. Before every model call, Rui rebuilds the whole request from SQLite, so the cost grows with every turn. Keeping the request built between calls removes the biggest piece without giving up crash safety, and it’s what I’m working on next.

Two routes from the same belief

Infinity and I agree that a waiting agent should cost almost nothing. Infinity gets there by keeping only the history. Rui gets there by putting the history on disk. For a swarm of mostly idle agents on one machine, that’s 1 GB against 4.7 MB.

How I measured

Both ran on one 4-core Linux machine in the cloud, from each project’s main branch on 28 October 2026. Infinity ran its own benchmark, agent_scale.rs, with its model and tool inside the same process. Rui ran the same turns against a separate model server over TLS and HTTP/2, and ran each tool call as a real bash command. Every model call completed in both, 380,000 of them for Rui with no errors. I count private memory, the memory only that process uses: the median of 3 runs for Infinity, and one run for Rui, which matched its three shorter runs to within 2%. I measured Rui up to 9,500 agents and extended its flat line to 10,000. CPU is the process’s own CPU time divided by its model calls.