Divyansh दिव्यांश

Replacing curl: a waiting agent in 1.8 KB


divyanshgolyan.com/posts/http2-client/

An agent spends most of its life waiting on the model. It sends a request, and the answer comes back a few tokens at a time, for seconds or sometimes minutes. Rui, my agent harness, runs many agents in one process, so at any moment most of them are waiting.

Ten agents over one minute. Each model call is a moment of sending and then a long stretch of answer arriving a few tokens at a time. At the marked moment, 9 of the 10 agents are waiting on the model.
Ten agents over a minute. At the marked moment, 9 of them are waiting on the model.

Rui used to make these requests with libcurl, and each waiting agent cost 8.0 KB of memory. I replaced curl with my own HTTP/2 client, and a waiting agent now costs 1.8 KB. This post explains where the difference comes from, starting from what a request is.

A request travels over a connection

To reach the model, a request needs a connection to the server. At the bottom is a TCP socket, which the kernel keeps. The bytes on it have to be private, so TLS wraps the socket. HTTP/2 runs on top of TLS.

TLS and HTTP/2 both keep state for each connection, about 55 KB together in Rui, before any request uses it.Most of the TLS state is the server’s certificates, kept after the handshake. Most of the HTTP/2 state is a fixed 16 KB buffer for outgoing frames.

Most of the TLS state is the server’s certificates, kept after the handshake. Most of the HTTP/2 state is a fixed 16 KB buffer for outgoing frames.
What one HTTP/2 connection holds in Rui, drawn to scale: an HTTP/2 session of 21.3 KB and TLS state of 33.9 KB, about 55 KB in all, on top of a TCP socket that the kernel keeps.
What one connection holds, drawn to scale.

If every waiting agent needed its own connection, 100 agents would spend 5.5 MB on connections.

Many requests share one connection

HTTP/1.1 carries one request at a time on a connection. HTTP/2 cuts every request and response into frames, and tags each frame with the id of the stream it belongs to. Frames from different streams then take turns on the same connection.

Frames from three streams, a, b and c, take turns on one connection from the model server to Rui. Each frame carries the id of its stream.
Frames from three streams take turns on one connection.

The server decides how many streams one connection may carry. In my benchmark it’s 100. My client puts a new stream on a connection that still has room, and opens a second connection only when the first is full. So 100 waiting agents share one connection, and each pays about 550 bytes of it. curl shares connections the same way, so this part costs both clients the same.

What a waiting request has to remember

With the connection shared, what’s left is what each request keeps for itself. It needs to know which connection and stream it’s on, where the answer goes and how much has arrived, and when it last heard from the server. Until the server replies, it also keeps its headers and where to find its body, in case it has to be sent again.

That fits in 96 bytes, kept in the agent’s own slot. nghttp2, the library that parses HTTP/2 frames for me, keeps another 128 bytes per stream.

The 96-byte record a waiting stream keeps in Rui, one row per field and one cell per 8 bytes: its connection and stream id, the answer’s file and bytes arrived, when it last heard from the server, its place in the timeout list, the request body kept in case it must be resent, and the headers until the reply, with its state.
The 96-byte stream record, one row per field and one cell per 8 bytes.

What curl keeps for the same request

curl is built for many jobs besides this one. It supports dozens of protocols, along with cookies, redirects and every kind of proxy, and that has a cost per request.

Each request in curl is an easy handle, about 2.9 KB, with room for the settings of every protocol curl supports. curl also keeps its own copy of every request header until the request ends. For a model request that’s about 1.4 KB, and most of it is the bearer token.

My client keeps one copy of the headers and hands them to nghttp2 without copying them again. It erases and frees that copy when the first response frame arrives. Until then the server might refuse the stream, and the request would have to be sent again. Once the server answers, it has the request, and the headers are no longer needed.

Moving the answer from the socket to the file

The answer’s bytes also have to get from the socket to the file, and that path needs memory too.

curl hands you bytes through a callback when it has them. You take them right then, or you pause the whole transfer. So Rui copied each chunk into a 256 KB ring shared by all agents, and a second thread wrote the ring to disk.

My client reads when it’s ready. It reads every socket into one 16 KB buffer that all connections share. nghttp2 decodes the frames where they sit, and each stream gets one write to its file for each read, however many events the read held.

With curl, Rui took bytes from a write callback, copied them into one 256 KB ring shared by all agents, and a second thread wrote them to each file. My client reads every socket into one 16 KB buffer, decodes frames in place and makes one write per stream, all on one thread.
The path from socket to file with curl, against my client.

Keeping the server from outrunning the disk

Reading only when ready raises a problem: the server could send faster than Rui writes to disk, and the bytes would pile up in memory.

HTTP/2 handles this with flow control. Each stream has a window: the number of bytes the server may send before it has to wait. My client opens a 16 KB window per stream, and it gives bytes back to the window only after it has written them to the file.

One stream’s 16 KB flow-control window. Bytes that have arrived but are not yet written to the file use up the window; when it is used up the server waits, and writing the bytes gives the window back.
One stream’s window. Bytes that have arrived but aren’t written yet use it up, and writing them gives it back.

So if the disk slows down, the server slows down with it, and at most 16 KB per stream waits in memory.

Timing out a stream when the server goes quiet

Someone also has to notice when the server goes quiet for too long. Every stream on my client has the same timeout, so the streams can sit in one list, ordered by when each last heard from the server. When bytes arrive for a stream, it moves to the end of the list. The stream at the front is the next to time out, so the client sleeps until that deadline without scanning the other streams. The list costs each stream 24 bytes of its record, with no timer of its own.Rui’s curl loop woke about 40 times a second to check its transfers, even when nothing had arrived.

Rui’s curl loop woke about 40 times a second to check its transfers, even when nothing had arrived.
Every stream shares one timeout, so the streams sit in one list ordered by how long each has been quiet. A stream that hears from the server moves to the tail, and the head is always the next to time out.
One list of streams. The front of the list is the next to time out.

Adding it up

Per waiting agent, my client uses 1.8 KB. About 550 bytes is the agent’s share of the connection, 260 bytes is a table of connections set up front, and 220 bytes is the stream record and nghttp2’s stream. The remaining 800 bytes is allocator overhead: the memory I count is whole pages the process has written, and the allocator keeps some room in them.

curl uses 8.0 KB. The easy handle and the header copies are 4.3 KB of it, and its stream and TLS state another 1.0 KB. I haven’t broken down the remaining 2.7 KB.

Private memory per waiting agent, to scale. My client: 1.8 KB, of which 0.55 KB is its share of the connection, 0.26 KB a connection table, 0.22 KB the stream record and nghttp2’s stream, and 0.8 KB allocator overhead. curl: 8.0 KB, of which 2.9 KB is the easy handle, 1.4 KB header copies, 1.0 KB other stream and TLS state, and 2.7 KB not broken down. For 1,000 waiting agents, each total adds about 1.2 MB of fixed cost: 3.1 MB against 9.2 MB.
Private memory per waiting agent, to scale, and for 1,000 waiting agents.

Both clients also pay about 1.2 MB that doesn’t grow with the number of agents. So 1,000 waiting agents come to 3.1 MB with my client and 9.2 MB with curl.My client also uses less CPU: 8.5 µs per streamed event against curl’s 20.1, and 0.13 ms a second for 1,000 idle streams against curl’s 2.7.

My client also uses less CPU: 8.5 µs per streamed event against curl’s 20.1, and 0.13 ms a second for 1,000 idle streams against curl’s 2.7.

My client speaks only HTTP/2 over TLS. It has no HTTP/1.1, redirects, cookies, server push or TLS session resumption, and it supports one kind of proxy. It’s about 6,200 lines of Zig with its tests, and I now maintain it myself instead of relying on curl’s maintainers.

If your program uses a general library for one narrow job, measure what the library keeps per request. Anyone can measure it with heaptrack, and the number tells you whether a narrow version is worth writing.

How I measured

One benchmark binary drove both clients against a Go server that plays the model’s API over TLS and HTTP/2 on the same machine, with 100 streams per connection and events of about 300 bytes. curl is libcurl 8.22, driven through one multi handle with HTTP/2 multiplexing and answers written to files, as in Rui. Both ran on a 4-core Linux machine in the cloud, as ReleaseSafe builds, on 25 October 2026. Memory is private memory (Private_Dirty) once every agent is waiting, with agents started 5 ms apart, the median of 3 runs. The per-agent number is the growth from 100 to 1,000 waiting agents, divided by 900. The parts come from heaptrack. curl’s 1.0 KB of stream and TLS state comes from a run where every agent started at once.