AWS open-sources Infinity, a runtime running 80,000 agents on one core

Key Points

1. AWS released Infinity under Apache 2.0, a runtime for running large numbers of concurrent LLM agents.
2. An idle agent costs about 100 kilobytes, so roughly 75,000 fit inside 8 gigabytes of memory.
3. Agent fleets may no longer require heavy hardware, reshaping cost math for companies building them.

The latest:

An idle AI agent now costs about 100 kilobytes of memory under Infinity, a runtime AWS released on 23 October 2026 under the Apache 2.0 license. The post, written by Shadaj Laddad, research lead on the Hydro team at AWS, describes the project as an early research release shared to collect feedback. Hydro is a collaboration between AWS and the University of California, Berkeley.

Details:

• The memory claim: According to the AWS open source blog, an idle agent consumes roughly 100 kilobytes, most of it conversation history. That puts about 75,000 agents inside 8 gigabytes — the memory of a Raspberry Pi 5 — the comparison AWS itself chose to frame the scale of the result.
• The benchmark: AWS said it ran 80,000 agents of 20 turns each, totaling 3.2 million completions, on a single thread of an AMD EPYC 9R14 processor using in-memory stores and a simulated model. Each completion ran about 1.2 kilobytes, and memory grew linearly with agent count. The post did not state how long the test took.
• How it works: Infinity is built on stackless coroutines, the technique languages such as Rust use to suspend and resume asynchronous functions. When an agent calls a tool, the runtime dispatches the call, saves the conversation and suspends the agent, resuming it only once the tool result arrives.
• The slice model: Each agent runs in bounded slices. A slice loads the agent history from a swappable store, executes one model completion, persists the output and issues at most one tool call. Resume events, including user text and tool results, arrive through a shared input queue and are processed in order per thread.
• Injected events: Asynchronous events reach models as synthetic tool calls named receive_event__injected, preserving the one-result-per-tool-call structure model APIs expect. The __injected suffix marks runtime-generated calls, and the model cannot invoke that name as a tool itself.
• Agent trees: Agents can create sub-threads through spawn_thread. Sub-threads return results via report_to_parent and terminate with close_thread. Events can be handled in a separate sub-thread, the default behavior, or appended directly to the parent thread history in what AWS calls associative mode.
• Serverless deployment: Infinity supports a step mode in which the platform calls step once per message batch and nothing persists in memory between calls. The AWS reference deployment uses SQS FIFO queues, Lambda, Aurora DSQL for histories, DynamoDB for deduplication and EventBridge Scheduler for timers, with CDK constructs in the repository.
• Unresolved questions: The authors listed open problems: how interleaved events affect agent decisions, scheduling and priority policies, compaction for large long-lived agent trees, and proving event authenticity while remaining stateless. Code is published at the hydro-project/infinity repository, with documentation and a quickstart at infinity.hydro.run.

Between the lines:

The economics here follow from the design, not from hardware. Because an agent is suspended rather than kept running while it waits on a tool, its cost collapses to the size of its stored history — the 100 kilobytes AWS reports. The step mode, which leaves nothing in memory between calls, points the same way: idle concurrency becomes a storage problem rather than a compute bill.

What’s next

Watch the hydro-project/infinity repository for the feedback AWS said it is seeking, and for answers on scheduling policy and compaction for long-lived agent trees — the two open problems most likely to decide whether the runtime moves past research status.