Ant Farm - A concurrent M:N jobs distributor

[Nathan Riddick]

On Monday, 24 August 2026 at 23:04:33 UTC, Nathan Riddick wrote:
> Hello, I had an idea for a non-FIFO concurrent callback-jobs 
> thing based on a ring buffer. It seems to have strong 
> performance and utility for my next set of tinkerings. I came 
> up with ~90% of the spec and made some attempts at a clean 
> implementation, but Kimi K3 ultimately came through with an 
> initial clean version and various other LLMs were very useful 
> at grinding out late-night debug sessions with infinite 
> patience.
>
> tl;dr synthetic throughput sits between moodycamel no-tokens 
> and with-tokens at a few million items/sec, seems to prefer 
> more concurrency rather than less, the latency at reaching 
> one-more item after a big block of work crushes FIFO designs 
> generally, it's adaptive to a homogenous thread pool that 
> shrinks or grows over time, API is @nogc nothrow @system and 
> probably thread-safe.
>
> https://github.com/nriddick/antfarm

More work done, and I think it's in a pretty good spot. The core 
features and parallel-for/actors/fork-join are betterC and work 
without DRuntime, and a C ABI includes those with or without the 
worker pool host (leaving out the DRuntime Fibers).

Ryzen 9 5900XT (16 cores, two L3 groups):
One Farm: 2.49 billion one-word payloads singly delivered per 
second at 24 threads, above two billion from 8 threads up. 
Private child lifecycles, which are spawn, dispatch, retire and 
refill, peak at 517 million per second at 16 threads.
One Farm per L3 group: 4.09 billion payloads per second at 32 
threads, and 886 million child lifecycles per second.

Coding agents (Opus 5.5 and Sonnet 5.5 at medium and high effort) 
implemented Game of Life and a composite scheduler frame against 
Ant Farm, oneTBB and Rayon. At 30 to 40 threads Ant Farm led the 
small frame by about 10 to 15% over oneTBB and tied it on the 
heavy frame; at 8 and 15 threads the three were within a few 
percent. Harness and prompts for reproduction: 
github.com/nriddick/antfarm-bench