Ant Farm - A concurrent M:N jobs distributor

[Nathan Riddick]

On Monday, 24 August 2026 at 23:04:33 UTC, Nathan Riddick wrote:
> Hello, I had an idea for a non-FIFO concurrent callback-jobs 
> thing based on a ring buffer. It seems to have strong 
> performance and utility for my next set of tinkerings. I came 
> up with ~90% of the spec and made some attempts at a clean 
> implementation, but Kimi K3 ultimately came through with an 
> initial clean version and various other LLMs were very useful 
> at grinding out late-night debug sessions with infinite 
> patience.
>
> tl;dr synthetic throughput sits between moodycamel no-tokens 
> and with-tokens at a few million items/sec, seems to prefer 
> more concurrency rather than less, the latency at reaching 
> one-more item after a big block of work crushes FIFO designs 
> generally, it's adaptive to a homogenous thread pool that 
> shrinks or grows over time, API is @nogc nothrow @system and 
> probably thread-safe.
>
> https://github.com/nriddick/antfarm

Whoops, I had meant a few *hundred* million items/sec as 
synthetic numbers. I've also added a topology-aware threadpool 
manager and Fiber compatibility layer using the Ant Farm as a 
concurrent distributor, that probably still needs TLC as I 
continue to tinker. I'd appreciate feedback in any form even if 
it's to just put down the crack pipe and seek a priest for LLM 
exorcism.

The std fiber package executing on a single thread appears better 
overall for async work. The Ant Farm backing remains interesting 
(to me at least) for work that actually wants concurrency but 
also wait-on signals and more normal control flow.