Ant Farm - A concurrent M:N jobs distributor

[Nathan Riddick]

On Sunday, 30 August 2026 at 21:21:46 UTC, Nathan Riddick wrote:
> On Sunday, 30 August 2026 at 21:11:01 UTC, Sebastiaan Koppe 
> wrote:
>> On Monday, 24 August 2026 at 23:04:33 UTC, Nathan Riddick 
>> wrote:
>>> [...]
>>> https://github.com/nriddick/antfarm
>>
>> This is amazing work, thank you.
>>
>> Do you have any plans for things on top?
>
> I'm working on an actor model with managed lifetimes, where 
> dispatch onto the ant farm is an exclusive borrow and the actor 
> can mutate external (non-buffer) state through its handle. The 
> WIP is in the main branch.

My psychosis has progressed. My idea of an "actor" system bolted 
onto the ring transport was barely nascent before, and now it 
comes in two flavors: "Public" identities with inboxes and 
copyable handles. And "private" actors where root actors are 
parked with no external owner and narrow external access in the 
form of requesting a cancellation, and children have one 
exclusive parent. All kinds can be dispatched in generationed 
"waves", where the wave is a fence guaranteeing for example that 
participants can safely view and edit their exclusively borrowed 
private state and publish a public-safe projection for the next 
generation. Plus assorted usability features and bug fixes.

I also went from synthetic ~300 million Payloads per second on 
the Ryzen 5500 to ~1.7 billion. That's single delivery of 1-word 
payloads w/ associated function pointer callback, concurrently 
from 4 producers to 8 consumers (6 cores 12 threads). And big 
batches, typical synthetic throughput caveats etc. The biggest 
gains were compacting/eliding per-Payload headers when tables 
have more uniform composition, often chopping 128 byte headers 
down to 8 or even 0. Actors can range from tens to hundreds of 
millions dispatched per second depending on your lifetime, 
allocation and execution strategies. Huge pages after these 
optimizations now don't do much good, where before they could net 
~50% extra throughput.
https://github.com/nriddick/antfarm/blob/main/PERFORMANCE.md

Conway's Game of Life, one actor per cell 
https://github.com/nriddick/antfarm/tree/main/examples/life

`examples/life` is a toroidal board where every cell is a private 
child actor. The cell callback reads its eight wrapped neighbours 
from the current projection and writes only its own slot in the 
next one. One actor per cell is deliberately "1:1" rather than 
targeting step speed:

```d
output[index] = cast(ubyte) (neighbours == 3 || (input[index] && 
neighbours == 2));
```

The root owns the cohort and runs one wave per generation:

```d
if (run.cells !is null)                         // resumed: the 
previous wave joined
{
     if (context.cancelled || run.cells.outcome(context).cancelled)
         return PrivateRootDisposition.finish;   // discard a 
partial wave
     run.board.current = 1 - run.board.current;  // swap 
projections
     ++run.board.generation;
     --run.remaining;
}
if (context.cancelled || run.remaining == 0)
     return PrivateRootDisposition.finish;
// first time through: createChildren!Cell(w * h), spawn one per 
cell
run.cells.start!stepCell(context);
return PrivateRootDisposition.waitChildren;
```

The calling application owns the two buffers, and there are no 
extra fences. Ordering chain goes like this, also see 
`docs/design`:

1. Cell callback writes.
2. Shard completion join.
3. Wave credits.
4. Root resumes from `waitChildren`.
5. Next wave is published.
6. Next wave's reads.

256×256 board, 200 generations per request, including cohort 
creation and collection:

| Workers | Generations/s | Cell updates/s |
| ---: | ---: | ---: |
| 0 (inline) | 1,631 | 107M |
| 2 | 2,707 | 177M |
| 6 | 4,079 | 267M |

65,536 actors stepping in joined waves now rival what the raw 
transport layer could do last time in updates per second.

Also builds to termbox2 TUI app. It has pause, single-step, 
randomize, a glider stamp, cursor editing, and speed control:

```text
make -C examples life
examples/.build/life 2      # 2 workers; 0 runs on the UI thread
```


Anyways, hope someone else finds my delirious ravings useful. I 
have a 16c/32t AMD 5900XT on the way to replace the 6x/12t 5500 
and probe higher thread counts / multi CCX.