Threads don't scale, processes do - Fibers

[Oliver Dathe]

> I find the concept of userland scheduling and green threads interesting 
> and would work on this with you if you wish.

I'd really like to follow up on this, in collaboration even better! 
Unfortunately I'm very busy until some time in October.

However, feel free to do what you like and I'd enjoy dicussing. Have you 
had any similar plans before?

> Interpreted languages such as Erlang can pause their green threads / 
> green processes easily; not so with natively compiled code. This may be 
> one of the largest issues with scheduling native fibers. You might be 
> able to get around it with enough hacks, though some would have to 
> involve your OS's kernel, I have no doubt.

Yes, preemptive greenthreading would fit into this even better! But its 
no prerequisite, so I chose the collaborative way with Fibers for this. 
Not sure if the preemptive way could get accomplished with help of 
signal handling in a safe way.

Since the proposal is for Fibers, the responsibility to yield is tied to 
the app developer. You're aiming for more genericity or even language 
support, right? A compiler could also place the yields based on 
heuristics or even supported by profiler runs and a chosen scheduling 
policy. But yes, preemption would be cooler. If blocking functions would 
be marked as such, then a compiler could also place the appropriate 
yields with the announcement that I proposed for that.

What makes me a bit suspicious about the model is that it tries to 
manage ressources that are already managed for it. Review point 1 where 
it should try to keep up to C=(number of CPUs) Threads in a schedulable 
state from the OS's point of view. Thats just a stupid heuristic. So I 
see two main operation areas for this concept: greedy applications like 
games and applications with weak realtime demands.

However it does not incur any restrictions on other types of 
applications and indeed they could profit from interactivity, reduced OS 
responsibility/activity/overhead and the multicore story. Some work that 
was previously only managable by the OS could be offloaded into the 
application. E.g. fiberspecific semaphores can get implemented with a 
Queue!(Fiber) for waiters plus the counter in collaboration with the 
userspace scheduler. That is what I was aiming for in point 4.