Coming IO features in Tango

[Sean Kelly]

Ingo Oeser wrote:
> Sean Kelly wrote:
> 
>> Paul Findlay wrote:
>>> Sean Kelly wrote:
>>>> Fibers/Coroutines are useful in that they vastly simplify the creation
>>>> of state machines in many cases.  They certainly aren't suitable for
>>>> every task, but I think they have enough general utility that they
>>>> should be available to the user.
> 
> So the state of that state machine becomes at least 4K always? 
> Ok, that's less state than a thread. But if you write a state machine,
> you need usually less. If you need more, you can use a thread.

Good point.  The state machines I use tend to require very little 
memory, though I'd be willing to trade that for 4k in some instances if 
it simplified the programming.

> There usually are heavy states and small states. Heavy states involve
> several synchronisation primitives and are usually done by threads or 
> processes. Small states are just a bunch of handles and are in the range of
> some hundred bytes and use the state machines of the OS kernel (e.g. file
> position, and 2 file handles -> FTP-Server-Control-Session).

Or even less.  I've used state machines for formatted IO, and the 
requirements there are often minuscule.

>>> Are fibers able to take advantage of the machine's cache any better than
>>> swapping threads (with their dramatically larger stack space) in and out?
>> Probably not.  But context switching fibers is much faster than context
>> switching threads, which is a selling point in some cases.  
> 
> So better improve the threading support :-)

Well, it's more an OS limitation than anything.

> Did you measure your claim? Did you dirty the caches 
> (aka "use the state of the state machine"), before context switching?

Nope.  But Mikola Lysenko, the author of StackThreads on which Tango 
fibers are based performed some tests when developing coroutines and his 
StackThreads were substantially faster for multiplexing tasks.  Though 
perhaps some of this difference was because StackThreads don't need 
mutexes to share data.

>> Another being that a non-running fiber can be passed between threads 
>> just like a delegate.
> 
> That's the first valid argument for me pro Fibers. But doesn't that make 
> scope and dataflow analysis harder for the compiler? With delegates you just
> have to prove, that the object members are not accessed. With Fibers, you
> have to prove the same thing for the whole stack the fiber uses.

Yes.  The compiler isn't aware of fibers, so dataflow analysis isn't any 
better than it would be with threads.

> Good that this "If you don't use it, we won't either." mantra is implemented 
> in Tango.

That's the goal :-)

> Only dirty hack left is inpl()/outpl() and friends. They are simply not
> available on many architectures and are getting unimportant on PC due to
> MMIO. A modern language like D should handle them as special address space, 
> which is a nice feature for a compiler and required for some 
> micro controllers and some DSPs. In C this is supported via CPP-hacks :-/

I'll have to read up on these instructions.  Hadn't heard of them before.


Sean