Coming IO features in Tango

[Ingo Oeser]

Hi Sean,

Sean Kelly wrote:
> I think the proposed IO system will largely be a generalised abstraction
> for multiplexing mechanisms provided by the OS: IOCP on Win32, epoll on
> Linux, etc.

Ok, so that context stuff will be dead/eliminated code on Linux, 
if I don't use it. I can live with that :-)

>> You'll also have much fun with unixoide systems and signals, if you do
>> that.
>> 
>> What about something like jobs and job queues?
> 
> I've been thinking about adding these anyway, though it might be kind of
> interesting to mix job processing with the IO dispatch mechanism.

They are orthogonal. Let the user decide, what he puts on a job. Any program 
sequence, which is required to be a ordered sequence is a job. Your thread
pools should then just pull these jobs from some queue. Putting jobs to 
thread pools is an important system library service. Job dispatch is a 
superset of IO dispatch.

>> Or even better: Support Open MP! GCC 4.2.x has everything there already,
>> and I'm sure DMD will get this one day :-)

What about this Open MP stuff? Might be useful in container classes 
and for loops, as Intel's TCB show. For D and its foreach() loops, 
it should be even easier to prove possible vectorisation and parallelism.

> Green may be another story however, because such designs typically imply
> some sort of scheduling mechanism and such.  I'm simply not convinced
> that there's any way to implement them effectively in a language like D.
>   And besides, some OSes take care of this for you--Solaris, for example.

Good to know that you did a wise decision here :-)

> Thread pools are a common tool for multiplexed IO.  So much so, in fact,
> that it's impossible to use IOCP without one.  

Oh, I didn't know of that deficiency.

> But I agree that libraries shouldn't take control away from the user. 
> In fact, my personal approach with Tango is that it should be easy to 
> use in the average case, but provide an elegant means to get "to the
> metal" for discerning programmers.  

I'm more concerned of features, which hide O(N^2) algorithms without 
stating it or have effects like priority inversion and deadlocks, 
due to behind you back decisions ("hey starting a realtime thread is a nice 
hack here"), which are considered "good for you".

People with really big iron (1024 CPU and more with matching RAM and 
storage) will become very angry with you :-)

At least your interfaces should not try to annoy them. Implementation
doesn't matter, because it can be changed without any effect.

> I think we have succeeded thus far, and hope that this can extend to the
> new IO package as well. 

Nearly. inpl() and outpl() are a No-Go for a multi platform library.
Requiring threads and such stuff is also ok, with per-platform stuff.
But No-Go as a mandatory library interface.

PS: Should we follow up to digitalmars.D?


Best Regards

Ingo Oeser