Coming IO features in Tango

[Sean Kelly]

Ingo Oeser wrote:
> 
>>> Or even better: Support Open MP! GCC 4.2.x has everything there already,
>>> and I'm sure DMD will get this one day :-)
> 
> What about this Open MP stuff? Might be useful in container classes 
> and for loops, as Intel's TCB show. For D and its foreach() loops, 
> it should be even easier to prove possible vectorisation and parallelism.

I think it's all pretty nice, but could really do with a redesign for D. 
  Intel's TBB in particular would be much cleaner with D-specific 
features like delegates.

As for parallel foreach, I've been wondering at the actual utility of 
foreach as a language feature vs. something implemented in library code. 
  Without a means of specifying whether order matters, I suspect we may 
end up doing something like this:

foreach( e; myArray.inParallel ) {
     // blah
}

where inParallel is a function that returns a struct or delegate that 
handles the actual parallelization work.  But this will close the door 
on compiler optimizations to do the same thing, which something like 
this would allow:

foreach(any)( e; myArray ) {
     // blah
}

Subtle difference I suppose, but I do think it would be prudent to plan 
ahead for compiler-based parallel loop unrolling and such, insofar as 
language features are concerned.  A bunch of other suggestex syntaxes 
were suggested a while back but I can't recall them offhand.

>> Green may be another story however, because such designs typically imply
>> some sort of scheduling mechanism and such.  I'm simply not convinced
>> that there's any way to implement them effectively in a language like D.
>>   And besides, some OSes take care of this for you--Solaris, for example.
> 
> Good to know that you did a wise decision here :-)
> 
>> Thread pools are a common tool for multiplexed IO.  So much so, in fact,
>> that it's impossible to use IOCP without one.  
> 
> Oh, I didn't know of that deficiency.

It's not necessarily a deficiency.  The IOCP design basically builds on 
a kernel-controlled thread pool such that a lot of context switches are 
simply eliminated so long as events occur in a timely fashion.  Though 
since one must assume a multithreaded programming model, it does 
typically require the use of mutexes and such somewhere in the program, 
unlike poll-oriented designs (I believe).

>> But I agree that libraries shouldn't take control away from the user. 
>> In fact, my personal approach with Tango is that it should be easy to 
>> use in the average case, but provide an elegant means to get "to the
>> metal" for discerning programmers.  
> 
> I'm more concerned of features, which hide O(N^2) algorithms without 
> stating it or have effects like priority inversion and deadlocks, 
> due to behind you back decisions ("hey starting a realtime thread is a nice 
> hack here"), which are considered "good for you".

We're committed to that not happening with Tango, though feedback from 
users always helps.


Sean