Bug? taskPool.map() with bufSize and writeln() gets stuck

[Ali Çehreli]

On 02/11/2012 12:56 AM, Martin Nowak wrote:
 > On Sat, 11 Feb 2012 02:31:29 +0100, Ali Çehreli <acehreli at yahoo.com> 
wrote:
 >
 >> Sorry for the double-post; I have asked the same question on D.learn
 >> earlier but I think this is more of a question to this forum.
 >>
 >> Tested on Ubuntu 11.10 64-bit dmd.
 >>
 >> The following program gets stuck during the writeln() call.
 >>
 >> - Note that the foo() call alone works fine.
 >>
 >> - Also note that the program works fine when there is no writeln()
 >> call nor foo() call. All elements get processed in that case and the
 >> results are ignored.
 >>
 >> Am I using taskPool.map incorrectly or is this a bug? Can you help
 >> identify where the problem may be? How is writeln() using the range
 >> differently than foo() to cause this behavior?
 >>
 >> import std.stdio;
 >> import std.parallelism;
 >> import core.thread;
 >>
 >> int func(int i)
 >> {
 >> writeln("processing ", i);
 >> return i;
 >> }
 >>
 >> void main()
 >> {
 >> auto results = taskPool.map!func([1,2,3,4,5,6,7,8], 2);
 >>
 >> writeln(results); // <-- Gets stuck HERE
 >>
 >> foo(results); // this works fine
 >> }
 >>
 >> void foo(R)(R range)
 >> {
 >> for ( ; !range.empty; range.popFront()) {
 >> writeln(range.front);
 >> }
 >> }
 >>
 >> Thank you,
 >> Ali
 >
 > Yeah, you have a deadlock in there, it's somewhat hidden though.
 > The issue is that writeln will take lock on stdout once. This will 
deadlock
 > with the lazy processing of the map range.

Thank you.

I was trying to visualize the semi-lazy nature of taskPool.map. Now I 
get what I want when the writeln() call in main() is changed to be on 
stderr:

import std.stdio;
import std.parallelism;
import core.thread;

int func(int i)
{
     writeln("processing ", i);
     Thread.sleep(dur!"seconds"(1));
     return i;
}

void main()
{
     auto results = taskPool.map!func([1,2,3,4,5,6,7,8], 2);

     stderr.writeln(results);    // <-- now on stderr
}

Now the output hints at how taskPool.map is semi-lazy:

processing 1
processing 2
[processing 3
1, 2processing 4
, 3processing 5
, 4processing 6
, 5, 6processing 7
processing 8
, 7, 8]

Good. :)

Ali