Bug? taskPool.map() with bufSize and writeln() gets stuck

[Martin Nowak]

On Sat, 11 Feb 2012 17:18:21 +0100, Ali Çehreli <acehreli at yahoo.com> wrote:

> On 02/11/2012 12:56 AM, Martin Nowak wrote:
>  > On Sat, 11 Feb 2012 02:31:29 +0100, Ali Çehreli <acehreli at yahoo.com>  
> wrote:
>  >
>  >> Sorry for the double-post; I have asked the same question on D.learn
>  >> earlier but I think this is more of a question to this forum.
>  >>
>  >> Tested on Ubuntu 11.10 64-bit dmd.
>  >>
>  >> The following program gets stuck during the writeln() call.
>  >>
>  >> - Note that the foo() call alone works fine.
>  >>
>  >> - Also note that the program works fine when there is no writeln()
>  >> call nor foo() call. All elements get processed in that case and the
>  >> results are ignored.
>  >>
>  >> Am I using taskPool.map incorrectly or is this a bug? Can you help
>  >> identify where the problem may be? How is writeln() using the range
>  >> differently than foo() to cause this behavior?
>  >>
>  >> import std.stdio;
>  >> import std.parallelism;
>  >> import core.thread;
>  >>
>  >> int func(int i)
>  >> {
>  >> writeln("processing ", i);
>  >> return i;
>  >> }
>  >>
>  >> void main()
>  >> {
>  >> auto results = taskPool.map!func([1,2,3,4,5,6,7,8], 2);
>  >>
>  >> writeln(results); // <-- Gets stuck HERE
>  >>
>  >> foo(results); // this works fine
>  >> }
>  >>
>  >> void foo(R)(R range)
>  >> {
>  >> for ( ; !range.empty; range.popFront()) {
>  >> writeln(range.front);
>  >> }
>  >> }
>  >>
>  >> Thank you,
>  >> Ali
>  >
>  > Yeah, you have a deadlock in there, it's somewhat hidden though.
>  > The issue is that writeln will take lock on stdout once. This will  
> deadlock
>  > with the lazy processing of the map range.
>
> Thank you.
>
> I was trying to visualize the semi-lazy nature of taskPool.map. Now I  
> get what I want when the writeln() call in main() is changed to be on  
> stderr:
>
> import std.stdio;
> import std.parallelism;
> import core.thread;
>
> int func(int i)
> {
>      writeln("processing ", i);
>      Thread.sleep(dur!"seconds"(1));
>      return i;
> }
>
> void main()
> {
>      auto results = taskPool.map!func([1,2,3,4,5,6,7,8], 2);
>
>      stderr.writeln(results);    // <-- now on stderr
> }
>
> Now the output hints at how taskPool.map is semi-lazy:
>
> processing 1
> processing 2
> [processing 3
> 1, 2processing 4
> , 3processing 5
> , 4processing 6
> , 5, 6processing 7
> processing 8
> , 7, 8]
>
> Good. :)
>
> Ali
>
Going with foreach(e; results) will work too and you can write to the  
right output.