std.process: how to process stdout chunk by chunk without waiting for process termination

[Steven Schveighoffer]

On Tue, 18 Jun 2013 17:41:57 -0400, Timothee Cour  
<thelastmammoth at gmail.com> wrote:

> I'd like to do the following:
>
> auto pipes = pipeShell(command, Redirect.stdout | Redirect.stderr);
>
> while(true){
> version(A1)
>   string line=pipes.stdout.readln;
> version(A2)
>   auto line=pipes.stdout.readChunk(10);
> version(A3)
>   auto line=pipes.stdout.readChar();
>
>   // do something with line
>
>   if(tryWait(pipes.pid).terminated)
>     break;
> }
>
>
> The problem is that 'string line=pipes.stdout.readln;' seems to block  
> until
> the process is terminated, ie if the command is a long running command  
> that
> prints a line every 1 second for 10 seconds, this program will wait 10
> seconds before starting the processing.
> I also tried with rawRead, readf, fgetc but couldn't make it work.
> I'm on OSX, if that matters.
>
> Is there any way to achieve this?

I think the issue is on the child process side.  If you are using buffered  
I/O you have to flush the buffer.

For instance, if the child is using D writeln or C printf, and you are  
using stdout, then it will only flush after writing 4096 bytes.  You can  
flush early by calling flush on stdout, or fflush in C.  Note that C will  
auto-detect if it is an interactive console, and flush via newlines  
instead.  So running the same program from the console will flush every  
line!

Alternatively, you can set the flush policy to flush after every line.   
See here:

https://developer.apple.com/library/ios/#documentation/System/Conceptual/ManPages_iPhoneOS/man3/setvbuf.3.html

And here:

http://dlang.org/phobos/std_stdio.html#.File.setvbuf

-Steve