How to get output of piped process?

[Steven Schveighoffer]

On 2/17/21 1:58 AM, Jedi wrote:
> I an using pipeShell, I have redirected stdout, stderr, and stdin.
> 
> I am trying to read from the output and display it in my app. I have 
> followed this code almost exactly except I use try wait and flush 
> because the app is continuously updating the output. (it outputs a 
> progress text on the same line and I'm trying to poll it to report to 
> the user)
> 
> 
> auto pipes = pipeProcess("my_application", Redirect.stdout | 
> Redirect.stderr);
> scope(exit) wait(pipes.pid);
> 
> // Store lines of output.
> string[] output;
> foreach (line; pipes.stdout.byLine) output ~= line.idup;
> 
> // Store lines of errors.
> string[] errors;
> foreach (line; pipes.stderr.byLine) errors ~= line.idup;
> 
> 
> My code
> 
> auto p = pipeShell(`app.exe "`~f.name~`"`, Redirect.stdout | 
> Redirect.stdin | Redirect.stderr);
> 
> 
>              while(!tryWait(p.pid).terminated)
>              {
>                  string[] output;
>                  foreach (line; p.stdout.byLine)

You need to be careful here -- this will wait until stdout is *closed*.

>                  {
>                      output ~= line.idup;
>                      writeln(line);
>                  }
> 
>                  string[] errors;
>                  foreach (line; p.stderr.byLine)
>                  {
>                      errors ~= line.idup;
>                      writeln("Err:"~line);
>                  }

Same thing here.

>              }
> 
> wait(p.pid);
> 
> None of this works though. What is strange is that when I close out the 
> debugger the app starts working(no console output but I able to see that 
> it is doing something) but is very slow.
> 
> auto p = executeShell(`app.exe "`~f.name~`"`);
> 
> Does work, except I have no output or input. I have another app that I 
> do the exact same code and I can get the output and parse it, but this 
> is after the app terminates. I imagine the issue here is that I'm trying 
> to get the output while the app is running.

Without knowing the pattern of what your app is outputting, it's hard to 
tell what will happen.

The most common problem with people dealing with piped output is not 
reading data off the pipe, which then makes the child process hang 
trying to write to the pipe, because the buffer is full.

For instance, if your process outputs tons of stuff to stderr, you will 
hang, because you are waiting for stdout to be closed first before you 
read anything from stderr, the child process fills up stderr pipe, and 
is put to sleep waiting for it to be writable, never closing stdout.

Unfortunately, std.process wraps all the pipes in File structs, so you 
have almost no good mechanisms to properly read the data.

-Steve