The new std.process is ready for review

[Lars T. Kyllingstad]

On Saturday, 9 March 2013 at 18:35:25 UTC, Steven Schveighoffer 
wrote:
> On Sat, 09 Mar 2013 11:05:14 -0500, Lars T. Kyllingstad 
> <public at kyllingen.net> wrote:
>
>> On Wednesday, 6 March 2013 at 16:45:51 UTC, Steven 
>> Schveighoffer wrote:
>>> On Tue, 05 Mar 2013 17:38:09 -0500, Vladimir Panteleev 
>>> <vladimir at thecybershadow.net> wrote:
>>
>>>> I've also initially tried writing a different program:
>>>>
>>>> [...]
>>>
>>> Linux should work here.  From what I can tell, you are doing 
>>> it right.
>>>
>>> If I get some time, I'll try and debug this.
>>
>> I think I know what the problem is, and it sucks bigtime. :(
>>
>> Since the child process inherits the parent's open file 
>> descriptors, both ends of a pipe will be open in the child 
>> process.  We have separated pipe creation and process 
>> creation, so spawnProcess() knows nothing about the "other" 
>> end of the pipe it receives, and is therefore unable to close 
>> it.
>>
>> In this particular case, the problem is that "sort" doesn't do 
>> anything until it receives EOF on standard input, which never 
>> happens, because even though the write end of the pipe is 
>> closed in the parent process, it is still open in the child.
>
> Oh crap, that is bad.
>
> Unlike Windows which is an opt-in strategy, unix has an opt-out 
> strategy (there is the F_CLOEXEC flag).  For consistency, I 
> think it would be good to close all the file descriptors before 
> calling exec.
>
>> I don't know how to solve this in a good way.  I can think of 
>> a few alternatives, and they all suck:
>>
>> 1. Make a "special" spawnProcess() function for pipe 
>> redirection.
>> 2. Use the "process object" approach, like Tango and Qt.
>> 3. After fork(), in the child process, loop over the full 
>> range of possible file descriptors and close the ones we don't 
>> want open.
>>
>> The last one would let us keep the current API (and would have 
>> the added benefit of cleaning up unused FDs) but I have no 
>> idea how it would impact performance.
>
> I think 3 is the correct answer, it is consistent with Windows, 
> and the most logical behavior.  For instance, if other threads 
> are open and doing other things that aren't related (like 
> network sockets), those too will be inherited!  We should close 
> all file descriptors.

I think so too.  In C, you have to know about these things, and 
they are specified in the documentation for fork() and exec().  
In D you shouldn't have to know, things should "just work" the 
way you expect them to.


> How do you loop over all open ones?  Just curious :)

You don't.  That is why I said solution (3) sucks too. :)  You 
have to loop over all possible non-std file descriptors, i.e. 
from 3 to the maximum number of open files.  (On my Ubuntu 
installation, this is by default 1024, but may be as much as 
4096.  I don't know about other *NIXes)

Here is how to do it:

import core.sys.posix.unistd, core.sys.posix.sys.resource;
rlimit r;
getrlimit(RLIMIT_NOFILE, &r);
for (int i = 0; i < r.rlim_cur; ++i)
     close(i);

Lars