[dmd-concurrency] Shutdown protocol

[Sean Kelly]

Perhaps this shutdown protocol is the reason for spawn_linked in  
erlang?  Last week, Andrei and I were trying in vain to come up with a  
use case for bidirectional linking.

Sent from my iPhone

On Jan 20, 2010, at 9:20 AM, Andrei Alexandrescu <andrei at erdani.com>  
wrote:

> I like this. All - any related thoughts?
>
> Truth be told we can fix the example by changing main:
>
> wait(spawn(&copy("in1", "out1")), spawn(&copy("in2", "out2")));
>
> which I think is entirely fair.
>
>
> Andrei
>
> Michel Fortin wrote:
>> I know this has been discussed at length while I was sleeping, but  
>> I also see two problems with the shutdown protocol currently in the  
>> draft.
>> First, it's a nice concept that the main thread waits for other  
>> threads to shutdown before exiting. It's a simple concept which  
>> allows other threads to finish their work before exiting. Except  
>> for this: any thread that requires communication to finish its work  
>> cannot exit properly. Another problem is that threads waiting a  
>> shutdown signal might not end until the very termination of the  
>> application.
>> For instance, here is a modified version of Andrei's copy example  
>> which copies two files in parallel by putting each copy operation  
>> in a separate thread:
>>    import std.algorithm, std.concurrency, std.stdio;
>>    void main() {
>>        spawn(&copy("in1", "out1"));
>>        spawn(&copy("in2", "out2"));
>>    }
>>    void copy(string input, string output) {
>>        File source = File(input, "r");
>>        enum bufferSize = 1024 * 100;
>>        auto tid = spawn(&writer, outputPath); // Read loop
>>        foreach (immutable(ubyte)[] buffer; stdin.byChunk 
>> (bufferSize)) {
>>            send(tid, buffer);
>>        }
>>    }
>>    void writer(string output) {
>>        File target = File(output, "w");
>>        // Write loop
>>        for (;;) {
>>            auto buffer = receiveOnly!(immutable(ubyte)[])();
>>            target.rawWrite(buffer);
>>        }
>>    }
>> Given the current shutdown protocol, this wouldn't work at all. The  
>> shutdown state would make writer() exit before copy has a chance to  
>> read most of the input. True, you could just a 'join()' to make the  
>> main thread explicitly wait for the two copy threads it spawned,  
>> but this would leave a writer thread inactive in the background  
>> while the other copy finishes. Add more files to copy and you'll  
>> get a lot of waisted writer threads doing nothing until the program  
>> terminates.
>> So I'd propose a simple change to the shutdown protocol to make it  
>> scale better: when any thread finishes, it puts all the threads it  
>> spawned in shutdown state and exits (except for the main thread  
>> witch must wait before exiting to keep the other threads running).
>> This somehow creates a ownership hierarchy between threads:
>>    main --> file copy thread 1 -> writer thread 1
>>             \-> file copy thread 2 -> writer thread 2
>> What happens in the above example is this: main puts the two file  
>> copy threads in shutdown mode. When a file copy thread finishes  
>> reading, it puts its writer thread in shutdown mode, which then  
>> exits.
>> I think this scales better.
>> (Note: it should be possible to change the owner thread for cases  
>> where this hierarchy isn't appropriate.)
> _______________________________________________
> dmd-concurrency mailing list
> dmd-concurrency at puremagic.com
> http://lists.puremagic.com/mailman/listinfo/dmd-concurrency