Safely writing to the same file in parallel foreach loop

[Jonathan M Davis]

On Thursday, November 15, 2012 15:33:31 Joseph Rushton Wakeling wrote:
> On 11/14/2012 10:17 PM, Jonathan M Davis wrote:
> > I would point out though that given how expensive disk writes are, unless
> > you're doing a lot of work within the parallel foreach loop, there's a
> > good
> > chance that it would be more efficient to use std.concurrency and pass the
> > writes to another thread to do the writing. And the loop itself should
> > still be able to be a parallel foreach, so you wouldn't have to change
> > much otherwise. But with the synchronized block, you'll probably end up
> > with each thread spending a lot of its time waiting on the lock, which
> > will end up making the whole thing effectively single-threaded.
> 
> Do you mean that the synchronized {} blocks have to all be completed before
> the threads can all be terminated?

No, I mean that if you have a bunch of threads all trying to get the mutex for 
the synchronized block, then the only one doing anything is the one in the 
synchronized block. Once it's done, it then just loops back around, quickly 
going through whatever calculations it has to do before hitting the 
synchronized block again. In the meantime one of the other threads got the 
synchronized block and is writing to disk. But all the other threads are still 
waiting. So, for each of the threads, almost all of the time is spent blocked, 
making it so that most of the time, only one thread is doing anything, which 
completely defeats the purpose of having multiple threads.

>From the sounds of it, this doesn't really affect you, because you're doing 
expensive calculations, but anything with very fast but parallelizable 
calculations could be totally screwed by the synchronized block.

- Jonathan M Davis