4x speedup of recursive rmdir in std.file

[Andrei Alexandrescu]

On 2/5/12 10:16 AM, Nick Sabalausky wrote:
> "Jay Norwood"<jayn at prismnet.com>  wrote in message
> news:jgm5vh$hbe$1 at digitalmars.com...
>> == Quote from Nick Sabalausky (a at a.a)'s article
>>> Interesting. How does it perform when just running on one core?
>>
>> The library without the threads is 1 min 5 secs for the 1.5GB
>> directory structure with about 32k files.  This is on an 510
>> series intel ssd.  The win7 os removes it in almost exactly the
>> same time, and you can see from their task manager it is also
>> being done single core and only a small percentage of cpu.  In
>> contrast, all 8 threads in the task manager max out for a period
>> when running this multi-thread remove. The regular file deletes
>> are occurring in parallel.  A single thread removes the directory
>> structure after waiting for all the regular files to be deleted by
>> the parallel threads.  I attached a screen capture.
>>
>
> What I'm wondering is this:
>
> Suppose all the cores but one are already preoccupied with other stuff, or
> maybe you're even running on a single-core. Does the threading add enough
> overhead that it would actually go slower than the original single-threaded
> version?
>
> If not, then this would indeed be a fantastic improvement to phobos.
> Otherwise, I wonder how such a situation could be mitigated?

There's a variety of ways, but the simplest approach is to pass a 
parameter to the function telling how many threads it's allowed to 
spawn. Jay?

Andrei