a pretty exciting result for parallel D lang rmd following defrag by name

[Jay Norwood]

On Sunday, 22 April 2012 at 09:33:59 UTC, Marco Leise wrote:
> So when you did your first measurements, with 160 seconds for 
> rmd, did you wait for the I/O to complete? Sorry if that's a 
> stupid question :p but that's the obvious difference when using 
> write-through from what the documentation says.

You are probably right that the buffers are just not flushed. I 
unzipped with 7zip, then waited 5 min, then used rmdir below, 
which still took 144 secs.

G:\>cmd /v:on /c "echo !TIME! & rmdir /q /s tz & echo !TIME!"
11:47:32.40
11:49:56.70

If I use 7zip to unzip, then reboot prior to rmdir, the rmdir 
takes only 27 sec

G:\>cmd /v:on /c "echo !TIME! & rmdir /q /s tz & echo !TIME!"
12:47:38.13
12:48:05.60

If I use 7zip to unzip, then run sync on G:, rmdir takes 2:36
http://technet.microsoft.com/en-us/sysinternals/bb897438
C:\Ntutils>sync g:

Sync 2.2: Disk Flusher for Windows 9x/Me/NT/2K/XP
Copyright (C) 1997-2004 Mark Russinovich
Sysinternals - www.sysinternals.com

Flushing: G

G:\>cmd /v:on /c "echo !TIME! & rmdir /q /s tz & echo !TIME!"
13:10:11.93
13:12:47.75





according to that ntfs document, the buffers should have been 
scheduled for flush within about 8 secs.

http://msdn.microsoft.com/en-us/library/windows/desktop/aa364218(v=vs.85).aspx

"To ensure that the right amount of flushing occurs, the cache 
manager spawns a process every second called a lazy writer. The 
lazy writer process queues one-eighth of the pages that have not 
been flushed recently to be written to disk. It constantly 
reevaluates the amount of data being flushed for optimal system 
performance, and if more data needs to be written it queues more 
data. Lazy writers do not flush temporary files, because the 
assumption is that they will be deleted by the application or 
system.
"

This is measurement below is for rmdir immediately following 
unzip with the writeThrough code  ... about 6 secs.
G:\>cmd /v:on /c "echo !TIME! & rmdir /q /s tz & echo !TIME!"
12:07:58.23
12:08:04.26

I can't explain why the writeThrough enables faster rmdir 
operation, but I saw similar improvement following myDefrag 
sortByName on the folder, which is what I was attempting to 
duplicate with the sorted unzip.  At this time it provides the 
only reasonable way to get consistent results ... the lazy 
flushing doesn't seem to work, myDefrag takes too long, the sync 
utility doesn't seem to improve anything, and reboot isn't really 
an option.