unzip parallel, 3x faster than 7zip
Jay Norwood
jayn at prismnet.com
Thu Apr 5 07:04:55 PDT 2012
I uploaded a parallel unzip here, and the main in the examples
folder. Testing on my ssd drive, unzips a 2GB directory
structure in 17.5 secs. 7zip took 55 secs on the same file.
This restores timestamps on the regular files. There is also a
loop which will restore timestams on folders. It can be
uncommented if the fix is added to std.file.setTimes that allows
timestamp updates on folders. I documented a fix that I tested
in issue 7819.
https://github.com/jnorwood/file_parallel
http://d.puremagic.com/issues/show_bug.cgi?id=7819
This has similar limitations to std.zip, Only does inflate or
store, doesn't do decryption. There is a 4GB limit based on the
32 bit offsets limit of the zip format used. It processes 40MB
blocks of files, and uses std.parallelism foreach loop. If the
archived entry is larger than 40MB it will attempt to load it
into memory, but there currently is no expansion technique in
there to split a large single entry into blocks.
I used the streams io to avoid the 2GB file limits still in stdio.
More information about the Digitalmars-d-announce
mailing list