unzip parallel, 3x faster than 7zip

Jay Norwood jayn at prismnet.com
Thu Apr 5 07:04:55 PDT 2012


I uploaded a parallel unzip here, and the main in the examples 
folder.  Testing on my ssd drive, unzips a 2GB directory 
structure in 17.5 secs.  7zip took 55 secs on the same file.  
This restores timestamps on the regular files.  There is also a 
loop which will restore timestams on folders.  It can be 
uncommented if the fix is added to std.file.setTimes that allows 
timestamp updates on folders.  I documented a fix that I tested 
in issue 7819.

https://github.com/jnorwood/file_parallel

http://d.puremagic.com/issues/show_bug.cgi?id=7819


This has similar limitations to std.zip,  Only does inflate or 
store, doesn't do decryption.  There is a 4GB limit based on the 
32 bit offsets limit of the zip format used.  It processes 40MB 
blocks of files, and uses std.parallelism foreach loop.  If the 
archived entry is larger than 40MB it will attempt to load it 
into memory, but there currently is no expansion technique in 
there to split a large single entry into blocks.

I used the streams io to avoid the 2GB file limits still in stdio.


More information about the Digitalmars-d-announce mailing list