[Issue 2137] New: Data not compressed on fly when adding to zip archive

[d-bugmail at puremagic.com]

http://d.puremagic.com/issues/show_bug.cgi?id=2137

           Summary: Data not compressed on fly when adding to zip archive
           Product: D
           Version: 2.014
          Platform: PC
        OS/Version: All
            Status: NEW
          Severity: minor
          Priority: P2
         Component: Phobos
        AssignedTo: bugzilla at digitalmars.com
        ReportedBy: dsimcha at yahoo.com


The fact that, when an ArchiveMember is added via addMember() to a ZipArchive,
the data in expandedData is not compressed on the fly, makes it very difficult
to create large zip files without running out of memory.  I have created a
simple workaround  by moving the following code from the build() method to the
addMember() method:

de.crc32 = std.zlib.crc32(0, cast(void[])de.expandedData);
de.compressedData =
cast(ubyte[])std.zlib.compress(cast(void[]))de.expandedData;
de.compressedData = de.compressedData[2 .. de.compressedData.length - 4];
de.compressedSize = de.compressedData.length;
de.expandedSize = de.expandedData.length;

I also added the following to free the memory previously occupied by
expandedData:

de.expandedData=null;

This is actually proving to be a significant issue for me, as I am attempting
to use std.zip to archive large amounts of DNA sequence, which compresses
extremely well, but won't fit in memory uncompressed.


--