std.compress

Marco Leise Marco.Leise at gmx.de
Tue Jun 4 18:17:01 PDT 2013


Am Mon, 03 Jun 2013 20:44:04 -0700
schrieb Walter Bright <newshound2 at digitalmars.com>:

> Comments welcome.

LZW is a nice and fast general purpose algorithm and I
welcome its addition to Phobos to build file format readers
from it (MS-DOS compress, GIF, TIFF) or even just to compress
data on the fly in RAM. Most people seem to have moved on to
zlib though for pretty much anything else.

Actually I just happened to attempt something similar.
Influenced by your talk about modularity and bioinfornatic's
micro benchmarking with reading FASTA files I try to wrap up
the concepts of bit streams and algorithms processing them.
But some of my design goals are different:

a) Not-Invented-Here must take precedence. :D
b) There is no other measure than bytes/second.
c) Every algorithm must run in its own thread for maximal
   parallelism. (like Unix process piping)

So it is not about parallel algorithms, but building
processing pipelines that work like Unix where only circular
buffers need to be shared from one algorithm to the next.

Am Mon, 3 Jun 2013 23:40:06 -0700
schrieb Timothee Cour <thelastmammoth at gmail.com>:

> A)
> there already is std.zlib; why not have:
> std.compress.zlib: public import std.zlib
> std.compress.lzw: put this new module there instead of in std.compress
> std.compress.image.png
> std.compress.image.jpg

Yes and no. Compression algorithms should be in std.compress and share
the same API, but image file formats in std.image.* or std.fileformat.*.
You don't look into std.compress when you want to open *.bmps and *.jpgs.

Am Tue, 04 Jun 2013 01:00:03 -0700
schrieb Walter Bright <newshound2 at digitalmars.com>:

> On 6/3/2013 11:40 PM, Timothee Cour wrote:
> > D)
> > CircularBuffer belongs somewhere else; maybe std.range or std.container
> 
> I have mixed feelings about that. If you'll notice, std.compress doesn't have 
> any imports! I wanted to make at least one module that doesn't pull in 100% of 
> everything in Phobos (one of my pet peeves).

I have nothing to add to the discussion on THAT matter, but
a compromise should be found between few massive imports (D)
and hundreds of tiny imports (Java). :)

-- 
Marco



More information about the Digitalmars-d mailing list