Streaming library

[Rainer Deyke]

On 10/14/2010 15:49, Andrei Alexandrescu wrote:
> Good point. Perhaps indeed it's best to only deal with bytes and
> characters at transport level.

Make that just bytes.

Characters data must be encoded into bytes before it is written and
decoded before it is read.  The low-level OS functions only deal with
bytes, not characters.

Text encoding is a complicated process - consider different unicode
encodings, different non-unicode encodings, byte order markers, and
Windows versus Unix line endings.  Furthermore, it is often useful to
wedge an additional translation layer between the low-level (binary)
stream and the high-level text encoding layer, such as an encryption or
compression layer.

Writing characters directly to streams made sense in the pre-Unicode
world where there was a one-to-one correspondence between characters and
bytes.  In a modern world, text encoding is an important service that
deserves its own standalone module.


-- 
Rainer Deyke - rainerd at eldwood.com