[phobos] Transcoded text stdio

[Andrei Alexandrescu]

Hi Shin and everyone,


Regarding transcoding output, please let me know I understand the 
problem correctly: under Windows (and possibly under other OSs under 
certain configurations) the console is not UTF and cannot be reasonably 
forced to be UTF.

I think for such situations, the classic Decorator-based design with 
stacked interfaces works well: you have a TranscodingStream wrapping a 
NativeStream or a UTFStream or whatever.

The streaming interface question comes again, i.e. what is the interface 
that allows such stacking with minimal cost in efficiency?

File was not designed for transcoding, but as long as it supports raw 
reads and writes, I think writing a wrapper over it should be possible. 
I'm talking about something like this:

auto nativeStdout = nativeTranscoder(stdout);
nativeStdout.writeln("yah");

The native transcoder would only use rawWrite and flush for stdout - not 
the higher level text functions.

Then we can define more sophisticated transcoders, e.g. one that 
transcodes from UTF to some Eurasian codepages etc.

Works?


Andrei

On 8/6/10 1:50 CDT, Shin Fujishiro wrote:
> Hello,
>
> I'm trying to integrate codeset conversion facility to std.stdio.
> But how can it be done?
>
> Mixing transcoded and non-transcoded (UTF-8) I/O in the same File
> structure will mess up the source.  I think separating UTF-8 based I/O
> and transcoded I/O is necessary.
>
> I could think of the following four ways.
>
> 1.  Integrate everything in the File anyway.
>
> 2.  Make the File to always perform conversion.
>
> 3.  Create a distinct type for transcoded I/O.
> ----------
>      shared TranscodedFile stdout;
>      stdout.writeln("Hallå, Värld!");
> ----------
> # http://github.com/sinfu/misc/blob/master/stdio/test01.d
>
> 4.  Simplify the File and define upper layer structures.
> ----------
>      // File itself doesn't provide byLine etc.
>      shared File stdout;
>
>      // these 'ports' perform actual I/O for specific purposes
>      shared UTF8TextIOPort stdoutUTF8;
>      shared NativeTextIOPort stdoutText;
>      shared BinaryIOPort stdoutBin;
>
>      // wrap stdout with various 'I/O ports'
>      stdoutUTF8 = UTF8TextIOPort(stdout);
>      stdoutText = NativeTextIOPort(stdout);
>      stdoutBin = BinaryIOPort(stdout);
>
>      // write text in UTF-8
>      stdoutUTF8.writeln("Hallå, Värld!");
>
>      // write text in console encoding
>      stdoutText.writeln("Hallå, Värld!");
>
>      // free functions use stdoutText
>      writeln("Hallå, Värld!");
> ----------
> # http://github.com/sinfu/misc/blob/master/stdio/test02.d
>
> ...
>
> I'm uncertain of which is the best.  Perhaps there are more reasonable
> ways.  What do you think?  Any ideas?
>
>
> Thanks,
> Shin
> _______________________________________________
> phobos mailing list
> phobos at puremagic.com
> http://lists.puremagic.com/mailman/listinfo/phobos