D 2015/2016 Vision?

[Marco Leise via Digitalmars-d]

Am Tue, 6 Oct 2015 18:27:28 -0700
schrieb Walter Bright <newshound2 at digitalmars.com>:

> On 10/4/2015 11:02 AM, bitwise wrote:
> > For example, streams.
> 
> No streams. InputRanges.

... what bitwise said ...
We had this discussion at least once and it did not change my
mind back then: Ranges and streams have overlapping use cases,
but they are not interchangeable. What they have in common is
that they are both lazy and process data from start to end.

=== Composability ===
Streams work on a series of bits coming from the sender. Class
polymorphism allows them to be wrapped into each other as
needed at any point during the transmission. This includes
applying or reversing compression, encryption and endianness
changes. New filter streams can be loaded through DLLs and
extend the hierarchy.

void main()
{
  InputStream is;

  // Simple stream
  is = new FileStream("./test.txt");

  // File from the web, unzipped with a dynamically loaded
  // filter plugin 
  is = new HttpStream("http://a.xy/test.gz");
  auto fs = cast(FilterStream) Object.factory("UnGzipStream");
  fs.wrapped = is;
  is = cast(InputStream) fs;

  // One common interface works with all combinations. No need
  // to know the stream types at compile time, no
  // combinatorial template explosion
  processTestFile(is);
}

void processTestFile(InputStream is);

=== Primitives ===
While ranges assume a fixed structure over all elements,
streamed data embeds information about upcoming data types and
sizes. To handle these, instead of getting the next "item"
with .front, streams provide a different set of primitives:
bulk reads, wrappers for basic data types (this is where
endianness filters apply) and often bit-wise reads to handle
compressed data.

=== Buffering ===
Since input ranges/streams can not generally be wound back,
one natural component of streams is a buffer, similar to
what .byLine does for text, but more flexible. A circular
buffer that can grow is a good candidate. Like .byLine it
offers slicing, or generally referencing its contents at the
current read position without copying. Several primitives are
provided to ask for a number of bytes to be buffered or
dropped. This does not only obviate the need to check for "end
of file" or "end of buffer" everywhere but also enables code
reuse in cases like reading a BigInt off a stream, which takes
a char[] as ctor argument. With a buffered stream you increase
the look-ahead until the entire number string is buffered and
can be passed to BigInt() by reference. BigInt could be
changed to be constructible from a range, but the raw
look-ahead enables use of SIMD to scan for end-of-line or
end-of-number that traditional input range primitives don't.

-- 
Marco