Splitter.opSlice(), ranges and const strings

[Jonathan M Davis]

On Thursday, February 24, 2011 12:23:16 Christopher Bergqvist wrote:
> On Thu, Feb 24, 2011 at 11:38 AM, Jonathan M Davis <jmdavisProg at gmx.com> 
wrote:
> > On Thursday 24 February 2011 01:53:33 spir wrote:
> >> On 02/24/2011 08:39 AM, Jonathan M Davis wrote:
> >> > On Wednesday 23 February 2011 22:41:53 Christopher Bergqvist wrote:
> >> >> Hi!
> >> >> 
> >> >> I've run into an issue which I don't understand.
> >> >> 
> >> >> Boiled down code:
> >> >> import std.regex;
> >> >> 
> >> >> void main()
> >> >> {
> >> >> 
> >> >>         //string str = "sdf"; // works
> >> >>         //const string str = "sdf"; // doesn't work
> >> >>         immutable str = "sdf"; // doesn't work
> >> >>         auto pat = regex(", *");
> >> >>         auto split = splitter(str, pat);
> >> >> 
> >> >> }
> >> >> 
> >> >> Error:
> >> >> /Library/Compilers/dmd2/osx/bin/../../src/phobos/std/regex.d(3022):
> >> >> Error: this is not mutable
> >> >> 
> >> >> Should splitter() be able to cope with const/immutable ranges?
> >> >> 
> >> >> (That's with the latest official v2.052 dmd/phobos distribution for
> >> >> mac. I got the same error before upgrading from the v2.051
> >> >> also).
> >> > 
> >> > Pretty much _nothing_ copes with const or immutable ranges. And if you
> >> > think about it, it generally makes sense. You can't pop the front off
> >> > of a const or immutable range. So, how could you possibly process it?
> >> > The are some cases where having tail const with ranges would work
> >> > (assuming that we could have tail const with ranges - which we
> >> > currently can't), but on the whole, const and immutable ranges don't
> >> > really make sense. They can hold const or immutable data, but a const
> >> > or immutable range is pretty useless on the whole.
> >> 
> >> That's one question I'm wondering about for months (but always forget to
> >> ask): Why should /collection/ traversal shrink them? Why does the
> >> regular range stepping func (popFront) read, for arrays:
> >>       this.data = thid.data[1..$];
> >> instead of:
> >>       ++ this.cursor;
> >> ??? Should then be caled eg "stepFront", which imo is much better to
> >> express the semantics of traversal / iteration.
> >> 
> >> I guess the issue expressed in this thread is /invented/ by the regular
> >> process of range, precisely by popFront. There is no reason to mutate a
> >> collection just to traverse it!
> >> 
> >> And then, how do you traverse the collection again?
> >> 
> >> unittest {
> >>      auto a = [1,2,3];
> >>      while (! a.empty()) {
> >>          write(a.front() ,' ');
> >>          a.popFront();
> >>      }
> >>      writeln();
> >> 
> >>      // below nothing written onto terminal
> >>      while (! a.empty()) {
> >>          write(a.front() ,' ');
> >>          a.popFront();
> >>      }
> >>      writeln();
> >> }
> > 
> > ??? Collection traversal _doesn't_ shrink them. Ranges get shrunken when
> > you iterate over them. That's how they work. They're a view into the
> > collection/container. They shrink. It's like car and cdr in lisp or head
> > and tail in haskell. Iterating over a range is very much like processing
> > an slist (as in the functional language type slist, not singly linked
> > lists or the type in std.container).
> > 
> > Now, arrays are a bit funny in that they're kind of both ranges and
> > containers. Now, as Steven pointed out in a recent thread, for a
> > dynamically allocated array, it's like the container is the GC heap, and
> > an array is a range over a portion of that heap. So, traversing an array
> > as a range shrinks it but does not affect its actual container - the GC
> > heap. Static arrays, on the other hand, really do own their memory and
> > are actual containers - hence why you have to slice them to pass to any
> > range-based functions. And you have to slice any other container type as
> > well if you want to pass it to a range-based function.
> > 
> > So, iterating over a collection does _not_ shrink the collection.
> > Iterating over a _range_ does, but a range is a view into a container -
> > a slice of it - so you're just shrinking your view of it as you process
> > it.
> > 
> > - Jonathan M Davis
> 
> I was thinking of something like a C++ const std::vector, but thinking
> of ranges as mutable views into possibly const data seems helpful.
> Thanks!

That's essentially what they are. The main problem, however, is that the primary 
type of range which is used is the dynamic array (in part because std.container 
is still quite new and generally lacking in container types and in part because 
D's dynamic arrays are very powerful) and they're a bit funny. You typically 
think of them as a container, but they're really a slice/range. So, when you go 
and deal with a range into a normal container, it can take a bit of getting used 
to, because with normal containers there's a clear distinction between a range 
over a container and the container itself - unlike dynamic arrays.

- Jonathan M Davis