Is there something like a consuming take?

[Jonathan M Davis]

On Saturday, July 6, 2019 12:17:26 PM MDT Jonathan M Davis via Digitalmars-
d-learn wrote:
> On Saturday, July 6, 2019 8:12:36 AM MDT berni via Digitalmars-d-learn
>
> wrote:
> > Now it's getting weird. Meanwhile I encountered, that take()
> > sometimes consumes and sometimes not. Where can I learn, what is
> > the reason behind this behavior? And how can I handle this?
>
> take _always_ consumes the range that it's given. The problem is that some
> types of ranges are implicitly saved when they're copied, whereas others
> aren't, and when a range is implicitly saved when it's copied, you end up
> with the copy being consumed. Dynamic arrays are implicitly saved when
> they're copied, so the range you pass is saved, and the copy is consumed
> instead of the original.
>
> In generic code, you have to assume that once a range has been copied, you
> can't use it anymore (just the copy) precisely because the semantics of
> copying differ depending on the type of the range. You can only use a
> range after copying it if you know what type of range you're dealing with
> and how it behaves. So, you can rely on a dynamic array implicitly saving
> when it's passed to take, but in generic code, you really shouldn't be
> using a range again once you pass it to take, because what actually
> happens is dependent on the type of the range.
>
> In general what this means is that if you pass a range to a function, and
> you then want to use the range again afterwards, you need to call save
> when passing it to the function, and otherwise, you just assume that it's
> consumed and don't use it again. You certainly don't pass it to a
> function with the expectation of some elements being consumed and then
> continue to use the rest of the range unless the function takes its
> argument by ref or by pointer (which relatively few range-based functions
> do).
>
> If you want a function that's guaranteed to not implicitly copy a range,
> then it needs to accept the argument by ref or take a pointer to it. In
> the case where a function doesn't have to do the work lazily, ref would
> work, but in a case like take where you're returning a wrapper range,
> pointers would be required. So, a version of take that didn't ever copy
> the range it's given and thus never risked implicitly saving the range it
> was passed would have to either take a pointer to it or take it by ref
> and then take the address of the ref. In either case, the code using such
> a take would then have to ensure that the original range didn't leave
> scope and get destroyed before the take range was consumed, or the take
> range would then refer to invalid memory.

Another thing to consider about lazy ranges that take pointers to avoid
implicit saving and ensure that they consume the original is that if the
original has anything popped from it before the wrapper range is fully
consumed, then that will screw up the wrapper range, because the elements in
the range it's wrapping will have changed. Wrapper ranges are really
designed with the idea that they're operating on their own copy of the
range. Even with take as it's currently implemented, you risk bugs if you
pass it a range which is a reference type without calling save, because if
you don't fully consume the take range before using the original range
again, you end up screwing up the elements in the take range when you pop
anything off of the original range.

It really doesn't work well to have a wrapper range which doesn't have its
own independent copy of the range to iterate over. The only case where it
works cleanly to have a wrapper range like take gives you and not have to
iterate through the elements again to have a range starting at the first
element after the take range is to have a random-access range that has
slicing, in which case, you can just slice it, and take isn't required.
Without a random-access range with slicing, if you want to only iterate
through the elements once, you really should just be iterating through the
range manually rather than using a wrapper range.

- Jonathan M Davis