Recommendations on avoiding range pipeline type hell

[Adam D. Ruppe]

On Saturday, 15 May 2021 at 13:46:57 UTC, Chris Piker wrote:
> I'm trying to do that, but range3 and range2 are written by me 
> not a Phobos wizard, and there's a whole library of template 
> functions a person needs to learn to make their own pipelines.  
> For example:

Phobos has plenty of design flaws, you don't want to copy that.

Generally you should just accept the range with a simple foreach 
in your handler.

```
void processRange(R)(R r) {
    foreach(item; r) {
       // use item
    }
}
```

If you want it forwarded in a pipeline, make a predicate that 
works on an individual item and pass it to `map` instead of 
trying to forward everything.

If you're creating a range, only worry about the basic three 
functions: empty, front, and popFront. That's the minimum then it 
works with most phobos things too. That's where I balance ease of 
use with compatibility - those three basics let the phobos ones 
iterate through your generated data. Can't jump around but you 
can do a lot with just that.


(personally btw I don't even use most of this stuff at all)



> // ... what's with the . before the ElementType statement?

Now that is good to know: that is a D language thing meaning 
"look this up at top level".

So like let's say you are writing a module with

```
void func();

class Foo {
     void func();

     void stuff() {
          func();
     }
}
```

The func inside stuff would normally refer to the local method; 
it is shorthand for `this.func();`.

But what if you want that `func` from outside the class? That's 
where the . comes in:

```
void func();

class Foo {
     void func();

     void stuff() {
          .func(); // now refers to top-level, no more `this`
     }
}
```

In fact, it might help to think of it as specifically NOT wanting 
`this.func`, so you leave the this out.


> What the heck is that?

idk i can't read that either, the awful error message are one 
reason why i don't even use this style myself (and the other is 
im just not on the functional bandwagon...)

Most the time std.algorithm vomits though it is because some 
future function required a capability that got dropped in the 
middle.

For example:

some_array.filter.sort


would vomit because sort needs random access, but filter drops 
that. So like sort says "give me the second element" but filter 
doesn't know what the second element is until it actually 
processes the sequence - it might filter out ALL the elements and 
it has no way of knowing if anything is left until it actually 
performs the filter.

And since all these algorithms are lazy, it puts off actually 
performing anything until it has some idea what the end result is 
supposed to be.


The frequent advice here is to stick ".array" in the middle, 
which performs the operation up to that point and puts the result 
in a freshly-created array. This works, but it also kinda 
obscures why it is there and sacrifices the high performance the 
lazy pipeline is supposed to offer, making it process 
intermediate data it might just discard at the next step anyway.

Rearranging the pipeline so the relatively destructive items are 
last can sometimes give better results. (But on the other hand, 
sorting 100,000 items when you know 99,000 are going to be 
filtered out is itself wasted time... so there's no one right 
answer.)


anyway idk what's going on in your case. it could even just be a 
compile error in a predicate, like a typo'd name. it won't tell 
you, it just vomits up so much spam it could fill a monty python 
sketch.

> messages. (Using interfaces or *gasp* casts, would break the 
> TMI situation.)

i <3 interfaces

it is a pity to me cuz D's java-style OOP is actually pretty 
excellent. a few little things I'd fix if I could, a few nice 
additions I could dream up, but I'm overall pretty happy with it 
and its error messages are much better.

but too many people in the core team are allergic to classes. and 
i get it, classes do cost you some theoretical performance, and a 
lot of people's class hierarchies are hideous af, but hey they 
work and give pretty helpful errors. Most the time.

> better know all of std.traits and std.meta cause you're going 
> to need them too implement a range-of-ranges consumer.

Write your function like it is Python or javascript - use the 
properties you want on an unconstrained template function.

void foo(T)(T thing) {
     // use thing.whatever
     // or thing[whatever]
     // or whatever you need
}

Even if that's a range of ranges:

void foo(T)(T thing) {
       foreach(range; thing)
       foreach(item; range)
            // use item.
}

It will work if you actually get a range of ranges and if not, 
you get an error anyway. It isn't like the constraint ones are 
readable, so just let this fail where it may. (In fact, I find 
the non-contraint messages to be a little better! I'd rather see 
like "cannot foreach over range" than "no match for <spam>")

I don't even think phobos benefits from its traits signatures. If 
you do it wrong it won't compile the same as if you do all the 
explicit checks.


But again, if you're doing some intermediate processing... try to 
use map, filter, fold, and friends... since doing the forwarding 
they do is legitimately complicated and my little foo consumers 
here don't even touch it.

> Here's an actual (though formatted by me) error message I got 
> stating that two things were different and thus couldn't share 
> an array.  Can you see the difference?  I can't.  Please point 
> it out if you do.

idk.... maybe with the full code i could guess and check my way 
to something but i too lazy rn tbh.