Iterators and Ranges: Comparing C++ to D to Rust

[Petar]

On Tuesday, 15 June 2021 at 10:13:07 UTC, Araq wrote:
> On Tuesday, 15 June 2021 at 08:22:07 UTC, Petar Kirov 
> [ZombineDev] wrote:
>> One of the things that I don't like about languages that I use 
>> that have built-in iterators like C# and JavaScript/TypeScript 
>> is that pretty much their whole ecosystems only cares about 
>> input iteration and things that in D are trivially `O(1)` like 
>> `array.map!foo[$/2 .. $].retro.map!bar.length` end up being 
>> unexpected performance pitfalls in practice in these languages.
>>
>
> Er, performance pitfalls can be found with a profiler.

Yes, but there's a big difference between "can" and "will". 
There's an increasing quantity of JS (not only JS, just it's the 
most prominent example) web, desktop(electron), and mobile 
applications written with complete disregard for performance that 
perform tasks slower than similar programs did way faster 20 
years ago on the hardware from that time. Defaults matter. If an 
inefficient way is also the easiest way to implement a feature it 
will likely be first choice for many and very rarely someone will 
got out of their way to optimize it later. And it's not just JS. 
I've seen this in large C++ codebases as well, where the 
development friction severely limits the approaches that people 
will consider when implementing features. As a random example 
(which you've probably seen): 
https://nee.lv/2021/02/28/How-I-cut-GTA-Online-loading-times-by-70

Also, from experience, profiling modern async UI applications is 
really not straightforward as opposed to CLI apps with no event 
loop. Often there are very few obvious bottlenecks. Instead you 
have "death by a thousand cuts" - small inefficiencies spread 
evenly across the application accumulate to produce an absolute 
bloatware.

> Crashes are much harder to track down. If I have to pick 
> between them, I take the performance pitfalls, as they are much 
> easier to fix. At least for your outlined `.retro.map.length` 
> snippets.

While, obviously correct programs are better than fast but 
incorrect programs, taking such a simplistic view would be 
dangerous mistake. When people who care (or are forced to care) 
about performance track-down a performance bug due to the a use 
of a high level language or library features, often their first 
instinct would be rewrite the code with more primitive 
constructs, which are easier to reason about wr.t. performance.

One may think that say SQL injection is a solved problem, but 
they would be surprised by the number of cases where teams have 
replaced ORM frameworks in parts of their applications with raw 
SQL queries, just because either they didn't understand well the 
ORM framework, or the framework itself was at fault.

In the case of iterator / range libraries, would you rather 
people use raw loops rather than said libraries? Because, 
realistically if people find iterator / range libraries slow, or 
their API too constraining for them to express they're ideas, raw 
loops would be the "solution".

>> That is not to say that I wouldn't want my libraries to be 
>> fool-proof against that kind of problems, but at same time, at 
>> least in my experience with D it hasn't been a big deal, so 
>> I'd much rather have a powerful algorithmic API foundation 
>> than prevent very useful and beautiful designs, just because 
>> they can be misused.
>
> Beauty is subjective, safety isn't.

 From a math point of view, safety/correctness may be a binary 
property, however in practice every type of bug has a different 
probability and impact. Using D ranges compared to raw loops 
greatly increases my ability to reason about the code, diminishes 
amount of bugs it has and also makes it easier to optimize, 
because it is easy to experiment with alternative approaches. If 
the language or ecosystem forced a less expressive iterator 
abstraction on me, most likely I would have to resort to raw 
loops more often and in turn have a greater probability bugs in 
my code.

>>
>>> We're slowly catching up with our support for "view types".
>>
>> Interesting, can you elaborate more?
>
> Wrong forum for that.

Why? The topic is comparing iterator abstractions in different 
languages. Where do you draw line?