D-ark corners - algorithms, ranges, and auto ref - best practices guides.

[aliak]

Hi, I recently ran in to a problem with stack corruption [0]. I 
basically used up more time than I'd like to admit trying to 
figure out heisenbugs - segfaults, out of memory exceptions, 
thread crashes, and once I saw an overlapping memory access error 
(which I'd never seen before, and never saw again).

Anyway, this is what I narrowed it down to eventually in client 
code.

@safe:
auto foundValue = makeRange("one")
   .algorithm;
useValue(foundValue); // BOOM

With some help from the forums, the essence of the problem was 
the following:

1. algorithm returned by auto ref.
2. the range's front was "ref front() {...}"

What was happening was that madeRange made a temporary and passed 
it on to algorithm, which also saw a temporary. The return type 
on algorithm was auto ref, and since front on the range also 
returned a ref, I was returning a ref to a value in a temporary. 
Algorithm was in a dependency, and the range type was inside a 
different dependency (confession - I wrote both those 
dependencies but, heh, anyway)

The solutions were to either:

1. slap a return on the front function: ref T front() return 
{...} (I think - not even sure)
2. remove auto ref from algorithm and return by value

There are a number of issues, and also a number of questions.

- The problem with solution 1 is that you usually do not have 
control over libraries and their ranges. They may have been 
declared with return, they may have not. I went through Phobos 
and saw a lot of ref front() {...}
- The problem with solution 2 is that you want to return by ref 
so you can to avoid copying.

(Side note: the specification for return ref - 
https://dlang.org/spec/function.html#return-ref-parameters - says 
that "inout ref parameters imply the return attribute.", but I 
got corruption with "ref inout(T) front() inout {...}". Is that a 
bug with inout?)

The access pattern is safe when:
- The object that is passed in to algorithm is not a temporary.
- The algorithm returns by copy

Best practices seem to be:
- If you are returning by ref from any object accessor, always 
use the return qualifier. Is there any reason not to?

The only reliable way to solve this from an algorithm author 
point of view seems to be to have the algorithm return by value. 
But then you sacrifice efficiency. Are there any other ways? Can 
it ever be safe to return by auto ref from a generic algorithm?

And finally, I've been keeping note of various gotchas in D, and 
traps, etc [1] and I'm wondering if anyone else takes note of 
these things? Can you post links to them here if you do?

I know about the D-idioms thing [2] (great info!!). But it feels 
that when you write libraries for D, there're a lot of nuanced 
and non-obvious gotchas like these and they're not documented 
anywhere that I know of. Should we start doing this somewhere - 
the name of which is in the title of the post :)

Cheers,
- Ali

[0]: 
https://forum.dlang.org/thread/hhxabgmrwvqvmezezyym@forum.dlang.org
[1]: https://github.com/aliak00/d-isms/tree/master/da-faq
[2]: https://p0nce.github.io/d-idioms/