dip1000 + pure is a DEADLY COMBO

[12345swordy]

On Wednesday, 12 May 2021 at 13:14:30 UTC, Dennis wrote:
> Sorry for the attention-grabbing title, but I think it's 
> warranted, because the gist of it is this:
>
> **With `-preview=dip1000` enabled, the compiler will happily 
> compile valid, `@safe` D code into memory corrupting machine 
> code.**
>
> The root cause is:
> [Issue 20150 - -dip1000 defeated by 
> pure](https://issues.dlang.org/show_bug.cgi?id=20150)
>
> The compiler ignores "reference to local variable `x` assigned 
> to non-scope parameter `y`" errors when the function is 
> annotated or inferred `pure`. The idea is, presumably, that 
> `pure` functions can't escape references because they have no 
> interaction with global variables. This is false of course, 
> since they can still return them or assign them to other 
> parameters.
>
> The deadly part it that using this flawed logic, the compiler 
> sometimes turns GC allocations into stack allocations too 
> eagerly. Here I got memory corruption because the compiler 
> allocated an array literal on the stack instead of the heap:
>
> [Issue 21291 - Array literal that escapes scope is allocated on 
> stack](https://issues.dlang.org/show_bug.cgi?id=21291)
>
> Later I encountered another instance of it where a closure was 
> not heap-allocated, which looked something like this:
>
> ```D
> import core.thread;
> @safe:
> void main() {
>     S s;
>     s.memberFunc();
> }
>
> struct S {
>     int a;
>     auto memberFunc() {
>         auto t = new Thread({
>             auto pa = &a; // pointer to stack frame of main!
>         });
>     }
> }
> ```
>
> I'm not the only one who encountered memory corruption bugs 
> this way, user Dechcaudron commented on my issue: "This has 
> also happened to me, no idea it could be due to -dip1000".
>
> And most recently:
> [Issue 21912 - Invalid stack closure when calling delegate 
> inside lambda](https://issues.dlang.org/show_bug.cgi?id=21912)
>
> ### Why is this not fixed?
>
> Walter made a PR for fixing the behavior in dmd: (March 2020)
> https://github.com/dlang/dmd/pull/10924
>
> Later, aG0aep6G made a better fix: (November 2020)
> https://github.com/dlang/dmd/pull/12010
>
> But they're both blocked by the fact that Phobos relies on the 
> bug to compile with -dip1000. This makes sense, because the 
> conversion process was mostly "add `scope` and `return` 
> annotations until the compile errors go away". `pure` functions 
> did not give error messages, so they did not get those 
> annotations.
>
> Regarding this extra work, [aG0aep6G 
> commented:](https://github.com/dlang/dmd/pull/12010#issuecomment-759070687) (January 2021)
>
>> I had started on it, but it's tedious work tracking down the 
>> errors through templates and overloads. If I remember 
>> correctly, dup gave me some trouble, too.
>>
>> So I've put it on ice for the time being. If someone else 
>> wants to give it a shot, that would be great.
>
> And that's where we are now.
>
> ### Future of dip1000
>
> Matthias asked "Is there a plan to enable DIP1000 by default?" 
> during [DConf Online 2020 Day One Q \& A Livestream, at 
> 4:50:11](https://youtu.be/o-2_mxaCL9w?t=17411).
> Walter mentioned "we can do it now" and Atila mentioned how the 
> first step would be to change -dip1000 errors into equivalent 
> deprecation warnings.
>
> Clearly, issue 20150 is a blocker for dip1000 by default.
>
> In the meantime, since I absolutely don't want another 
> unfortunate soul debugging memory corruption bugs that dip1000 
> introduces, this post is meant to raise awareness, and discuss 
> intermediate solutions.
>
> Maybe the compiler can defensively heap-allocate for now, 
> though that would break `@nogc` code. Or maybe we can add 
> another switch, `-preview=dip1000proper`, since the fix is a 
> breaking change. What do you think?

Should it be a bug with Pure rather than dip1000?

-Alex