We should deprecate -release

[Richard (Rikki) Andrew Cattermole]

On 18/07/2024 3:51 AM, Walter Bright wrote:
> On 7/15/2024 11:20 AM, Richard (Rikki) Andrew Cattermole wrote:
>> Right, but it's dmd we are talking about.
>>
>> A compiler that no matter what switches are set internally or 
>> externally cannot compete with ldc or gdc.
> 
> People benchmark it anyway.

And get bad results.

I really need you to understand just how much things have changed, and 
this bit of experience no longer translates to this lesson. I do accept 
and enjoy the history of the switch when it did translate to this lesson 
however!

This is a real benchmark that came from a Go article that made the 
rounds a while ago, here is my version of it with results I just reran:

https://news.ycombinator.com/item?id=39106972

https://www.reddit.com/r/golang/comments/199u7np/from_slow_to_simd_a_go_optimization_story/

```
$ dmd -O -release -Iinterface main.d -run attempt.d
27033600000     12 secs, 689 ms, 420 ╬╝s, and 5 hnsecs

$ dmd -O -release -Iinterface main.d -run attempt2.d
27033600000     15 secs, 764 ms, 182 ╬╝s, and 9 hnsecs

$ ldc2 -O3 -Iinterface main.d -run attempt.d
27033600000     6 secs, 28 ms, 747 μs, and 4 hnsecs

$ ldc2 -O3 -Iinterface main.d -run attempt2.d
27033600000     14 secs, 939 ms, 722 μs, and 3 hnsecs
```

attempt.d is completely naive, attempt2 does an explicit loop unroll 
with static foreach.

If I turn on release for ldc (or swap to .ptr on the slices):

```
$ ldc2 -O3 -release -Iinterface main.d -run attempt.d
27033600000     6 secs, 70 ms, 191 μs, and 4 hnsecs

$ ldc2 -O3 -release -Iinterface main.d -run attempt2.d
27033600000     5 hnsecs
```

Dmd simply does not have the optimizations to get anywhere close to a 
modern backend. It _cannot_ come close to winning benchmarks.

Note: attempt2 with the second optimization is as good as hand rolled 
assembly by the author of the article.

>> It is better to slap a message both in the version/help message of the 
>> compiler and on the site that says "don't use dmd for reviewing of 
>> performance of D use ldc/gdc instead".
> 
> People doing benchmarks do not read the site. That's the whole problem.

And D goes to the bottom of the benchmark.

Except in cases where the D community takes over, like here:

https://github.com/jinyus/related_post_gen/tree/main

A lot of effort went into that by quite a few people.

If anything all of this is making me go: "drop dmd from download page". 
Problem solved.

>> Given this, it is better to simply remove the switch as it fools 
>> people into doing unsafe things that give them a very bad day.
> 
> People don't develop code using a -release switch. Those spending time 
> coding in D (rather than spending as little time as possible writing a 
> benchmark) will be likely to spend time reading what a switch does 
> before using it.

They don't.

We the community catch it and tell them not to use it.

Repeatedly over the last 12 years it has been a very common occurrence.

This is a lesson that has been learned by many people and keeps coming 
up as not what they wanted to use.

> I've been in this business a long time. What people should do and what 
> they do do is rarely aligned. People who do comparative benchmarks spend 
> as little time as possible benchmarking a language/compiler they are not 
> familiar with.

Yes that is what happened in the past, I agree it's a good story for 
showing how things use to be. For benchmarks that make the rounds today 
that isn't what happens.

I linked two different benchmarks that did make the rounds in the last 
year. One was benchmarking Go's backend, the other accepted 
contributions and we won because we could drive LLVM better.

Neither was benchmarking the frontend or language itself.