Differences in results when using the same function in CTFE and Runtime

[Carsten Schlote]

On Thursday, 8 August 2024 at 13:31:43 UTC, user1234 wrote:

> That being said, that'd be intersting to verify if it's the 
> problem. Maybe update your code to use `real` variables and 
> let's see.

Replaced the 'doubles' with 'real' and now the the results of the 
CTFE and RT executions are exactly the same.

So, as you expected and described, the CTFE lost some precision 
by splitting the ops and storing the intermediate results back 
from FPU registers with internal precision to the limited 
'double' type somewhere in memory.

However, the other fractals (mandelbot, julia, newton, lyapunov, 
...) work prefectly with the 'double's. I suspect, that their 
default/starting views do no involve enough recursions and 
iterations in the calculation, so that the missing precision is 
not important, yet. I will pimp my unittests to catch such 
precision issues for the other fractals as well, and then fix it 
by using 'real' types.

So, what are the lessions learned here:

1. When using floats use the 'real' type - it uses the maximum 
available FPU precision and no precision is lost, when the FPU 
register is stored/read back to/from memory. So CTFE and RT 
execute have the best chance to produce the same results.

2. Use 'float' and 'double' only if you need to save some bits of 
storage and if the reduced precision is acceptable.

3. Unittests proofed again to be great. Especially when support 
for unittesting is built into the language. Even simplest 
assumptions can be wrong and having assertions for such 
assumptions will be helpful.

4. Use assert()/enforce(), in/out/do to check everything to match 
your expectations and assumptions.

At the end ```assert(burningShipImageCTFE == burningShipImageRT, 
"Same results expected.");``` in my unitest did find it. And I 
got a chance to think about it and work on my assumptions ;-)