Pragma msg goes out of memory when supplied with large data.

[realhet]

On Wednesday, 21 May 2025 at 15:20:57 UTC, realhet wrote:
> Why are these simple looking things are so slow in compile time?

Now I learned why: I had the misconception that CTFE is using the 
compiler itself to generate code and then runs it with the CPU.  
In reality I've found out it's an interpreter.
On top of this, mutating data, even a single byte of a 1 MByte 
array will allocate the whole array again, that's why I got the 
out of memory errors even with 64K data.
I also know that the compiler only deallocates at the very end.

I will keep these limitations in my head, CTFE is still an 
awesome thing.

My experiments with integrating external data processing into the 
LDC2 compiling process were a success:
- export: It was possible to export data (20MByte) from the 
compiler in seconds, using pragma(msg, ...)  Just don't touch the 
large data with CTFE.
- import: It was also possible to import large data fast, by 
using import(filename). Where the file was served using 
ProjectedFS.

This is basically a modified version of the StringImport (-J) 
language construct: It is a StringImport that redirects data to 
an external program and imports the resulting data from that 
external programs into the D source code.
(Also an even bigger security risk than the original one :D)