Compilation memory use

[Stefan Koch]

On Monday, 4 May 2020 at 17:00:21 UTC, Anonymouse wrote:
> TL;DR: Is there a way to tell what module or other section of a 
> codebase is eating memory when compiling?
>
> I'm keeping track of compilation memory use using zsh `time` 
> with some environmental variables. It typically looks like this.
>
> ```
> $ time dub build -c dev
> Performing "debug" build using /usr/bin/dmd for x86_64.
> [...]
> Linking...
> To force a rebuild of up-to-date targets, run again with 
> --force.
> dub build -c dev   9.47s  user 1.53s system 105% cpu 10.438 
> total
> avg shared (code):         0 KB
> avg unshared (data/stack): 0 KB
> total (sum):               0 KB
> max memory:                4533 MB
> page faults from disk:     1
> other page faults:         1237356
> ```
>
> So it tells me the maximum memory that was required to compile 
> it all. However, it only tells me just that; there's no way to 
> know what part of the code is expensive and what part isn't.
>
> I can copy dub's dmd command and run it with `-v` and try to 
> infer that the modules that are slow to pass semantic3 are also 
> the hungry ones. But are they?
>
> Is there a better metric?

I do have a custom dmd build with tracing functionality, but the 
profiles are not very user friendly and woefully under-documented.

https://github.com/UplinkCoder/dmd/tree/tracing_dmd

You can use the source of the file `src/printTraceHeader.d` to 
see how the profile is written, and by extension read.

The actual trace file format is in `src/dmd/trace_file.di`

you have to throw the -trace=$yourfilename switch when compiling.

I am happy to assist with interpreting the results. though for 
big projects it's usually too much of a mess to really figure out.