Templates are slow.

Lewis via Digitalmars-d digitalmars-d at puremagic.com
Thu Sep 8 12:01:20 PDT 2016


I recently went through the process of optimizing the build time 
on one of my projects. I started at ~3.08s, and got it down to 
~1.6s. The project is around 7000 non-comment-non-whitespace LOC. 
I timed the build in a pretty non-rigourous fashion (I just timed 
the python script that kicks off a command-line call to dmd and 
waits for it to finish), however I only cared about making 
changes that resulted in large improvements to the build time, so 
this was good enough for my purposes.

I too found that template instantiation was responsible for a lot 
of the extra build time. I found running dmd -v very helpful in 
tracking down excessive template instantiations or other places 
where the compiler was doing a lot of work that could be avoided.

The steps I took were as follows:

- Start (3.08s)
- I was using custom assert function that grabbed __LINE__ and 
__FILE__ as template arguments, meaning each of the ~130 assert 
calls required a separate instantiation. I switched to passing 
those in as run-time arguments (2.85s)
- I had a similar wrapper around some logging functions in 
std.experimental.logger. I made a small change to 
std.experimental.logger to allow a call path with no template 
instantiations, and similarly fixed my own wrapper. I had ~70 
logging calls (2.7s)
- Recompiled DMD with VS2015 (2.5s)
- Overclocked my CPU :D (2.3s)
- Created a file called heavytemplates.d that built to a .lib in 
a separate build step. The first templates I pulled out were a 
couple std.regex calls and instantiations (1.9s)
- Changed some tuples into structs (negligible improvement)
- Pulled several templates into heavytemplates.d that instantiate 
recursively over the Gamestate (a very large struct) (1.75s)
- Pulled out template instantiations used by msgpack-d, which 
also instantiate recursively over the Gamestate for save/load of 
the game (1.6s)

Of all of these, I was most surprised by the gain I got from 
pulling out std.regex calls into a separate build (0.4ms). 
Whether or not I used compile-time regexes didn't seem to affect 
build time substantially, just that I used anything at all. Also, 
whether I had one regex call or five didn't seem to matter, 
likely because std.regex instantiates using the string type as a 
parameter, and I just used plain old 'string' for all regex uses.

There's still work I could do, but at some point I start to get 
diminishing returns, and have to actually work on features 
instead of just optimizing my build  :D


More information about the Digitalmars-d mailing list