profiling issues

[Kiith-Sa via Digitalmars-d-learn]

On Friday, 12 September 2014 at 03:23:55 UTC, Vlad Levenfeld 
wrote:
> I've got a library I've been building up over a few projects, 
> and I've only ever run it under "debug" "unittest" and 
> "release" (with dub "buildOptions").
> Lately I've needed to control the performance more carefully, 
> but unfortunately trying to compile with dub --profile gives me 
> some strange errors:
>
> 1) A few lines in one of my modules are reported as 
> "unreachable" by dmd. The data they operate on are defined 
> entirely in code (i.e. not read as external input) so maybe 
> they're getting CTFE'd into oblivion?
> All I know is they're apparently reachable in non-profiled code 
> (and very essential to the business logic... but they're just 
> math functions, nothing crazy, one of the unreachable lines 
> computes the areas of some polygons, another sums the areas up).
>
> 2) The linker complains about undefined references to 
> std.exception.enforce being called from std.stdio.rawRead.
>
> 3) If I try to compile with "buildOptions":["profile"] instead 
> of dub --profile, then it compiles and links but then I 
> segfault on launch at gc_malloc.
>
> I also recall (but can't seem to find) something about 
> profiling not working with multithreaded code? Because almost 
> every encapsulated service in this library runs on its own 
> thread.
>
> And the code base (>15k LOC) isn't easily reduced, as any 
> remotely interesting main method I write pretty much pulls from 
> the entire library. I don't want to have to turn this whole 
> thing inside out. Its like 95% templates and inlining wreaks 
> havoc on the logic as well, but that's another problem for 
> another day...
>
> Does anyone else have these kinds of issues? Are there any 
> alternative methods of coarse-grained profiling (i.e., not 
> manually peppering timer calls into my code)? Whats with the 
> unreachable statements? Any hints on what I can try next to get 
> closer to a performance profile of my code?


Instrumenting 'conventional' profilers such as DMD's builtin 
profiler or gprof are pretty useless for getting reliable data as 
they distort the results. I recommend using a sampling profiler.


With sampling profilers you usually get profiling results down to 
source line or even instruction level and you don't need to 
recompile your binary (having debug symbols is needed for source 
lines, though). They also tend to be able to measure more than 
just time (e.g. cache misses for individual caches, branches 
_and_ branch mispredictions, FPU usage, etc, etc)


If you're on Linux, 'perf' is good (on Ubuntu/Mint, possibly 
other distros just type 'perf' into the console and it will tell 
you what package to install, usually it's 'linux-tools-common').

https://perf.wiki.kernel.org/index.php/Tutorial

It also has the awesome 'perf top' utility that allows you to 
profile in real-time, like 'top' but with functions instead of 
processes.

OProfile is good *if you can get it to run*, very similar in 
usage to perf but I almost always run into some issue.


AMD CodeXL is also decent and on both Linux and Windows, although 
on non-AMD CPUs it can only measure execution time (still very 
useful, down to instruction level).

RotateRight Zoom, Intel VTune should also be good, but both are 
commercial.




If you're writing a game or any other real-time interactive 
application and need to profile occasional lags, you might need a 
different approach
(but in this case you won't avoid manual instrumentation, 
although it's rather easy to use):

http://defenestrate.eu/2014/09/05/frame_based_game_profiling.html

https://github.com/kiith-sa/tharsis.prof