Optimization fun

[Kiith-Sa via Digitalmars-d]

On Saturday, 8 November 2014 at 01:53:33 UTC, H. S. Teoh via 
Digitalmars-d wrote:
> On Fri, Nov 07, 2014 at 05:31:44PM -0800, Walter Bright via 
> Digitalmars-d wrote:
>> On 11/7/2014 4:41 PM, H. S. Teoh via Digitalmars-d wrote:
> [...]
>> >But speaking of which, I found dmd -profile's output in 
>> >trace.log a
>> >little difficult to understand because of the lack of
>> >self-documenting headers in the first section. gprof, for 
>> >example,
>> >produces nicely-labelled output that describes what the data 
>> >means.
>> >Would you accept a PR along these lines?
>> 
>> I don't really know what you have in mind, so I'll have to say 
>> "it
>> depends".  Keep in mind that subsequent runs of the profiler 
>> parse the
>> previous output file and add it to the results.
>
> Just add a simple header that describes what each column means.
>
>
>> >Also, in the second section, I'm getting some negative 
>> >numbers for
>> >the call times, which seem to indicate integer overflow? This
>> >basically destroys the usefulness of dmd -profile for my test 
>> >cases,
>> >since most of the interesting cases for me are those with long
>> >running times, which for sure will run into integer overflow 
>> >at the
>> >sample rate that dmd -profile is using, causing the second 
>> >section to
>> >be essentially randomly sorted.
>> 
>> Yeah, a negative number is likely an overflow. Reduce the test 
>> case!
>
> Unfortunately, I can't. The behaviour I'm trying to profile is 
> the
> long-term running case, because there's a whole bunch of setup 
> at the
> beginning that initially takes a while, but eventually it's the
> long-running part that dominates the runtime behaviour. 
> Reducing the
> test case will only increase the initial noise, which I'm not 
> interested
> in, and reduce the running time of the main loop of interest. 
> Besides,
> what I'm trying to optimize for is the large, complex case; the 
> simple
> cases already run fast enough that the performance 
> characteristics are
> not that interesting to me. It's the long-term part that's 
> interesting
> because that's when the GC starts kicking in and pressure on 
> system RAM
> starts to increase.
>
> I'm surprised that dmd's profiler can't even handle something 
> that only
> runs for 7-8 seconds or so! gprof certainly has no such 
> limitation.
>
> Is it relatively simple to make dmd -profile use larger integer 
> widths
> for profiling? If not, I'm afraid I'm just gonna have to stick 
> with
> gdc/gprof instead.
>
>
> T

Except for very specific cases, neither gprof or DMD's profiler 
are good for profiling. If you're on Linux, you have perf, which 
works well with D and is way ahead of anything the DMD profiler 
will be able to do *after* man-years of further development.

See (shameless advertisement):

http://defenestrate.eu/_static/profiling-slides/index.html#22

(especially perf record/perf report)