std.variant benchmark

[Peter Alexander]

On Sunday, 29 July 2012 at 14:43:09 UTC, Dmitry Olshansky wrote:
> On 29-Jul-12 18:17, Andrei Alexandrescu wrote:
>> On 7/29/12 8:17 AM, Gor Gyolchanyan wrote:
>>> std.variant is so incredibly slow! It's practically unusable 
>>> for
>>> anything, which requires even a tiny bit of performance.
>>
>> You do realize you actually benchmark against a function that 
>> does
>> nothing, right? Clearly there are ways in which we can improve
>> std.variant to the point initialization costs assignment of 
>> two words,
>> but this benchmark doesn't help. (Incidentally I just prepared 
>> a class
>> at C++ and Beyond on benchmarking, and this benchmark makes a 
>> lot of the
>> mistakes described therein...)
>>
>>
>> Andrei
>
>
> This should be more relevant then:
>
> //fib.d
> import std.datetime, std.stdio, std.variant;
>
> auto fib(Int)()
> {
> 	Int a = 1, b = 1;
> 	for(size_t i=0; i<100; i++){
> 		Int c = a + b;
> 		a = b;
> 		b = c;
> 	}
> 	return a;	
> }
>
> void main()
> {
> 	writeln(benchmark!(fib!int, fib!long, fib!Variant)(10_000));
> }
>
>
> dmd -O -inline -release fib.d
>
> Output:
>
> [TickDuration(197), TickDuration(276), TickDuration(93370107)]
>
> I'm horrified. Who was working on std.variant enhancements? 
> Please chime in.

You underestimate DMD's optimisations :-)

For int and long, DMD does the whole loop at compile time, so 
again you are benchmarking against an empty function. It's easy 
to see that this is the case by changing the size of the loop and 
noting that the int and long versions take the same amount of 
time.

Here's my results with optimisations turned *off*:

int: 2,304,868
long: 1,559,679
Variant: 2,667,320,252

Yes, it's not good to test without optimisations, but I think 
this gives a clearer picture of the relative differences between 
the two.

Still not great at ~1000x difference, but much better than a 
million :-)