Humble benchmark (fisher's exact test)

[russhy]

On Monday, 23 August 2021 at 13:12:21 UTC, bachmeier wrote:
> On Saturday, 14 August 2021 at 14:08:05 UTC, Ki Rill wrote:
>
>>> If you really want performance, you can determine which case 
>>> applies to your code and then make the underlying `.Call` 
>>> yourself. If you don't do that, you're comparing Fisher's 
>>> exact test against a routine that does a lot more than 
>>> Fisher's exact test. In any event, you're not comparing 
>>> against an R implementation of this test.
>>
>> That is the point of this benchmark, to test it against 
>> Python/R implementation irrespective of what it does 
>> additionally. And to test compiled languages in general.
>
> That might have been the point of your benchmark, but that 
> doesn't mean the benchmark is meaningful, in this case for at 
> least three reasons:
>
> 1. You're measuring the performance of completely different 
> tasks in R and C, where the R task is much bigger.
> 2. What you've done is only one way to use R. Anyone that 
> wanted performance would use .Call rather than what you're 
> doing.
> 3. R has a JIT compiler, and you're likely not making use of it.
>
> The comparison against R is not what you're after anyway. If 
> you don't want to do it in a way that's meaningful - and that's 
> perfectly understandable - it's best to delete it.

JIT isn't something you want if you need fast execution time

And nobody gonna warm JIT 1000000 times to call a task, you want 
result immediately

That's why languages like R/C#/Java sucks and cheat at benchmark, 
they are only reliable if the programs is calling the same code 
100000000 times, wich never happen, except under heavy load, wich 
also almost never happen for most use cases other than webdev; 
and even then you have crappy execution time because of cold 
startup

This benchmarks even mention it:

> It's a simple benchmark examining:
> * execution time (sec)
> * memory consumption (kb)
> * binary size (kb)
> * conciseness of a programming language (lines of code)