Compiler benchmarker for D, C, C++, Go, Rust with more to come

[Jacob Carlborg]

On 2019-03-16 22:23, Per Nordlöw wrote:
> After reading
> 
> https://forum.dlang.org/thread/nlhybmioaoxkydicmtjj@forum.dlang.org
> 
> it's about time I announce my little Python snippet that exercises the 
> compilation speed of D and other languages with a superficial test 
> program generator at
> 
> https://github.com/nordlow/compiler-benchmark
> 
> Run as
> 
>     ./benchmark.py
> 
> This generates a single source file for each supported language, 
> currently D, C, C++, Go and Rust, containing 50k number of trivial 
> functions that from the main function are called and reduced to a single 
> integer which is returned at the end of main. The benchmark then 
> measures the time it takes for each compiler to run semantic analysis on 
> the source file (excluding output of any object files).
> 
> My only question is:
> How is it possible for Rust be to 50x slower than D in this test!!!??? 
> Am I calling rustc in the wrong way? Especially taking into account the 
> amount of time (and money) that has been put into making Rust fast(er). 
> Through fancy caching of IR-files and more.
> 
> I very much welcome PRs that add more languages.
> 
> Output on Ubuntu 18.04 system is
> 
> Code-generation:
> Generating generated/c/foo.c took 0.073 seconds (C)
> Generating generated/c++/foo.c++ took 0.073 seconds (C++)
> Generating generated/d/foo.d took 0.076 seconds (D)
> Generating generated/rust/foo.rs took 0.076 seconds (Rust)
> Generating generated/zig/foo.zig took 0.077 seconds (Zig)
> Generating generated/go/foo.go took 0.074 seconds (Go)
> 
> Clang:
> - Checking of generated/c/foo.c took 1.373 seconds (using 
> "/usr/bin/clang-7")
> - Checking of generated/c++/foo.c++ took 1.345 seconds (using 
> "/usr/bin/clang++-7")
> 
> GCC:
> - Checking of generated/c/foo.c took 0.385 seconds (using "/usr/bin/gcc-5")
> - Checking of generated/c++/foo.c++ took 0.675 seconds (using 
> "/usr/bin/g++-5")
> - Checking of generated/c/foo.c took 0.551 seconds (using "/usr/bin/gcc-6")
> - Checking of generated/c/foo.c took 0.518 seconds (using "/usr/bin/gcc-7")
> - Checking of generated/c++/foo.c++ took 1.294 seconds (using 
> "/usr/bin/g++-7")
> - Checking of generated/c/foo.c took 0.527 seconds (using "/usr/bin/gcc-8")
> - Checking of generated/c++/foo.c++ took 1.363 seconds (using 
> "/usr/bin/g++-8")
> 
> D:
> - Checking of generated/d/foo.d took 0.403 seconds (using "/usr/bin/dmd")
> - Checking of generated/d/foo.d took 0.352 seconds (using 
> "/home/per/.local/ldc2-1.15.0-beta1-linux-x86_64/bin/ldmd2")
> 
> Go:
> - Checking of generated/go/foo.go took 1.716 seconds (using 
> "/usr/bin/gccgo")
> 
> Rust:
> - Checking of generated/rust/foo.rs took 17.828 seconds (using 
> "/home/per/.cargo/bin/rustc")
> 
> Zig:
> Speedups:
> - D to gcc-8: 1.50
> - D to g++-8: 3.88
> - D to Clang: 3.90
> - D to Clang++: 3.83
> - D to Go: 4.88
> - D to Rust: 50.71

It would be interesting to measure D's speed with DMD compiled with LDC 
as the host D compiler.

-- 
/Jacob Carlborg