Performance of std.json

[Masahiro Nakagawa via Digitalmars-d]

On Monday, 2 June 2014 at 00:18:19 UTC, David Soria Parra wrote:
> Hi,
>
> I have recently had to deal with large amounts of JSON data in 
> D. While doing that I've found that std.json is remarkable slow 
> in comparison to other languages standard json implementation. 
> I've create a small and simple benchmark parsing a local copy 
> of a github API call 
> "https://api.github.com/repos/D-Programming-Language/dmd/pulls" 
> and parsing it 100% times and writing the title to stdout.
>
> My results as follows:
>   ./d-test > /dev/null  3.54s user 0.02s system 99% cpu 3.560 
> total
>   ./hs-test > /dev/null  0.02s user 0.00s system 93% cpu 0.023 
> total
>   python test.py > /dev/null  0.77s user 0.02s system 99% cpu 
> 0.792 total
>
> The concrete implementations (sorry for my terrible haskell 
> implementation) can be found here:
>
>    https://github.com/dsp/D-Json-Tests/
>
> This is comapring D's std.json vs Haskells Data.Aeson and 
> python standard library json. I am a bit concerned with the 
> current state of our JSON parser given that a lot of 
> applications these day use JSON. I personally consider a high 
> speed implementation of JSON a critical part of a standard 
> library.
>
> Would it make sense to start thinking about using ujson4c as an 
> external library, or maybe come up with a better 
> implementation. I know Orvid has something and might add some 
> analysis as to why std.json is slow. Any ideas or pointers as 
> to how to start with that?

I don't know the status of another D based JSON library.
If you can install yajl library, then yajl-d is an another 
candidate.

% time ./yajl_test > /dev/null
./yajl_test > /dev/null  0.42s user 0.01s system 99% cpu 0.434 
total
% time python test.py> /dev/null
python test.py > /dev/null  0.65s user 0.02s system 99% cpu 0.671 
total
% time ./test > /dev/null
./test > /dev/null  3.10s user 0.02s system 99% cpu 3.125 total

import yajl.yajl, std.datetime, std.file, std.stdio;

void parse() {
     foreach(elem; readText("test.json").decode.array) {
         writeln(elem.object["title"]);
     }
}
int main(string[] args) {
     for(uint i = 0; i < 100; i++) {
         parse();
     }
     return 0;
}

http://code.dlang.org/packages/yajl

NOTE: yajl-d doesn't expose yajl's SAX style API unlike Sean's 
implementation