Getting underlying struct for parseJSON

Alexey H via Digitalmars-d-learn digitalmars-d-learn at puremagic.com
Tue Feb 28 13:07:27 PST 2017


On Tuesday, 28 February 2017 at 20:48:33 UTC, Petar Kirov 
[ZombineDev] wrote:
> On Tuesday, 28 February 2017 at 20:27:25 UTC, Alexey H wrote:
>> Hello, guys!
>>
>> I'm working on a project that involves parsing of huge JSON 
>> datasets in real-time.
>> Just an example of what i'm dealing with is here:
>>
>> https://gist.githubusercontent.com/gdmka/125014058bb7d7f01b867fac56300a61/raw/f0c6b5be5fb01b16dd83f07c577b72f76f72c855/data.json
>>
>> Can't think of any tools other that D or Go to solve this 
>> problem.
>>
>> My experience of solving the problem with Go has led me to 
>> Stackoverflow and the community out there seemed too reluctant 
>> to help so i assumed that the language cannot handle such a 
>> set of operations on complex datastructures.
>>
>> My experience with D was like a charm. Where i have had ~100 
>> lines of Go code i did the same  with 12 in with D. But, 
>> nevertheless, i did some profiling (unfortunately on OS X) and 
>> saw much heavier CPU usage with D than Go. Probably because 
>> the Go solution was unpacking all the data strictly to struct.
>>
>> So, my real question is: can i actually, by any change, get 
>> the description of an underlying struct that the call to 
>> parseJSON generates?
>>
>> The goers have this thing https://mholt.github.io/json-to-go/ 
>> to generate structs from JSON automatically.
>>
>> Since D easily parses JSON by type inference, i assume it 
>> builds a JSONValue struct which holds all the fields and the 
>> data.
>>
>> If it is possible, then i can build a similar JSON to D tool 
>> just for the sake of saving people's time and patience.
>
> If you really care about performance, have a look this: 
> http://forum.dlang.org/post/20151014090114.60780ad6@marco-toshiba
>
> std.json is not tuned for performance, so don't expect good 
> results from it.

I am not expecting good results from stdlib's json. As for now i 
just need a concise way to get 1.2-1.5 MB JSON and dump it into a 
struct to perform numeric computations. Since it's not the only 
data source i will be parsing, i need a straightforward way to 
generate D structs right out of the predefined JSON schema. So i 
am willing to sacrifice some speed for convenience at this point.

Fastjson might be good when dealing with trusted input, but this 
is not my case.



More information about the Digitalmars-d-learn mailing list