Getting underlying struct for parseJSON
Alexey H via Digitalmars-d-learn
digitalmars-d-learn at puremagic.com
Tue Feb 28 13:07:27 PST 2017
On Tuesday, 28 February 2017 at 20:48:33 UTC, Petar Kirov
[ZombineDev] wrote:
> On Tuesday, 28 February 2017 at 20:27:25 UTC, Alexey H wrote:
>> Hello, guys!
>>
>> I'm working on a project that involves parsing of huge JSON
>> datasets in real-time.
>> Just an example of what i'm dealing with is here:
>>
>> https://gist.githubusercontent.com/gdmka/125014058bb7d7f01b867fac56300a61/raw/f0c6b5be5fb01b16dd83f07c577b72f76f72c855/data.json
>>
>> Can't think of any tools other that D or Go to solve this
>> problem.
>>
>> My experience of solving the problem with Go has led me to
>> Stackoverflow and the community out there seemed too reluctant
>> to help so i assumed that the language cannot handle such a
>> set of operations on complex datastructures.
>>
>> My experience with D was like a charm. Where i have had ~100
>> lines of Go code i did the same with 12 in with D. But,
>> nevertheless, i did some profiling (unfortunately on OS X) and
>> saw much heavier CPU usage with D than Go. Probably because
>> the Go solution was unpacking all the data strictly to struct.
>>
>> So, my real question is: can i actually, by any change, get
>> the description of an underlying struct that the call to
>> parseJSON generates?
>>
>> The goers have this thing https://mholt.github.io/json-to-go/
>> to generate structs from JSON automatically.
>>
>> Since D easily parses JSON by type inference, i assume it
>> builds a JSONValue struct which holds all the fields and the
>> data.
>>
>> If it is possible, then i can build a similar JSON to D tool
>> just for the sake of saving people's time and patience.
>
> If you really care about performance, have a look this:
> http://forum.dlang.org/post/20151014090114.60780ad6@marco-toshiba
>
> std.json is not tuned for performance, so don't expect good
> results from it.
I am not expecting good results from stdlib's json. As for now i
just need a concise way to get 1.2-1.5 MB JSON and dump it into a
struct to perform numeric computations. Since it's not the only
data source i will be parsing, i need a straightforward way to
generate D structs right out of the predefined JSON schema. So i
am willing to sacrifice some speed for convenience at this point.
Fastjson might be good when dealing with trusted input, but this
is not my case.
More information about the Digitalmars-d-learn
mailing list