[phobos] For review: Improvements to std.variant and std.json

[Masahiro Nakagawa]

On Sat, 19 Feb 2011 06:57:00 +0900, Robert Jacques <sandford at jhu.edu>  
wrote:

> On Sun, 13 Feb 2011 13:50:19 -0500, Lutger Blijdestijn  
> <lutger.blijdestijn at gmail.com> wrote:
>> Hi, I just wanted to comment on the dson file format.
>>
>> While perhaps less efficient, the mongodb nosql database also uses a  
>> binary
>> json (bson) format to store data. It implements some extra types  
>> though. I
>> suggest that if you want to implement binary json, to seriously consider
>> adopting bson instead:
>>
>> http://bsonspec.org/
>>
>> <http://bsonspec.org/>
>
> Hi, I'm aware of the BSON format and have given it a decent  
> review/reading (along with a few other serialization formats). As was  
> noted in the JSON documentation BSON doesn't support all JSON types and  
> has serious size and performance issues. First and foremost, BSON  
> doesn't support true arrays; "arrays" are actually maps with integer  
> keys. And it doesn't support 80-bit reals, (which will probably be a  
> deficiency of all non-D specific formats.) Add in all the  
> legacy/proprietary stuff and the in-memory/C design optimization, and  
> it's really not attractive as a stand alone serial format. Also, Mongodb  
> has a 4mb Document limit, which IIRC is assumed by many of the BSON  
> implementations to be the BSON size limit as well.

You are right. BSON is not a JSON. BSON purposes traversable and editable  
format.
So, BSON is less efficient than other binary formats.
If you try to implement MongoDB-like database, BSON is better.

>> 4mb

Currently, 16mb.

> My main purpose behind a binary json format was to store/read a data  
> table, which was originally quite small and now is quite large, in a  
> more efficient manner. Long term, I'm thinking of dropping this  
> functionality in favor of a customized D binary serial format.

I think original binary format is not necessary.
When user uses JSON, object size and performance are not important.
Adding halfway implementation increases maintenance cost. It's a demerit.


Masahiro