Code review: JSON unmarshaller

[Jacob Carlborg]

On 2012-10-15 22:35, Tyler Jameson Little wrote:

> I'm basically trying to reproduce other JSON marshallers, like Go's, but
> using compile-time reflection. Go uses runtime reflection, which D
> notably does not support. I like the idea of compile-time reflection
> better anyway. There are a few things that would make it easier (like a
> __traits call like allMembers that excludes functions).

Most other languages are not as complicated as D, it's basically only C 
and C++ that are. Implementing a marshaller in Ruby would be dead 
simple. No pointers, no array slices (in the same way as D), support for 
full runtime reflection.

> I use a lot of JSON, so a JSON marshaller/unmarshaller is going to save
> a lot of time, and make my code a lot cleaner.

Most of these points are when unmarshalling. I haven't actually looked 
if your marshaller can handle these cases but looking at the small 
amount of code I would guess no.

>> * Pointers
>
> I've done this, but haven't fully tested it. Basic pointers work.

Are they correctly setup when unmarshaling. Example:

int a = 3; // global/TLS

class Foo
{
     int b = 4;
     int* c;
     int* d;
}

auto foo = new Foo;
foo.c = &a;
foo.d = &foo.b;

When unmarshaling will "foo.d" point to "foo.b"?

>> * Array slices
>
> I think this is handled.

This is basically the same as pointers:

class Foo
{
     int[] a;
     int[] b;
}

auto foo = new Foo;
foo.a = [3, 4, 5, 6];
foo.b = foo.a[1 .. 3];

When unmarshaling will "foo.b" point to "foo.a"?

>> * Serializing through base class references
>
> Doesn't __traits(allMembers, T) give everything from all super classes?

__traits only work at compile time.

class A
{
     int a;
}

class B : A
{
     int b;
}

A b = new B;

The static type of "b" is "A" so all information about "B" is lost at 
compile time. You either need to provide a way to register all 
subclasses that should be be marshaled through a base class reference or 
you need to implement proper runtime reflection.

>> * const/immutable fields
>
> Hmm, not sure to handle this. These have to be set in the constructor,
> right?

You shouldn't call the constructor when unmarshaling. That's another 
problem. Do you want to limit your marshaller to only work with classes 
that have a default constructor or none.

You need to create the class instances without calling the constructor. 
Then you could provide a method that will be called before/after 
unmarshaling.

Have a look that this post:

http://www.digitalmars.com/d/archives/digitalmars/D/Deserializing_const_fields_175774.html

>> * Any reference type (not really hard but it's more work)
>
> Are you talking about aliases? What other kind of reference types are
> there in structs/classes? I'm assuming this will have more to do with
> marshalling as opposed to unmarshalling.

Yes, you don't want to marshal the same object twice. References types 
in D are: objects, pointers, associative arrays and arrays. These are 
the ones I can think of for now.

>> Have a look at for a basically fully working serialization library
>> Orange:
>>
>> https://github.com/jacob-carlborg/orange
>
> Hmm, looks interesting. This looks like it only supports XML, which I
> don't use, but I'm sure you've already solved a lot of the corner cases.
> Thanks, I'll take a look!

I have solved a lot of corner cases but there are a few left. I have a 
branch for handling const/immutable fields but it needs more testing 
before merging it with the master branch.

I'm also not really happy about the deserializing of arrays. It's quite 
slow.

Apparently it's also breaks as soon as you turn on some kind of 
optimization when compiling.

The goal of Orange was to be able serialize basically everything found 
in D. Also to support multiple archive types, i.e. XML, JSON, binary and 
so on.

-- 
/Jacob Carlborg