IAP Tools for D

[Jakob Jenkov via Digitalmars-d-announce]

> How does the performance of ION compare with Protocol Buffers 
> (https://developers.google.com/protocol-buffers/?hl=en) and 
> Apache Thrift ( https://thrift.apache.org/)?

That depends on what API you use, and how much "meta data" (e.g. 
class names and property names) you write in the serialized ION 
data. ION is quite flexible about how much meta you want to 
include.

If you remove property names and rely only the sequence of 
fields, ION can write faster than Google Protocol Buffers. When 
reading, if you only rely in the sequence of fields, ION is a bit 
slower than Google Protocol Buffers. All in all I believe 
performance will be on-par with Google Protocol Buffers.

We have some benchmarks here:

http://tutorials.jenkov.com/iap/ion-performance-benchmarks.html

We still have a few minor optimizations to do, and more 
benchmarks to run, but perhaps also some validations to add etc, 
so the benchmarks on this page (for Java) are probably not too 
far off from the final numbers.

Regarding Apache Avro and Thrift, I looked at them today. It 
seems that Avro's encoding is similar to ION (and MessagePack and 
CBOR), although without e.g. tables. According to Thrift's own 
docs their binary encoding is not compact. For compact encoding 
it seems they refer to Protobuf.

ION has several advantages over Protobuf as a general purpose 
data format. ION is self describing, so you can iterate it 
without a schema. This means that you can do pretty fast 
arbitrary hierarchical navigation of an ION "file/message".

Protobuf's own docs say that Protobuf is not good for large 
amounts of raw bytes (e.g. files). ION is capable of modeling 
both raw binary data (e.g. files), JSON, XML and CSV efficiently. 
You could even convert ION to a restricted XML format, edit it in 
a text editor, and convert it back to ION (we have not 
implemented this yet, but we have planned it). We also believe 
that ION can support cyclic object graphs, but this is also not 
fully implemented and tested yet.

ION has a very compact encoding of arrays of objects in "Tables" 
which are similar to CSV files with only 1 header row, and N 
value rows. It is very common to transport arrays of object over 
the network, e.g. N search results from a service. Thus ION 
tables are a major advantage. Tables can also be used inside 
object graphs where an object has 0..N children (in an array).

We have a comparison of ION to other data formats here:

http://tutorials.jenkov.com/iap/ion-vs-other-formats.html