Binary file grammar

wobbles via Digitalmars-d digitalmars-d at puremagic.com
Mon Aug 10 05:29:42 PDT 2015


I have to read a binary file.
I can use std.stdio.File.rawRead to do this (and it's even 
typesafe, awesome!!)

Currently, I'm doing this with a little helper function I've 
called Get:
     /// Creates a buffer to read the file. Handles a special case 
string
     T[] Get(T)(File f, size_t num=1){
	static if(is(T == char)){
	    ubyte[] strLen = new ubyte[1];
	    f.rawRead(strLen);
	    ubyte[] strBuf = new ubyte[strLen[0]];
	    return (f.rawRead(strBuf));
	} else {	
	    T[] buf = new T[num];
             return f.rawRead(buf);
	}
     }

Then to use it:
File f = File(filename, "rb");

auto versionNumber = f.Get!(ushort); // reads 2 bytes from file 
into a ushort[]
auto nextByte = f.Get!(byte); // reads 1 byte into a byte[]
auto next5Bytes = f.Get!(byte)(5); // reads 5 bytes and puts them 
into a byte[]

There is a lot of improvements to be made to the above, mostly 
around returning a Range. Also, reusing buffers so as not to 
allocate so much. That's probably Priority 1 actually.
That's all fine and dandy, will get to that.

There's some more complicated data in there too, like a string 
which in this binary file is defined as a byte representing the 
number of bytes in this string, followed by that number of bytes. 
(Similar to a char[] in C/D I imagine).

So, to read that, I'd write:
     auto myString = f.Get!(char)(f.Get!(byte)); // reads 
f.Get!byte number of bytes into a char array.

While doing this I had the idea of implementing a more general 
approach to this, using CTFE to build a struct / parserFunction 
(Similar to Pegged [1] ).
You describe at compile time how this binary file looks, and then 
the parser handles everything else.

Does anyone know of a good Binary Description Grammar out in the 
wild? I'd rather not re-invent the wheel on that front, and just 
use something standard. My Googling didn't come up with something 
that could be considered "standard" however.

Any ideas?

Thanks

[1] https://github.com/PhilippeSigaud/Pegged


More information about the Digitalmars-d mailing list