Reading a structured binary file?

[captaindet]

On 2013-08-02 17:13, Jonathan M Davis wrote:
> On Friday, August 02, 2013 19:49:54 Gary Willoughby wrote:
>> What library commands do i use to read from a structured binary
>> file? I want to read the byte stream 1, 2 maybe 4 bytes at a time
>> and cast these to bytes, shorts and ints respectively. I can't
>> seem to find anything like readByte().
>
> I'd probably use std.mmfile and std.bitmanip to do it. MmFile will allow you to
> efficiently operate on the file as a ubyte[] in memory thanks to mmap, and
> std.bitmanip's peek and read functions make it easy to convert multiple bytes
> into integral values.
>
> - Jonathan M Davis

FWIW
i have to deal with big data files that can be a few GB. for some data analysis software i wrote in C a while back i did some testing with caching and such. turns out that for Win7-64 the automatic caching done by the OS is really good and any attempt to speed things up actually slowed it down. no kidding, i have seen more than 2GB of data being automatically cached. of course the system RAM must be larger than the file size (if i remember my tests correctly by a factor of ~2, but this is maybe not a linear relationship, i did not actually change the RAM just the size of the data file) and it will hold it in the cache only as long as there are no concurrent applications requiring RAM or caching. i guess my point is, if your target is Win7 and your files are >5x smaller than the installed RAM i would not bother at all trying to optimize file access. i suppose -nix machine will do a similar good job these days.

/det