Reading a structured binary file?

[H. S. Teoh]

On Sat, Aug 03, 2013 at 02:25:23PM -0700, Jonathan M Davis wrote:
> On Saturday, August 03, 2013 23:10:12 John Colvin wrote:
> > On Saturday, 3 August 2013 at 18:23:58 UTC, Gary Willoughby wrote:
> > > On Saturday, 3 August 2013 at 18:14:47 UTC, Gary Willoughby
> > > 
> > > wrote:
> > >> This sounds a great idea but once the file has been opened as
> > >> a MmFile how to i convert this to a ubyte[] so the
> > >> std.bitmanip functions work with it?
> > > 
> > > I'm currently doing this:
> > > 	auto file = new MmFile("file.dat");
> > > 	ubyte[] buffer = cast(ubyte[])file[];
> > > 	buffer.read!uint(); //etc.
> > > 
> > > Is this how you would recommend?
> > 
> > That defeats the object of memory mapping, as the [] at the end
> > of cast(ubyte[])file[] implies copying the whole file in to
> > memory.
> 
> Are you sure about that? Maybe I'm just not familiar enough with mmap,
> but I don't see anything in MmFile which would result in it copying
> the whole file into memory. I guess that I'll have to do some more
> reading up on mmap.  Certainly, if slicing it like that copies it all
> into memory, that's a big problem.
[...]

I think he meant that the OS will have to load the entire file into
memory if you sliced the mmap'ed file, not that you'll copy all the
data.

I'm not certain this is true, though, because slicing as I understand it
only returns the address of the start of the mmap'ed addresses coupled
with its length. I don't think the OS will actually load anything into
memory until you reference an address within that mmap'ed range. And
even then, only those disk blocks that correspond with the referenced
addresses will actually be loaded -- this is the point of virtual
memory, after all.


T

-- 
The computer is only a tool. Unfortunately, so is the user. -- Armaphine, K5