memory-mapped files

[BCS]

Hello Lionello,

>> The memory-mapped version takes 2.15 seconds on average. I was
>> fighting against Perl's equivalent 2.45. At some point I decided to
>> try without memory mapping and I consistently got 1.75 seconds. What
>> the heck is going on? When does memory mapping actually help?
>> 
> Random seeking in large files :)
> 
> Sequential read can't possibly gain anything by using MM because
> that's what the OS will end up doing, but MM is using the paging
> system, which has some overhead (a page fault has quite a penalty, or
> so I've heard.)

paging is going to be built to move date in the fastest possible way so it 
would be expected that using MM would be fast. The only thing I see getting 
in the way would be 1) it uses up lots of address space and 2) you might 
be able to lump reads or hint to the OS to pre load when you load the file 
other ways. 

It would be neat to see what happens if you MM a file and force page faults 
on the whole thing right up front (IIRC the is an asm op that forces a page 
fault but doesn't wait for it). Even better might be to force a page fault 
for N pages ahead of where you are processing.