Is there any reasons to not use "mmap" to read files?

[rempas]

On Sunday, 6 February 2022 at 16:45:59 UTC, Ali Çehreli wrote:
> So big that they can't fit in memory. For example, I benefit 
> from mmap on a 16G system where a file would be 30G.

Oh, this small...

> As others said, it depends on the use case. If the entire file 
> will be read anyway especially in sequential order, then mmap 
> may not have much benefit. In my use case though it is common 
> to just read unknown small amounts of bytes from unknown places 
> of the huge file. (Say, 5G total out of a 30G.)
>
> Instead of my making multiple reads to those interesting parts 
> of the file, mmap handles everything transparently: Just mmap 
> the whole thing as a single array and access parts of that 
> memory as needed.

Thank you! I will have that in mind in case I want to do 
something like that in the future. In my use-case tho, I will 
read the whole file.

> One huge improvement is to add madvise(2) system call to the 
> picture to tell the system the exact amount of memory that will 
> be touched so the OS reads in a single shot. Otherwise, the 
> system reads by a default amount, which I think is 4K, which 
> can turn out to be pathetically slow e.g. when the file is 
> accessed over a slow network. (Why read 4K when the need is 
> just 200 bytes and why read in 4K steps when the need is 
> already to be 1M?)
>
> I haven't used mmap on Windows but it's in Phobos, so it should 
> work. After all, mmap uses the virtual memory system of the OS 
> and non-ancient Windows versions do use virtual memory and 
> std.mmfile does include 'version (windows)' sections; so, yes. 
> :)
>
> Ali

"mmap" is a system call that doesn't exist (natively) on Windows. 
I don't know what D does with Phobos (which I'm not gonna use 
anyway) but even if it works (how?), I will end up creating my 
own library so I'm in the same spot. "madvise" seems cool, I'll 
check it out! Thanks! In the end, I like advising and telling 
others how to do their work, XD!