std.csv Performance Review

[Patrick Schluter via Digitalmars-d]

On Sunday, 4 June 2017 at 06:15:24 UTC, H. S. Teoh wrote:
> On Sun, Jun 04, 2017 at 05:41:10AM +0000, Jesse Phillips via 
> (Note that this is much less of a limitation than it seems; for 
> example you could use std.mmfile to memory-map the file into 
> your address space so that it doesn't actually have to fit into 
> memory, and you can still take slices of it. The OS will manage 
> the paging from/to disk for you. Of course, it will be slower 
> when something has to be paged from disk, but IME this is often 
> much faster than if you read the data into memory yourself.

If the file is in the file cache of the kernel, memory mapping 
does not need to reload the file as it is already in memory. In 
fact, calling mmap() changes only the sharing of the pages in 
general. That's where most of the performance win from memory 
mapping comes from.

This stackoverflow [1] discussion links to a realworldtech 
discussion with Linus Torvalds explaining it in detail. On 
windows and Solaris the mechanism is the same.

[1] 
https://stackoverflow.com/questions/5902629/mmap-msync-and-linux-process-termination/6219962#6219962