tolf and detab

[Jonathan M Davis]

On Monday, August 09, 2010 05:30:33 Michel Fortin wrote:
> On 2010-08-09 07:12:38 -0400, bearophile <bearophileHUGS at lycos.com> said:
> > Kagamin:
> >> Don't you minimize heap allocation etc by reading whole file in one io
> >> call?
> > 
> > The whole thread was about lazy read of file lines. If the file is very
> > large it's not wise to load it all in RAM at once.
> 
> For non-huge files that can fit in the memory space, I'd just
> memory-map the whole file and treat it as a giant string that I could
> then slice and keep the slices around (yeah!). The virtual memory
> system will take care of loading the file content's as you read from
> its memory space, so the file isn't loaded all at once.
> 
> But that's not compatible with the C file IO functions. Does Python
> uses C file IO calls when reading from a file? If not, perhaps that's
> why it's faster.

Well, you can just read the whole file in as a string with readText(), and any 
slices to that could stick around, but presumably, that's using the C file I/O 
calls underneath.

- Jonathan M Davis