Working with files over 2GB in D2

[Jeremie Pelletier]

Frank Benoit wrote:
> dsimcha schrieb:
>> == Quote from Jeremie Pelletier (jeremiep at gmail.com)'s article
>>> dsimcha wrote:
>>>> Does anyone know how to work with huge (2GB+) files in D2?  std.stream has
>>>> overflow bugs (I haven't isolated them yet) and can't return their size
>>>> correctly, std.stdio.File throws a ConvOverflowError in seek() because fseek()
>>>> apparently takes an int when it should take a long, and std.file only supports
>>>> reading the whole file, which I can't do in 2GB address space.
>>>>
>>>> It appears none of the file I/O on Phobos has been tested on huge files (until
>>>> now).
>>> What platform are you using? You should report your issue on bugzilla.
>>> I had similar issues on windows when using stdio's fseek and ftell, I
>>> had no problems using GetFilePointerEx, you could try that while it is
>>> fixed.
>>> Jeremie
>> Mostly Linux.  Everything seems to be working on Windows, though I haven't tested
>> it that thoroughly.  I will file Bugzillas eventually, but I'm still trying to
>> understand some of these issues, i.e. to what extent they're limitations vs. real
>> bugs.
>>
>> What I'm really interested in knowing is:
>>
>> 1.  To what extent is the fact that working with 2GB+ files a platform limitation
>> rather than a real bug?  (I vaguely understand that it has to do with files being
>> indexed by signed ints, but I don't know the details of how it's implemented on
>> each platform and what is different between platforms.)
>>
>> 2.  Does anyone know of a method of doing file I/O in D2 that is well-tested with
>> files above 2GB?
> 
> Tango has full support for that. On linux platform, there are two C
> APIs, one up to 2GB and one for LFS - Large File Support.
> 

I just had a quick peek at std.stdio, it is using the C standard library 
for file I/O on every platform.

Phobos should support the CreateFile related APIs on windows and LFS on 
linux to get around quirks like that 2Gb limitation.

Jeremie