File.byLine for either Windows / Unix newlines

[Steven Schveighoffer]

On 12/11/17 10:40 AM, Dennis wrote:
> I'm on Windows and I recently got confused by how Phobos functions 
> handle newlines.
> 
> ```
> void main() {
>      import std.stdio;
>      import std.path : buildPath, tempDir;
> 
>      auto path = buildPath(tempDir(), "test.txt");
>      auto file = new File(path, "w");
> 
>      file.write("hello there!\n");   //actually writes hello there!\r\n
>      file.write('\n');               //actually writes \r\n
>      file.rawWrite("\n");            //actually writes \n
>      //file.rawWrite('\n');          //doesn't compile
>      file.close();
> 
>      //byLine uses \n as default terminator, so a trailing \r is kept
>      writeln(File(path).byLine.front~"huh?");   //prints "huh?o there!"
> }
> ```
> 
> - Why do these functions behave like that?
> - What is the easiest way to iterate over a file by line that works with 
> both \r\n and \n line endings? I prefer not to write a new method for this.

Please note that D uses FILE * as its underlying implementation. This 
means that whatever the C library does, D's library does.

In the case of Windows, there is this lovely concept that line endings 
should be \r\n, not just \n. So the C library helpfully inserts \r 
whenever it sees a \n. This is known as "text mode" as opposed to 
"binary mode".

rawWrite temporarily sets the mode to binary mode, then restores the 
original mode. This is why the \n is not translated there.

Here's the fun part: the default open mode is "rb". See the docs here: 
https://dlang.org/phobos/std_stdio.html#.File.this

So you are actually opening the file in binary mode, whereas you were 
writing it in text mode.

You can fix this in one of 3 ways:
1. Open your file for writing with "wb". This will not store the \r by 
default.
2. Open your file for reading with "r" (e.g. File(path, "r")). This will 
replace the \r\n with just \n in the underlying C library.
3. Stop using Windows ;)

-Steve