What are the best available D (not C) File input/output options?

[Julian Fondren]

On Thursday, 2 November 2023 at 15:46:23 UTC, confuzzled wrote:
> I've ported a small script from C to D. The original C version 
> takes roughly 6.5 minutes to parse a 12G file while the port 
> originally took about 48 minutes. My naïve attempt to improve 
> the situation pushed it over an hour and 15 minutes. However, 
> replacing std.stdio:File with core.stdc.stdio:FILE* and 
> changing my output code in this latest version from:
>
> 	outputFile.writefln("%c\t%u\t%u\t%d.%09u\t%c", ...)
>
> to
> 	fprintf(outputFile, "%c,%u,%u,%llu.%09llu,%c\n", ...)
>
> reduced the processing time to roughly 7.5 minutes. Why is 
> File.writefln() so appallingly slow? Is there a better D 
> alternative?

First, strace your program. The slowest thing about I/O is the 
syscall itself. If the D program does more syscalls, it's going 
to be slower almost no matter what else is going on. Both D and C 
are using libc to buffer I/O to reduce syscalls, but you might be 
defeating that by constantly flushing the buffer.

>
> I tried std.io but write() only outputs ubyte[] while I'm 
> trying to output text so I abandoned idea early.

string -> immutable(ubyte)[]: alias with 
std.string.representation(st)

'alias' meaning, this doesn't allocate. If gives you a byte slice 
of the same memory the string is using.

You'd still need to do the formatting, before writing.

> Now that I've got the program execution time within an 
> acceptable range, I tried replacing core.stdc.fread() with 
> std.io.read() but that increased the time to 24 minutes. Now 
> I'm starting to think there is something seriously wrong with 
> my understanding of how to use D correctly because there's no 
> way D's input/output capabilities can suck so bad in comparison 
> to C's.