Improving IO Speed

[Craig Dillabaugh]

On Friday, 14 March 2014 at 18:00:58 UTC, TJB wrote:
> I have a program in C++ that I am translating to D as a way to 
> investigate and learn D. The program is used to process 
> potentially hundreds of TB's of financial transactions data so 
> it is crucial that it be performant. Right now the C++ version 
> is orders of magnitude faster.
>
> Here is a simple example of what I am doing in D:
>
> import std.stdio : writefln;
> import std.stream;
>
> align(1) struct TaqIdx
> {
>   align(1) char[10] symbol;
>   align(1) int tdate;
>   align(1) int begrec;
>   align(1) int endrec;
> }
>
> void main()
> {
>   auto input = new File("T201212A.IDX");
>   TaqIdx tmp;
>   int count;
>
>   while(!input.eof())
>   {
>     input.readExact(&tmp, TaqIdx.sizeof);
>    // Do something with the data
>   }
> }
>
> Do you have any suggestions for improving the speed in this 
> situation?
>
> Thank you!
>
> TJB

I am not sure how std.stream buffers data (the library has been 
marked for removal, so perhaps not very efficiently), but what 
happens if you read in a large array of your TaqIdx structs with 
each read.