Speed of csvReader

[Saurabh Das via Digitalmars-d-learn]

On Thursday, 21 January 2016 at 14:32:52 UTC, Saurabh Das wrote:
> On Thursday, 21 January 2016 at 13:42:11 UTC, Edwin van Leeuwen 
> wrote:
>> On Thursday, 21 January 2016 at 09:39:30 UTC, data pulverizer 
>> wrote:
>>>   StopWatch sw;
>>>   sw.start();
>>>   auto buffer = std.file.readText("Acquisition_2009Q2.txt");
>>>   auto records = csvReader!row_type(buffer, '|').array;
>>>   sw.stop();
>>
>>
>> Is it csvReader or readText that is slow? i.e. could you move 
>> sw.start(); one line down (after the readText command) and see 
>> how long just the csvReader part takes?
>
> Please try this:
>
> auto records = 
> File("Acquisition_2009Q2.txt").byLine.joiner("\n").csvReader!row_type('|').array;
>
> Can you put up some sample data and share the number of records 
> in the file as well.

Actually since you're aiming for speed, this might be better:

sw.start();
auto records = 
File("Acquisition_2009Q2.txt").byChunk(1024*1024).joiner.map!(a 
=> cast(dchar)a).csvReader!row_type('|').array
sw.stop();

Please do verify that the end result is the same - I'm not 100% 
confident of the cast.

Thanks,
Saurabh