Speed of csvReader
wobbles via Digitalmars-d-learn
digitalmars-d-learn at puremagic.com
Thu Jan 21 08:01:33 PST 2016
On Thursday, 21 January 2016 at 15:17:08 UTC, data pulverizer
wrote:
> On Thursday, 21 January 2016 at 14:56:13 UTC, Saurabh Das wrote:
>> On Thursday, 21 January 2016 at 14:32:52 UTC, Saurabh Das
>> wrote:
>>> [...]
>>
>> Actually since you're aiming for speed, this might be better:
>>
>> sw.start();
>> auto records =
>> File("Acquisition_2009Q2.txt").byChunk(1024*1024).joiner.map!(a => cast(dchar)a).csvReader!row_type('|').array
>> sw.stop();
>>
>> Please do verify that the end result is the same - I'm not
>> 100% confident of the cast.
>>
>> Thanks,
>> Saurabh
>
> @Saurabh I have tried your latest suggestion and the time
> reduces fractionally to:
>
> Time (s): 6.345
>
> the previous suggestion actually increased the time
>
> @Edwin van Leeuwen The csvReader is what takes the most time,
> the readText takes 0.229 s
Interesting that reading a file is so slow.
Your timings from R, is that including reading the file also?
More information about the Digitalmars-d-learn
mailing list