Speed of csvReader

data pulverizer via Digitalmars-d-learn digitalmars-d-learn at puremagic.com
Thu Jan 21 10:31:17 PST 2016


On Thursday, 21 January 2016 at 17:17:52 UTC, Saurabh Das wrote:
> On Thursday, 21 January 2016 at 17:10:39 UTC, data pulverizer 
> wrote:
>> On Thursday, 21 January 2016 at 16:01:33 UTC, wobbles wrote:
>>> Interesting that reading a file is so slow.
>>>
>>> Your timings from R, is that including reading the file also?
>>
>> Yes, its just insane isn't it?
>
> It is insane. Earlier in the thread we were tackling the wrong 
> problem clearly. Hence the adage, "measure first" :-/.
>
> As suggested by Edwin van Leeuwen, can you give us a timing of:
> auto records = File("Acquisition_2009Q2.txt", 
> "r").byLine.map!(a => a.split("|").array).array;
>
> Thanks,
> Saurabh

Good news and bad new. I was going for something similar to what 
you have above and both slash the time alot:

Time (s): 1.024

But now the output is a little garbled. For some reason the 
splitter isn't splitting correctly - or we are not applying it 
properly. Line 0:

["100001703051", "RETAIL", "BANK OF AMERICA, 
N.A.|4.875|207000|3", "0", "03/200", "|05", "2009|75", "75|1|26", 
"80", "|N", "|", "O ", "ASH", "OU", " REFINANCE|PUD|1|INVE", 
"TOR", "C", "|801||FRM", "\n\n", "863", "", "FRM"]


More information about the Digitalmars-d-learn mailing list