csvReader read file byLine()?

Christophe Travert travert at phare.normalesup.org
Fri Jun 22 08:11:14 PDT 2012


Jens Mueller , dans le message (digitalmars.D:170448), a écrit :
> Jesse Phillips wrote:
>> On Friday, 22 June 2012 at 08:12:59 UTC, Jens Mueller wrote:
>> >The last line throws a CSVException due to some conversion error
>> >'Floating point conversion error for input "".' for the attached
>> >input.
>> >
>> >If you change the input to
>> >3.0
>> >4.0
>> >you get no exception but wrong a output of
>> >[[4], [4]]
>> >.
>> 
>> Yes, and it seems .joiner isn't as lazy as I'd have thought.
>> byLine() reuses its buffer so it will overwrite previous lines in
>> the file. This can be resolved by mapping a dup to it.
>> 
>> import std.stdio;
>> import std.algorithm;
>> import std.csv;
>> 
>> void main()
>> {
>>     struct Record {
>>         double one;
>>     }
>>     auto filename = "file.csv";
>>     auto file = File(filename, "r");
>>     auto input = map!(a => a.idup)(file.byLine()).joiner("\n");
>>     auto records = csvReader!Record(input);
>>     foreach(r; records)
>>     {
>>         writeln(r);
>>     }
>> }
> 
> Thanks. That works. But this should be either mentioned in the
> documentation or fixed. I would prefer a fix because the code above
> looks like a work around. Probably byLine or joiner then need some
> fixing. What do you think?
> 
> Jens

Yes, and that increases GC usage a lot.

Looking at the implementation, joiner as a behavior that is incompatible 
with ranges reusing some buffer:

joiner immidiately call's the range of range's popFront after having 
taken its front range. Instead, it should wait until it is necessary 
before calling popFront (at least until all the data has be read by the 
next tool of the chain).

Fixing this should not be very hard. Is there an issue preventing to 
make this change?

-- 
Christophe


More information about the Digitalmars-d mailing list