[phobos] CSVRange: RFC

Jesse Phillips jesse.k.phillips at gmail.com
Mon Jan 31 22:15:36 PST 2011


Ok, so to prevent theoretical debate on performance I decided to get a
CSV file large enough to do testing. I didn't come across any with a
quick Google search, so I wrote a program to generate one[1], just
needs a list of words. Though the file is hard coded for
english.words[2]

1. https://gist.github.com/805392
2. ftp://nic.funet.fi/pub/unix/security/dictionaries/DEC-collection/

Anyway looping over a 25MB file took about 10 sec while David's was
only about 5 sec. With some digging around I found that a Record did
not modify the RecordList data, storing a ref parameter is of no use.
A quick attempt to use pointers failed. To check my hypotheses I
modified[3] the code to not use Record when given a struct. The result
was about half a second faster than David's. Interestingly,
optimizations will slow mine down by one second and do nothing to
David's.

I don't really know the best way to test memory footprint.

3. https://github.com/he-the-great/JPDLibs/tree/csvoptimize

Also my implementation accepts a heading now.


More information about the phobos mailing list