How to read fastly files ( I/O operation)

[FG]

On 2013-02-07 00:41, Lee Braiden wrote:
> I wasn't going to mention this as I thought the CPU usage might be trivial, but
> if both CPU and IO are factors, then it would probably be beneficial to have a
> separate IO thread/task.

This wasn't an issue in my version of the program. It took 1m55s to process the
file, but then again it takes 1m44s just to read it (as shown previously).

> Also, if you don't strictly need to parse the file in order, then you could
> divide and conquer it by breaking it into more sections/tasks. For example, if
> you're parsing records, you cold split the file in half, find the remaining
> parts of the record in the second half, move it to the first, and then process
> the two halves in two threads.  If you've a nice function to do that split
> cleanly, and n cpus, then just call it some more.

Now, this could make a big difference!
If only parsing out of order is acceptable in this case.