Using an async buffer

[bioinfornatics]

On Friday, 8 November 2019 at 01:12:37 UTC, Ali Çehreli wrote:
> On 11/07/2019 07:07 AM, bioinfornatics wrote:
> > Dear,
> >
> > I try to use the async buffer describe into std.parallelism
> > documentation but my test code core dump!
>
> I admit I don't fully understand the lifetime issues but 
> removing the "code smell" of the modul-level File object solved 
> the issue for me, which requires four changes:
>
> // (1) Comment out the module-level variable
> // File file;
>
> // ...
>
> @system
> void next(File file, ref ubyte[] buf)
> {
>   // (2.a) Use the parameter 'file'
>   // (2.b) Adjust the length of the buffer
>   //       (rawRead may read less than the requested size)
>     buf = file.rawRead(buf);
> }
>
> // ...
>
>     // (3) Define a local variable
>     auto file = File(filePath, "rb");
>
> // ...
>
>     // (4) Use "callables" that use the local 'file':
>     auto asyncReader = taskPool.asyncBuf((ref ubyte[] buf) => 
> next(file, buf),
>                                          () => file.eof,
>                                          bufferSize);
>
> Ali

Thanks a lot Ali, the error message was understandable to me, and 
the way you fixed differ from the documentation. Does that means 
they are a bug ?

Moreover, this snippet is used to evaluate how to parse 
efficiently versus the wc -l command.

fixed snippet: 
https://paste.fedoraproject.org/paste/B~PjobIlVXIaJPfMjCcWSA

And I have 2 problems
1/ the executable is at least twice a time slower than wc -l
   I try to increase
     a) the buffer with a multiple of page size (parameter -n)
     b) the number of thread (parameter -t)
2/ the result is not exactly the same as wc -l give wich imply a 
bug while counting \n inside the buffer.

$ time ./build/wc_File_by_chunks_async_buffer -n 4 -t 8 -i 
test_100M
1638656 test_100M

real    0m0.106s
user    0m0.101s
sys     0m0.051s

$ time wc -l test_100M
1638400 test_100M

real    0m0.067s
user    0m0.030s
sys     0m0.037s


Thanks again Ali without you It was impossible to me