[DConf] Ali's talk: parallelism bug is caught by ThreadSanitizer

Johan j at j.nl
Sun Nov 22 12:49:25 UTC 2020


Hi all, Ali,
   Thanks Ali for your talk on DConf Online 2020, it was nice to 
watch.

The std.parallelism.parallel example [1] immediately triggered 
me. I had to stop the video, and see whether ThreadSanitizer 
(TSan) shipped with LDC 1.24 can catch it. It does!

Testcase:
```
import std.parallelism;

void main() {
     // defaultPoolThreads(4); // [2]

     int[] elements = [0, 1, 2, 3];
     int[] results;
     foreach (e; elements.parallel) {  // test.d:8
         results ~= e;                 // test.d:9
     }
}
```

Let's compile it with ThreadSanitizer enabled and run it!
```
❯ bin/ldc2 -g -fsanitize=thread -run test.d
==================
WARNING: ThreadSanitizer: data race (pid=19606)
   Read of size 1 at 0x7f311b105010 by thread T7:
     #0 memcpy <null> (libtsan.so.0+0x32505)
     #1 _d_arrayappendcTX <null> (test-b40741e-06cb93+0x5fe76)
     #2 
_D3std11parallelism__T15ParallelForeachTAiZQv7opApplyMFMDFKiZiZ4doItMFZv phobos/std/parallelism.d-mixin-4048:4094 (test-b40741e-06cb93+0x2886e)
     #3 _D3std11parallelism8TaskPool15executeWorkLoopMFZv <null> 
(test-b40741e-06cb93+0x293af)

   Previous write of size 4 at 0x7f311b105010 by thread T6:
     #0 _D4test4mainFZ14__foreachbody1MFKiZi test.d:9 
(test-b40741e-06cb93+0x28284)
     #1 
_D3std11parallelism__T15ParallelForeachTAiZQv7opApplyMFMDFKiZiZ4doItMFZv phobos/std/parallelism.d-mixin-4048:4094 (test-b40741e-06cb93+0x2886e)
     #2 _D3std11parallelism8TaskPool15executeWorkLoopMFZv <null> 
(test-b40741e-06cb93+0x293af)

   Thread T7 (tid=19614, running) created by main thread at:
     #0 pthread_create <null> (libtsan.so.0+0x2bcee)
     #1 _D4core6thread8osthread6Thread5startMFNbZCQBoQBmQBiQBc 
<null> (test-b40741e-06cb93+0x5508b)
     #2 _Dmain test.d:8 (test-b40741e-06cb93+0x27f97)
     #3 _D2rt6dmain212_d_run_main2UAAamPUQgZiZ6runAllMFZv <null> 
(test-b40741e-06cb93+0x5aeeb)
     #4 __libc_start_main <null> (libc.so.6+0x21b96)

   Thread T6 (tid=19613, running) created by main thread at:
     #0 pthread_create <null> (libtsan.so.0+0x2bcee)
     #1 _D4core6thread8osthread6Thread5startMFNbZCQBoQBmQBiQBc 
<null> (test-b40741e-06cb93+0x5508b)
     #2 _Dmain test.d:8 (test-b40741e-06cb93+0x27f97)
     #3 _D2rt6dmain212_d_run_main2UAAamPUQgZiZ6runAllMFZv <null> 
(test-b40741e-06cb93+0x5aeeb)
     #4 __libc_start_main <null> (libc.so.6+0x21b96)

SUMMARY: ThreadSanitizer: data race 
(/usr/lib/x86_64-linux-gnu/libtsan.so.0+0x32505) in memcpy
==================
==================
WARNING: ThreadSanitizer: data race (pid=19606)
   Read of size 1 at 0x7f311b105010 by thread T4:
     #0 memcpy <null> (libtsan.so.0+0x32505)
     #1 _d_arrayappendcTX <null> (test-b40741e-06cb93+0x5fe76)
     #2 
_D3std11parallelism__T15ParallelForeachTAiZQv7opApplyMFMDFKiZiZ4doItMFZv phobos/std/parallelism.d-mixin-4048:4094 (test-b40741e-06cb93+0x2886e)
     #3 _D3std11parallelism8TaskPool15executeWorkLoopMFZv <null> 
(test-b40741e-06cb93+0x293af)

   Previous write of size 4 at 0x7f311b105010 by thread T6:
     #0 _D4test4mainFZ14__foreachbody1MFKiZi test.d:9 
(test-b40741e-06cb93+0x28284)
     #1 
_D3std11parallelism__T15ParallelForeachTAiZQv7opApplyMFMDFKiZiZ4doItMFZv phobos/std/parallelism.d-mixin-4048:4094 (test-b40741e-06cb93+0x2886e)
     #2 _D3std11parallelism8TaskPool15executeWorkLoopMFZv <null> 
(test-b40741e-06cb93+0x293af)

   Thread T4 (tid=19611, running) created by main thread at:
     #0 pthread_create <null> (libtsan.so.0+0x2bcee)
     #1 _D4core6thread8osthread6Thread5startMFNbZCQBoQBmQBiQBc 
<null> (test-b40741e-06cb93+0x5508b)
     #2 _Dmain test.d:8 (test-b40741e-06cb93+0x27f97)
     #3 _D2rt6dmain212_d_run_main2UAAamPUQgZiZ6runAllMFZv <null> 
(test-b40741e-06cb93+0x5aeeb)
     #4 __libc_start_main <null> (libc.so.6+0x21b96)

   Thread T6 (tid=19613, running) created by main thread at:
     #0 pthread_create <null> (libtsan.so.0+0x2bcee)
     #1 _D4core6thread8osthread6Thread5startMFNbZCQBoQBmQBiQBc 
<null> (test-b40741e-06cb93+0x5508b)
     #2 _Dmain test.d:8 (test-b40741e-06cb93+0x27f97)
     #3 _D2rt6dmain212_d_run_main2UAAamPUQgZiZ6runAllMFZv <null> 
(test-b40741e-06cb93+0x5aeeb)
     #4 __libc_start_main <null> (libc.so.6+0x21b96)

SUMMARY: ThreadSanitizer: data race 
(/usr/lib/x86_64-linux-gnu/libtsan.so.0+0x32505) in memcpy
==================
ThreadSanitizer: reported 2 warnings
Error: /tmp/johan/test-b40741e-06cb93 failed with status: 66
```

I tested this using WSL2, the Linux environment available on 
Windows 10. I think TSan should be supported on Windows and macOS 
too, but I did not test this example.

Cheers,
   Johan


[1] 
https://www.youtube.com/watch?v=dRORNQIB2wA&feature=youtu.be&list=PLIldXzSkPUXWsvFA4AuawPoMnq9Bh1lIZ&t=1471
[2] Use this line of code if by default only a single thread is 
available in the pool (e.g. in a container).


More information about the Digitalmars-d mailing list