[DConf] Ali's talk: parallelism bug is caught by ThreadSanitizer
Johan
j at j.nl
Sun Nov 22 12:49:25 UTC 2020
Hi all, Ali,
Thanks Ali for your talk on DConf Online 2020, it was nice to
watch.
The std.parallelism.parallel example [1] immediately triggered
me. I had to stop the video, and see whether ThreadSanitizer
(TSan) shipped with LDC 1.24 can catch it. It does!
Testcase:
```
import std.parallelism;
void main() {
// defaultPoolThreads(4); // [2]
int[] elements = [0, 1, 2, 3];
int[] results;
foreach (e; elements.parallel) { // test.d:8
results ~= e; // test.d:9
}
}
```
Let's compile it with ThreadSanitizer enabled and run it!
```
❯ bin/ldc2 -g -fsanitize=thread -run test.d
==================
WARNING: ThreadSanitizer: data race (pid=19606)
Read of size 1 at 0x7f311b105010 by thread T7:
#0 memcpy <null> (libtsan.so.0+0x32505)
#1 _d_arrayappendcTX <null> (test-b40741e-06cb93+0x5fe76)
#2
_D3std11parallelism__T15ParallelForeachTAiZQv7opApplyMFMDFKiZiZ4doItMFZv phobos/std/parallelism.d-mixin-4048:4094 (test-b40741e-06cb93+0x2886e)
#3 _D3std11parallelism8TaskPool15executeWorkLoopMFZv <null>
(test-b40741e-06cb93+0x293af)
Previous write of size 4 at 0x7f311b105010 by thread T6:
#0 _D4test4mainFZ14__foreachbody1MFKiZi test.d:9
(test-b40741e-06cb93+0x28284)
#1
_D3std11parallelism__T15ParallelForeachTAiZQv7opApplyMFMDFKiZiZ4doItMFZv phobos/std/parallelism.d-mixin-4048:4094 (test-b40741e-06cb93+0x2886e)
#2 _D3std11parallelism8TaskPool15executeWorkLoopMFZv <null>
(test-b40741e-06cb93+0x293af)
Thread T7 (tid=19614, running) created by main thread at:
#0 pthread_create <null> (libtsan.so.0+0x2bcee)
#1 _D4core6thread8osthread6Thread5startMFNbZCQBoQBmQBiQBc
<null> (test-b40741e-06cb93+0x5508b)
#2 _Dmain test.d:8 (test-b40741e-06cb93+0x27f97)
#3 _D2rt6dmain212_d_run_main2UAAamPUQgZiZ6runAllMFZv <null>
(test-b40741e-06cb93+0x5aeeb)
#4 __libc_start_main <null> (libc.so.6+0x21b96)
Thread T6 (tid=19613, running) created by main thread at:
#0 pthread_create <null> (libtsan.so.0+0x2bcee)
#1 _D4core6thread8osthread6Thread5startMFNbZCQBoQBmQBiQBc
<null> (test-b40741e-06cb93+0x5508b)
#2 _Dmain test.d:8 (test-b40741e-06cb93+0x27f97)
#3 _D2rt6dmain212_d_run_main2UAAamPUQgZiZ6runAllMFZv <null>
(test-b40741e-06cb93+0x5aeeb)
#4 __libc_start_main <null> (libc.so.6+0x21b96)
SUMMARY: ThreadSanitizer: data race
(/usr/lib/x86_64-linux-gnu/libtsan.so.0+0x32505) in memcpy
==================
==================
WARNING: ThreadSanitizer: data race (pid=19606)
Read of size 1 at 0x7f311b105010 by thread T4:
#0 memcpy <null> (libtsan.so.0+0x32505)
#1 _d_arrayappendcTX <null> (test-b40741e-06cb93+0x5fe76)
#2
_D3std11parallelism__T15ParallelForeachTAiZQv7opApplyMFMDFKiZiZ4doItMFZv phobos/std/parallelism.d-mixin-4048:4094 (test-b40741e-06cb93+0x2886e)
#3 _D3std11parallelism8TaskPool15executeWorkLoopMFZv <null>
(test-b40741e-06cb93+0x293af)
Previous write of size 4 at 0x7f311b105010 by thread T6:
#0 _D4test4mainFZ14__foreachbody1MFKiZi test.d:9
(test-b40741e-06cb93+0x28284)
#1
_D3std11parallelism__T15ParallelForeachTAiZQv7opApplyMFMDFKiZiZ4doItMFZv phobos/std/parallelism.d-mixin-4048:4094 (test-b40741e-06cb93+0x2886e)
#2 _D3std11parallelism8TaskPool15executeWorkLoopMFZv <null>
(test-b40741e-06cb93+0x293af)
Thread T4 (tid=19611, running) created by main thread at:
#0 pthread_create <null> (libtsan.so.0+0x2bcee)
#1 _D4core6thread8osthread6Thread5startMFNbZCQBoQBmQBiQBc
<null> (test-b40741e-06cb93+0x5508b)
#2 _Dmain test.d:8 (test-b40741e-06cb93+0x27f97)
#3 _D2rt6dmain212_d_run_main2UAAamPUQgZiZ6runAllMFZv <null>
(test-b40741e-06cb93+0x5aeeb)
#4 __libc_start_main <null> (libc.so.6+0x21b96)
Thread T6 (tid=19613, running) created by main thread at:
#0 pthread_create <null> (libtsan.so.0+0x2bcee)
#1 _D4core6thread8osthread6Thread5startMFNbZCQBoQBmQBiQBc
<null> (test-b40741e-06cb93+0x5508b)
#2 _Dmain test.d:8 (test-b40741e-06cb93+0x27f97)
#3 _D2rt6dmain212_d_run_main2UAAamPUQgZiZ6runAllMFZv <null>
(test-b40741e-06cb93+0x5aeeb)
#4 __libc_start_main <null> (libc.so.6+0x21b96)
SUMMARY: ThreadSanitizer: data race
(/usr/lib/x86_64-linux-gnu/libtsan.so.0+0x32505) in memcpy
==================
ThreadSanitizer: reported 2 warnings
Error: /tmp/johan/test-b40741e-06cb93 failed with status: 66
```
I tested this using WSL2, the Linux environment available on
Windows 10. I think TSan should be supported on Windows and macOS
too, but I did not test this example.
Cheers,
Johan
[1]
https://www.youtube.com/watch?v=dRORNQIB2wA&feature=youtu.be&list=PLIldXzSkPUXWsvFA4AuawPoMnq9Bh1lIZ&t=1471
[2] Use this line of code if by default only a single thread is
available in the pool (e.g. in a container).
More information about the Digitalmars-d
mailing list