std.experimental.allocator API and nearly useless StatsCollector
Luis
Luis.panadero at gmail.com
Sun Jun 20 17:58:41 UTC 2021
On Sunday, 20 June 2021 at 16:53:06 UTC, Basile B. wrote:
> On Sunday, 20 June 2021 at 15:49:13 UTC, Luis wrote:
>> Well, this last week I was playing using malloc/free to do
>> some @nogc code, and using the std.experimental.allocator API.
>> Also, I was trying to see a way to profile my code and search
>> for memory leaks, etc...
>>
>> [...]
>>
>> The stats that are reported by reportPerCallStatistics, shows
>> the file & line number of the allocator!
>>
>> Like this :
>>
>> ```
>> /usr/include/dmd/phobos/std/experimental/allocator/package.d(1585): [numAllocate:1, numAllocateOK:1, bytesAllocated:40]
>> ```
>>
>> This could be useful for someone writing his own allocator,
>> but it's pretty useless for anyone trying to track where a
>> memory leak originated in his own code!
>
>
> To understand leaks I'd use valgrind instead. As it is based on
> DWARF info, reported leaking blocks are accompanied with a
> pretty (after demangling) back trace. So the first entry for
> each block will be unsurprinsingly always the same (e.g glibc
> malloc) but the other can be more interesting.
Yeah. I know Valgrind. I used it a lot, when I did C or C++
stuff, and it's a wonderful tool. However have his problems :
1. Makes anything to run painfull slow.
2. At least with DMD, the mangled information & stack traces are
hard to follow.
Compare this two outputs when I comment the code that calls to
allocator.dispose():
My simple & dirty profiler :
```
$ ./bin/ddiv-test-unittest -t 1 -i "Stack"
✓ ddiv.container.stack SimpleStack
Summary: 1 passed, 0 failed in 0 ms
WARNING! Possible memory leaks!
----- Memory allocation information -----
Total amount of allocated memory that has not been released:
65536 bytes
Addr: 0x55C70F4D0E60 - [int[]] 65536 bytes -
source/ddiv/container/stack.d:84
```
Valgrind (fun thing. I need to use --leak-check=full to see the
memory leak of my code, at at 0x483DFAF realloc. The rest looks
that it's from Silly and/or D runtime):
```
$ valgrind --leak-check=full ./bin/ddiv-test-unittest -t 1 -i
"Stack"
==185906== Memcheck, a memory error detector
==185906== Copyright (C) 2002-2017, and GNU GPL'd, by Julian
Seward et al.
==185906== Using Valgrind-3.15.0 and LibVEX; rerun with -h for
copyright info
==185906== Command: ./bin/ddiv-test-unittest -t 1 -i Stack
==185906==
--185906-- WARNING: Serious error when reading debug info
--185906-- When reading debug info from
/home/luis/repos/dlang/ddiv/bin/ddiv-test-unittest:
--185906-- DWARF line info appears to be corrupt - the section is
too small
--185906-- WARNING: Serious error when reading debug info
--185906-- When reading debug info from
/home/luis/repos/dlang/ddiv/bin/ddiv-test-unittest:
--185906-- read_filename_table: .debug_line is missing?
✓ ddiv.container.stack SimpleStack
Summary: 1 passed, 0 failed in 89 ms
==185906== Conditional jump or move depends on uninitialised
value(s)
==185906== at 0x44B2C8:
_D4core8internal2gc4impl12conservativeQw3Gcx__T4markVbi0Vbi0ZQoMFNbNlSQCqQCoQCiQCiQCgQCrQBw__T9ScanRangeVbi0ZQpZv (in /home/luis/repos/dlang/ddiv/bin/ddiv-test-unittest)
==185906== by 0x445469:
_D4core8internal2gc4impl12conservativeQw3Gcx16markConservativeMFNbNlPvQcZv (in /home/luis/repos/dlang/ddiv/bin/ddiv-test-unittest)
==185906== by 0x44BFC7:
_D4core8internal2gc4impl12conservativeQw3Gcx__T7markAllS_DQCeQCcQBwQBwQBuQCfQBk16markConservativeMFNbNlPvQcZvZQClMFNbbZ14__foreachbody3MFNbKSQFjQEy11gcinterface5RangeZi (in /home/luis/repos/dlang/ddiv/bin/ddiv-test-unittest)
==185906== by 0x44FFB3:
_D4core8internal9container5treap__T5TreapTSQBp2gc11gcinterface5RangeZQBi7opApplyMFNbMDFNbKQBwZiZ9__lambda2MFNbKxSQEhQCsQCsQCiZi (in /home/luis/repos/dlang/ddiv/bin/ddiv-test-unittest)
==185906== by 0x450359:
_D4core8internal9container5treap__T5TreapTSQBp2gc11gcinterface5RangeZQBi13opApplyHelperFNbxPSQDnQDlQDfQCy__TQCvTQCsZQDd4NodeMDFNbKxSQFaQDlQDlQDbZiZi (in /home/luis/repos/dlang/ddiv/bin/ddiv-test-unittest)
==185906== by 0x44FF81:
_D4core8internal9container5treap__T5TreapTSQBp2gc11gcinterface5RangeZQBi7opApplyMFNbMDFNbKQBwZiZi (in /home/luis/repos/dlang/ddiv/bin/ddiv-test-unittest)
==185906== by 0x44BF60:
_D4core8internal2gc4impl12conservativeQw3Gcx__T7markAllS_DQCeQCcQBwQBwQBuQCfQBk16markConservativeMFNbNlPvQcZvZQClMFNbbZv (in /home/luis/repos/dlang/ddiv/bin/ddiv-test-unittest)
==185906== by 0x446752:
_D4core8internal2gc4impl12conservativeQw3Gcx11fullcollectMFNbbZm
(in /home/luis/repos/dlang/ddiv/bin/ddiv-test-unittest)
==185906== by 0x44B017:
_D4core8internal2gc4impl12conservativeQw14ConservativeGC__T9runLockedS_DQCsQCqQCkQCkQCiQCtQBy18fullCollectNoStackMFNbZ2goFNbPSQEuQEsQEmQEmQEkQEv3GcxZmTQBbZQDsMFNbKQBnZm (in /home/luis/repos/dlang/ddiv/bin/ddiv-test-unittest)
==185906== by 0x443397:
_D4core8internal2gc4impl12conservativeQw14ConservativeGC18fullCollectNoStackMFNbZv (in /home/luis/repos/dlang/ddiv/bin/ddiv-test-unittest)
==185906== by 0x44332D:
_D4core8internal2gc4impl12conservativeQw14ConservativeGC14collectNoStackMFNbZv (in /home/luis/repos/dlang/ddiv/bin/ddiv-test-unittest)
==185906== by 0x4177B6: gc_term (in
/home/luis/repos/dlang/ddiv/bin/ddiv-test-unittest)
==185906==
==185906==
==185906== HEAP SUMMARY:
==185906== in use at exit: 66,568 bytes in 6 blocks
==185906== total heap usage: 454 allocs, 448 frees, 346,252
bytes allocated
==185906==
==185906== 32 bytes in 1 blocks are possibly lost in loss record
2 of 6
==185906== at 0x483B7F3: malloc (in
/usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==185906== by 0x441335:
_D4core8internal2gc4impl12conservativeQw10initializeFZCQCbQBq11gcinterface2GC (in /home/luis/repos/dlang/ddiv/bin/ddiv-test-unittest)
==185906== by 0x43F608:
_D4core2gc8registry16createGCInstanceFAyaZCQBpQBn11gcinterface2GC
(in /home/luis/repos/dlang/ddiv/bin/ddiv-test-unittest)
==185906== by 0x43A2CA: gc_init_nothrow (in
/home/luis/repos/dlang/ddiv/bin/ddiv-test-unittest)
==185906== by 0x417C93:
_D4core8internal2gc4impl5protoQo7ProtoGC6qallocMFNbmkxC8TypeInfoZSQCm6memory8BlkInfo_ (in /home/luis/repos/dlang/ddiv/bin/ddiv-test-unittest)
==185906== by 0x3DD216: gc_qalloc (in
/home/luis/repos/dlang/ddiv/bin/ddiv-test-unittest)
==185906== by 0x3E33FF: _d_newitemT (in
/home/luis/repos/dlang/ddiv/bin/ddiv-test-unittest)
==185906== by 0x41F66B:
_D3std5array__T8AppenderTAAyaZQp6__ctorMFNaNbNcNeQyZSQBzQBy__TQBvTQBpZQCd (in /home/luis/repos/dlang/ddiv/bin/ddiv-test-unittest)
==185906== by 0x41F564:
_D3std5array__T8appenderTAAyaZQpFNaNbNfZSQBnQBm__T8AppenderTQBjZQo (in /home/luis/repos/dlang/ddiv/bin/ddiv-test-unittest)
==185906== by 0x41F4E8:
_D3std5array__TQjTSQr9algorithm9iteration__T8splitterVAyaa6_61203d3d2062VEQCu8typecons__T4FlagVQBpa14_6b656570536570617261746f7273ZQBqi0TQDfTQDjZQDxFQDrQDuZ6ResultZQGcFNaNbNfQGaZAQEv (in /home/luis/repos/dlang/ddiv/bin/ddiv-test-unittest)
==185906== by 0x41F4CD:
_D3std5array__T5splitTAyaTQeZQoFNaNbNfQqQsZAQw (in
/home/luis/repos/dlang/ddiv/bin/ddiv-test-unittest)
==185906== by 0x3EC601:
_D3std6getopt11splitAndGetFNaNbNeAyaZSQBkQBj6Option (in
/home/luis/repos/dlang/ddiv/bin/ddiv-test-unittest)
==185906==
==185906== 65,536 bytes in 1 blocks are definitely lost in loss
record 6 of 6
==185906== at 0x483DFAF: realloc (in
/usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==185906== by 0x4206AA:
_D4core6memory__T11pureReallocZQoFNaNbNiPvmZQe (in
/home/luis/repos/dlang/ddiv/bin/ddiv-test-unittest)
==185906== by 0x3FB799:
_D3std12experimental9allocator10mallocator10Mallocator10reallocateMOFNaNbNiKAvmZb (in /home/luis/repos/dlang/ddiv/bin/ddiv-test-unittest)
==185906== by 0x3A611B:
_D3std12experimental9allocator__T11expandArrayTiTOSQBxQBwQBl10mallocator10MallocatorZQCaFNaNbNiKOQBvKAimZb (package.d:2143)
==185906== by 0x3A6024:
_D4ddiv4core6memory9allocator__T11expandArrayTiTOS3std12experimentalQBx10mallocator10MallocatorZQCmFNbNiKOQCfKAimAyaiZb (allocator.d:178)
==185906== by 0x3A04A7:
_D4ddiv9container5stack__T11SimpleStackTiTS3std12experimental9allocator10mallocator10MallocatorZQCs6expandMFNbNiNemZv (stack.d:84)
==185906== by 0x3A0578:
_D4ddiv9container5stack__T11SimpleStackTiTS3std12experimental9allocator10mallocator10MallocatorZQCs4pushMFNbNiNfiZv (stack.d:105)
==185906== by 0x39FAE9:
_D4ddiv9container5stack18__unittest_L142_C1FNfZv (stack.d:166)
==185906== by 0x3BD004:
_D5silly11executeTestFSQv4TestZSQBe10TestResult (silly.d:189)
==185906== by 0x3BC89B:
_D5silly24_sharedStaticCtor_L24_C1FZ9__lambda1FZ15__foreachbody15MFKSQCp4TestZi (silly.d:109)
==185906== by 0x3C2614:
_D3std11parallelism__T14doSizeZeroCaseTAS5silly4TestTDFKQqZiZQBnFKSQCnQCm__T15ParallelForeachTQCdZQwQBvZi (parallelism.d:3748)
==185906== by 0x3C2013:
_D3std11parallelism__T15ParallelForeachTAS5silly4TestZQBg7opApplyMFMDFKQBeZiZi (parallelism.d-mixin-4039:4043)
==185906==
==185906== LEAK SUMMARY:
==185906== definitely lost: 65,536 bytes in 1 blocks
==185906== indirectly lost: 0 bytes in 0 blocks
==185906== possibly lost: 32 bytes in 1 blocks
==185906== still reachable: 1,000 bytes in 4 blocks
==185906== suppressed: 0 bytes in 0 blocks
==185906== Reachable blocks (those to which a pointer was found)
are not shown.
==185906== To see them, rerun with: --leak-check=full
--show-leak-kinds=all
==185906==
==185906== Use --track-origins=yes to see where uninitialised
values come from
==185906== For lists of detected and suppressed errors, rerun
with: -s
==185906== ERROR SUMMARY: 3 errors from 3 contexts (suppressed: 0
from 0)
```
More information about the Digitalmars-d
mailing list