Profiling the memory in D

[kerdemdemir]

On Wednesday, 4 December 2019 at 15:38:36 UTC, Steven 
Schveighoffer wrote:
> On 12/4/19 3:10 AM, Erdem wrote:
>> I am used to have cool tools like valgrid massif to visualize 
>> the memory usage from C++ but in D it seems I am blind folded 
>> looking for the problem.
>> 
>> Until now I tried:
>> 
>> --vgc option which show million things which makes it not 
>> useful
>> --build=profile-gc
>> which seems to slow down my program like *200 times and I have 
>> some operation depending on time which makes creating the real 
>> case of the application impossible. The observation itself 
>> effect the experiment very badly.
>> 
>> Since this options are not useful for me, I tried to put debug 
>> logs in critical places to estimate where I am losing memory. 
>> I couldn't find a D native call to get the memory usage of the 
>> program. I had to call a shell command which also slows down 
>> the app. This slow down causes me to not be able to put this 
>> debug log too many places or in critical loops.
>> 
>> I manage to narrow it down a bit. Than I wanted to check the 
>> object instances to find out which objects are getting bigger. 
>> I tried
>> GC.sizeOf(&object) which always prints 0. GC.sizeOf also does 
>> not works with associative arrays , I don't see any usefull 
>> case than GC.sizeOf(array.ptr).
>> 
>> That is the background of my questions and my questions are:
>> 
>> Is there a better way to visualize the memory in D something 
>> like valgrid massif?
>> 
>> profile-gc literally makes to program not do anything due to 
>> slow down is there any way to profile-gc in a faster fashion?
>> 
>> Is there a native function which will return the memory usage 
>> of my program?
>
> If it's total GC memory only you are interested in, then try 
> the D runtime switch for the GC: --DRT-gcopt=profile:1
>
> This will print out a summary of GC usage at the end of your 
> program, and shouldn't significantly affect runtime.
>
>> 
>> Is there a function to return sizes of AAs and even class 
>> instances ?
>> 
>
> Class instances have a compiler-defined size. Try 
> __traits(classInstanceSize, SomeClass)
>
> This is the compile-time size of that specific type. If you 
> have a class instance, and you want the actual class size, in 
> the case of a derived instance, you may retrieve that using 
> typeid(classInstance).initializer.length.
>
> Note also that this is not necessarily the size consumed from 
> the GC! If you want *that* size, you need to use 
> GC.sizeof(cast(void*)object) (you were almost correct, what you 
> did was get the GC size of the *class reference* which is 
> really a pointer, and really lives on the stack, hence the 0).
>
> -Steve

GC.sizeof(cast(void*)object) will be super useful. I will use 
that.

I also tried GC: --DRT-gcopt=profile:1 already. It provides so 
little information.
I need to find out which member AA or array of which object is 
causing this memory problem of mine.I am ending up around 2GB of 
ram usage in a single day.

Is there any way to manipulate profile-gc flag on run time? Like 
I will start my program without it somehow and after the my 
program initializes I will turn it on.

One last thing in my program I am getting a message from vibe 
sometimes like
"leaking eventcore driver because there are still active 
handles". I use websockets and do web requests. I wanted to add 
that because I saw you fixed something with Redis about that in 
https://github.com/vibe-d/vibe.d/issues/2245.

Thanks for your help and replies Steve.