Debugging memory leak.
Bill Baxter
dnewsgroup at billbaxter.com
Tue Oct 9 12:19:15 PDT 2007
Bill Baxter wrote:
> Frits van Bommel wrote:
>> David Brown wrote:
>>> On Mon, Oct 08, 2007 at 10:23:17PM -0700, David Brown wrote:
>>>
>>>> I seem to have something else in my program that is making lots of
>>>> stray
>>>> pointers for the GC to follow. My program consistently leaks memory
>>>> until
>>>> it is killed by running out of address space (or I kill it because
>>>> it is
>>>> trying to swap my machine to death).
>>>
>>> I think I found my problem. I have something like this:
>>>
>>> class Foo {
>>> ubyte[20] hash;
>>> ...
>>> }
>>>
>>> and a _lot_ of these are alive at any given time. The hash is a sha1
>>> hash,
>>> and so tends to be evenly distributed.
>>>
>>> It appears that heap objects only have a single flag indicating
>>> whether or
>>> not they have pointers, and since classes do have pointers, the GC will
>>
>> Yeah, it's just a single bit currently. The vtable and monitor
>> pointers don't count though (the first points to static memory and the
>> second to malloc()ed (non-gc) memory), so class bodies only gets
>> tagged as containing pointers if it has explicit
>> pointer/reference/dyn. array/assoc.array members.
>>
>>> look at the "pointers" in the hash to see if they hit anything.
>>> Build up a
>>> reasonable set of these and something will point to almost everything.
>>>
>>> It's going to take me a little while to change the code to not do
>>> this (it
>>> isn't quite as simple as my example). I was concerned about having 20
>>> bytes living in its own gc allocation being wasteful, but leaking nearly
>>> everything is much more wasteful :-)
>>
>> A precise GC (or at least something more in that direction) would
>> really help in these kinds of situations...
>
> I think there's another big potential source of leaks: Associative Arrays.
>
> A node in an AA has two pointers and a hash value, so the gc will treat
> all those hash values as pointers too. I think that's probably what's
> killing me, because I use a lot of AA's.
>
> --bb
Here's a simple test case:
module memunion;
import std.stdio;
import std.gc;
// Just an ordinary AA with a lot of values.
class BigAA
{
int[int] aa;
this() {
for(int i=0;i<1000;i++) {
aa[i] = i;
}
}
}
void main()
{
int nloops = 10_000;
auto b = new BigAA[100];
for(int i=0; i<nloops; ++i)
{
// Create some AAs (overwriting old ones)
foreach(ref v; b) { v = new BigAA; }
// See how we're doing
std.gc.GCStats stats;
std.gc.fullCollect();
std.gc.getStats(stats);
writefln("Loop %-5s - poolsize=%-10s %s Mbytes (%s KB)",
i, stats.poolsize,
stats.usedsize/1024/1024,
stats.usedsize/1024);
}
}
With phobos, mem usage goes up pretty steadily.
--bb
More information about the Digitalmars-d
mailing list