A program leaking memory.

[Thomas Kuehne]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Johan Grönqvist schrieb am 2006-03-13:

[snip]

> 2) Garbage collection
> ------------------------------------------
> My main loop calls step each time, and I pasted your code checking the
> state of the gc into my loop.
>
> First some (selected) lines:
> [3] pool:1048576 used:5072 free:19504 FREE:28 PAGE:3
> [24] pool:3997696 used:10224 free:18448 FREE:81 PAGE:12
> [38] pool:3997696 used:5200 free:19376 FREE:230 PAGE:10
> [66] pool:16121856 used:25280 free:15680 FREE:300 PAGE:49
> [77] pool:22675456 used:5200 free:19376 FREE:276 PAGE:71
>
> So far everything is fine. Memory usage seems to fluctuate, steadily
> upwards with each iteration, and dropping to ~5200 with irregular intervals.
>
> From here on, however, it never reaches down to the same levels again,
> and (restricting to 200 iterations) the end of the output is
>
> [199] pool:99352576 used:35744 free:29792 FREE:930 PAGE:315
> [200] pool:99680256 used:38272 free:27264 FREE:862 PAGE:317
>
> I would like it to do gc before my system starts swapping.
> My interpretation of the above is that it does gc to rarely.

This isn't the only possible cause. What happens if you call
fullCollect() after iteration 199?
(no fullCollect() calls from your code between iteration 1 and 199)

> After I insert a fullCollect() every iteration, my output ends with
>
> [193] pool:4653056 used:2736 free:17744 FREE:95 PAGE:14
> [194] pool:5308416 used:2704 free:17776 FREE:107 PAGE:16
> [195] pool:5963776 used:2736 free:17744 FREE:119 PAGE:18
> [196] pool:6619136 used:2720 free:17760 FREE:131 PAGE:20
> [197] pool:7274496 used:2736 free:17744 FREE:143 PAGE:22
> [198] pool:7929856 used:2736 free:17744 FREE:155 PAGE:24
> [199] pool:8585216 used:2736 free:17744 FREE:167 PAGE:26
> [200] pool:9240576 used:2736 free:17744 FREE:179 PAGE:28
> and I can see that this is much better.
>
> My next guess would be to include calls to the generational garbage
> collector at each iteration (or when memory exceeds a certain amount),
> but I do not know would be the best solution.

Let's try to make sure that the array is the cause.

1) add the imports
# import std.c.stdlib;
# import std.string;
# import std.gc;

2) replace

# real[] grad;
# grad.length = 1000;

with

# real[] grad;
# grad = (cast(real*) std.c.stdlib.calloc(1000, typeof(grad[0]).sizeof))[0 .. 1000];
# addRoot(grad.ptr);
	

3) add the following to the end of the function:

# memset(grad.ptr, 0, grad.length * typeof(grad[0]).sizeof);
# removeRoot(grad.ptr);
# std.c.stdlib.free(grad.ptr);

Even better would be if you could use single allocation of the array.
requires:
* the size doesn't change between repeated calles to "step"
* only on thread at a time executes "step"
* step isn't recursive

If too few collection calls are part of the problem, then 
phobos/internal/gc/gcx.d:mallocNoSync might be a starting point.

Thomas


-----BEGIN PGP SIGNATURE-----

iD8DBQFEFsSa3w+/yD4P9tIRAnoDAKCiOFgfji5xk0WxWJd1MkymFDh/bQCfZUzF
0/5IZCR6dL/f4uKHlw/7gug=
=L+71
-----END PGP SIGNATURE-----