std.container.Array/RefCounted(T) leaking memory?

[Steven Schveighoffer]

On Wed, 12 Jan 2011 19:29:30 -0500, Jonathan M Davis <jmdavisProg at gmx.com>  
wrote:

> On Wednesday, January 12, 2011 15:29:51 %u wrote:
>> Sorry to bump this up, but is RefCounted(T) really leaking, or am I  
>> missing
>> something? I would like to use this in my program, and I'm curious as to
>> why no one responded, since if it's actually leaking, it would be an
>> important issue.
>
> There probably aren't all that many people who saw your post, and out of  
> those
> who did, there are probably very few - if any - who have actually done  
> much with
> RefCounted. It's fairly new.
>
> There's at least one major bug on Array at the moment (
> http://d.puremagic.com/issues/show_bug.cgi?id=4942 ). There are also  
> several
> bugs having to do with destructors at the moment which could be causing  
> you
> problems.
>
> Now, even assuming that you're not seeing any problem with a  
> destructor-related
> bug and that you're not hitting a known bug with Array, there are three  
> things
> that you need to be aware of which would likely show high memory usage
> regardless:
>
> 1. Array uses an array internally, and there is some caching that goes  
> on with
> regards to arrays that has to do with appending. This means that if  
> you're
> dealing with large arrays, you could have several which haven't been  
> garbage
> collected yet simply because they're cached. Steven Schveighoffer has  
> talked
> about it in several posts, and he has done some work to improve the  
> situation,
> but I'm not sure that any of it has been in a release yet.

No, there is no release yet, but the code is checked into svn.  But Array  
doesn't use D appending anyways.

>
> 2. The garbage collector does not currently run in its own thread. IIUC,  
> it only
> gets run when you try and allocate memory. So, if you allocate a bunch of
> memory, and then you never try and allocate memory again, no memory will  
> be
> collected, regardless of whether it's currently being used or not.

>
> 3. As I understand it, the current garbage collector _never_ gives  
> memory back
> to the OS. It will reclaim memory that you're not referencing any longer  
> so that
> it doesn't necessarily need to go grab more memory from the OS when you  
> try and
> allocate something, but once the garbage collector has gotten a block of  
> memory
> from the OS, it doesn't give it back. So, currently you will _never_ see  
> the
> memory usage of a D program go down, unless you're explicitly using  
> malloc and
> free instead of GC-allocated memory.

Um... Array acutally uses malloc and free to allocate its data.

But even so, malloc and free have the same property where they don't  
always give back memory to the OS.  IIUC, Linux can only change the size  
of memory it wants, it cannot free pages in the middle of the block.

-Steve