is it a bug or what?

[Steven Schveighoffer via Digitalmars-d]

On 1/22/15 9:40 AM, collerblade wrote:
> On Thursday, 22 January 2015 at 14:19:10 UTC, Steven Schveighoffer wrote:
>> On 1/22/15 8:44 AM, collerblade wrote:
>>> Dear D user, i have this code:
>>>
>>> import core.sys.windows.windows;
>>>
>>> real[] a;
>>> while(1) {
>>>   a.length=4096*4096;
>>>   a=null;
>>>
>>>   Sleep(2000);
>>> }
>>>
>>> It allocates memory, but its never gets freed, just keep going up, and
>>> after 10 secs, memoryexception is thrown. I checked the pointer it is GC
>>> safe (attributes is 10). Am i missing something?
>>
>> Are you compiling 64-bit code or 32-bit? It makes a huge difference.
>>
>> A real is going to be 16-byte aligned. This means real[4096*4096] is
>> allocating 1/4GB per allocation. This consumes 1/16th of your total
>> address space on 32-bit address space.
>>
>> The GC in D is conservative. This means that on 32-bit architecture,
>> any 4-byte segment scanned by the GC (whether it's a pointer or not)
>> that happens to point into the array you allocated is going to keep
>> that memory pinned. Because any 4-byte garbage has a 1:16 chance of
>> pointing there, odds are good that this will happen. Then it keeps
>> happening, you eventually run out of memory.
>>
>> Now, some possible explanation of why GC.malloc might work while this
>> does not:
>>
>> GC.malloc allocates exactly 4096*4096*16 bytes. Setting a.length goes
>> through the append code. Because extending length is considered an
>> appending event, it *over-allocates* by a certain factor, expecting
>> you to keep appending. (this is how appending can be amortized O(1)
>> performance). This means the amount of memory allocated is larger than
>> 1/4GB. Therefore the chance of failing by setting a.length is going to
>> be higher than with using GC.malloc directly, but it could fail with
>> either case.
>>
>> Answers?
>>
>> 1. Manually free such a large array when done with it.
>> 2. Do not allocate such large arrays.
>> 3. Use 64-bit code generation (-m64 on DMD windows, even if you are on
>> Win64) if possible.
>>
>> -Steve
>
> TY for your answer.

Before I respond, first question to answer is if you are using 32-bit 
compiled code. If not, then it's not typical for what you are doing to 
cause problems.

> I checked malloc and it is the same. It runs out if memory too.
> Also tried with int[]->result is the same.
> Memory is initialized to 0-s, so there is no pointer pointing to memory.
> At least at my side. Are u saiying, that most likely other GC allocated
> memory points to my arrays? Hm thats maybe true. If thats a case i have
> to manage memory myself.

Most likely it's a stack variable or global/TLS variable. Those are 
scanned too :)

Also note that reals are NOT initialized to 0, but NaN.

> Also what about scpoed alloc?
>
> scoped int[] a=new int[whatever_big_size];
>
> It sould free memory but it does not :( :(.

I would not use this, I think it's scheduled for deprecation, or at 
least the meaning is scheduled to change.

Note that a must go out of scope pointing at the array that was 
allocated in order to free the memory. You can't set it to null, because 
then nothing would be freed!

-Steve