How to free memory allocated via double[][] using dmd-2.0.12?

[Bill Baxter]

Markus Dittrich wrote:
> Hi,
> 
> For a data processing application I need to read a large number
> of data sets from disk. Due to their size, they have to be read and 
> processed sequentially, i.e. in pseudocode
> 
> int main()
> {
>     while (some condition)
>     {
>          double[][] myLargeDataset = read_data();
>          process_data(myLargeDataset);
>          // free all memory here otherwise next cycle will 
>          // run out of memory
>      }
>     
>    return 0;
> }
> 
> Now, the "problem" is the fact that each single data-set saturates
> system memory and hence I need to make sure that all memory
> is freed after each process_data step is complete. Unfortunately,
> using dmd-2.012 I have not been able to achieve this. Whatever
> I do (including nothing, i.e., letting the GC do its job), the resulting 
> binary keeps accumulating memory and crashing shortly after). 
> I've tried deleting the array, setting the array lengths to 0, and manually
> forcing the GC to collect, to no avail. Hence, is there something I am 
> doing terribly wrong or is this a bug in dmd?

Markus,  you do not show us what either read_data or process_data do. 
It is possible that one of those is somehow holding on to references to 
the data.  This would prevent the GC from collecting the memory.

Another problem is if you allocate the memory initially as void[] then 
the GC will scan it for pointers, and in a big float buffer you'll get a 
lot of false hits.  To prevent that, allocate the buffer initially as 
byte[] (or double[] --- just not void).

Anyway, if you want a speedy fix, you'll need to distill this down into 
something that is actually reproducible by Walter.

--bb