"The total size of a static array cannot exceed 16Mb."

[Christopher Wright]

Vladimir Panteleev wrote:
> On Tue, 02 Oct 2007 09:20:29 +0300, Vladimir Panteleev <thecybershadow at gmail.com> wrote:
> 
>> Why?
>>
>> I posted a bug report yesterday, thinking it was just some accidentally left over code.
>> But I noticed this statement in the documentation today. And, frankly, I don't understand why that limitation is there.
>>
>> The problem with this restriction comes when using multi-dimensional arrays. Branched dynamic arrays are MUCH slower than rectangular/cubic/etc. static arrays - and this is a severe limitation for anyone wanting to work with large amounts of data (e.g. game maps). Of course I could work around this by calculating the positions manually using x*width+y, but I'd like to know the reason why this limitation was put there in the first place, and what's the reasoning behind it. No other compiled language that I used (C/C++/Delphi) has such a limitation.
>>
>> The check is in mtype.c, line 1835 (DMD 1.021).
> 
> I think I found the cause of this limitation - OPTLINK hangs, crashes and/or generates corrupt data if I try to force it such input. It works if I use a pointer to a huge static array, but DMD wouldn't accept that as input anyway. I thought that putting the array in a struct or class would prevent the linker from generating a >16MB data segment, but I forgot that D needs(!) to have an initial value for every type, even if it's completely null. Once again, I'll mention that Borland and Microsoft's linkers don't have such a limitation, and they can generate data segments over 16 MB with no problems.
> 
> So, suggestions for fixing this:
> 
> 1) fix or rewrite OPTLINK
>    ("rewrite" because I keep hearing about how OPTLINK holds D back for one reason or another; I know it's no small task, however D doesn't need a multi-purpose linker, just one that can handle what the DMD compiler generates)
> 2) if a type doesn't contain any non-0 initial values, instead of generating a big block of zeroes for the initializer, initialize it with a memset (rep stosd/stosb).
> 

T[][] SquareArray (T) (int x, int y) {
    T[] block = new T[x * y];
    T[][] ret = [];
    ret.length = x;
    for (int i = 0; i < x; i++) {
       ret[i] = block[i * y .. (i + 1) * y];
    }

    return ret;
}

void main () {
    int[][] array = SquareArray!(int)(9000, 9000);
}

Slightly tedious, and it gives you reference semantics and places the 
array on the heap (but do you really *want* 16MB stack frames? I imagine 
the OS might have something to say about that).