Is DMD 2.052 32-bit?

[Jonathan M Davis]

On Friday, March 11, 2011 13:11:53 lurker wrote:
> Jonathan M Davis Wrote:
> > On Wednesday 09 March 2011 17:56:13 Walter Bright wrote:
> > > On 3/9/2011 4:30 PM, Jonathan M Davis wrote:
> > > > Much as I'd love to have a 64-bit binary of dmd, I don't think that
> > > > the gain is even vaguely worth the risk at this point.
> > > 
> > > What is the gain? The only thing I can think of is some 64 bit OS
> > > distributions are hostile to 32 bit binaries.
> > 
> > Well, the fact that you then have a binary native to your system is
> > obviously a gain (and is likely the one which people will cite most
> > often), and that _does_ count for quite a lot. However, regardless of
> > that, it's actually pretty easy to get dmd to run out of memory when
> > compiling if you do much in the way of CTFE or template stuff. Granted,
> > fixing some of the worst memory-related bugs in dmd will go a _long_ way
> > towards fixing that, but even if they are, you're theoretically
> > eventually supposed to be able to do pretty much anything at compile
> > time which you can do at runtime in SafeD. And using enough memory that
> > you require the 64- bit address space would be one of the things that
> > you can do in SafeD when compiling for 64-bit. As long as the compiler
> > is only 32-bit, you can't do that at compile time even though you can do
> > it at runtime (though the current limitations of CTFE do reduce the
> > problem in that you can't do a lot of stuff at compile time period).
> > 
> > In any case, the fact that dmd runs out of memory fairly easily makes
> > having a 64-bit version which could use all of my machine's memory
> > really attractive. And honestly, having an actual, 64-bit binary to run
> > on a 64-bit system is something that people generally want, and it _is_
> > definitely a problem to get a 32-bit binary into the 64-bit release of a
> > Liunx distro.
> > 
> > Truth be told, I would have thought that it would be a given that there
> > would be a 64-bit version of dmd when going to support 64-bit
> > compilation and was quite surprised when that was not your intention.
> 
> I think porting DMD to 64 bits would be a pragmatic solution to this.
> Computers are getting more memory faster than Walter is able to fix
> possible leaks in DMD. There's awful lots of template and CTFE code using
> more than 2 or 3 GB of RAM. I can't even imagine how one could develop
> some modern application if this was a hard limit. Luckily there are GDC
> and LDC, which allow enterprise users to take full advantage of the 24-64
> GB available.
> 
> Some simple use case would be Facebook's infrastructure. Assume Andrei
> wanted to rewrite it all in D. Probably more than 100M LOC. Would need
> hundreds of gigabytes of RAM to compile. It would also take days to
> compile, and maybe 50% less on a 64 bit system.

It's not that bad. dmd has some serious memory leaks with regards to CTFE and 
templates in that it generally doesn't release memory for a lot of it until it's 
done. So, it uses _way_ more memory that it needs to. I don't know why it does 
things the way it does, but theoretically, it should be able to reduce that to 
sane levels on 32-bit. I expect that it just requires taking the time to do it.

Also, in most cases, if using too much memory due to CTFE or templates is a 
problem, then all you have to do is do incremental builds and build each module 
separately. Then you're usually fine.

So, while having a 64-bit dmd would definitely help alleviate dmd's memory 
issues, those memory issues _do_ need to be fixed regardless. And fixing them 
would almost certainly make it dmd's memory consumption acceptable with 32-bit 
in most cases.

- Jonathan M Davis