[Issue 2766] DMD hangs with 0%cpu

[d-bugmail at puremagic.com]

http://d.puremagic.com/issues/show_bug.cgi?id=2766





------- Comment #1 from unknown at simplemachines.org  2009-03-29 04:50 -------
Created an attachment (id=299)
 --> (http://d.puremagic.com/issues/attachment.cgi?id=299&action=view)
Potential fix - prevent recursion if optimized type matches.

I'm not sure this is the right solution, but this does solve this problem.

The stack is overflowing (eventually), for some reason this is causing a hang. 
It's caused by keyword #1, |const|.  It's trying to cast from const(byte[]) to
byte*.

This seems like something one might see a lot, actually.

What's happening is, |Expression::implicitConvTo()| gets called being
|cast(const(byte)*)cast(const(byte)*)[cast(byte)0]|.  The optimize() call
strips off one of those casts and returns a new |e|, so it calls itself on the
optimized one.  But somehow, that mangy cast gets back in there, and it
recurses endlessly like this.

Visual Studio doesn't show any calls between calls to
|Expression::implicitConvTo()|, but I'm not seeing where it is changing. 
Nonetheless, |e| and |this| are different each time.

However, the purpose of this recursion seems to simply be to properly check the
optimized type.  If the optimizer didn't change this type, it hardly seems
necessary to recurse.  This also happens in |Expression::implicitCastTo()|.

Still, there is a deeper problem where the cast is doubling up.  This works
around the issue (hopefully not incorrectly), but does not solve the root
issue.

-[Unknown]


--