DMD crash, LDC crash, out of memory, no error display, etc

[Nierjerson via Digitalmars-d]

On Sunday, 9 April 2017 at 08:22:02 UTC, Rainer Schuetze wrote:
>
>
> On 08.04.2017 22:33, Nierjerson wrote:
>> The project I am working on, which is near completion:
>>
>> https://github.com/IllusionSoftware/COM2D
>>
>> Automates COM wrapping almost completely. That is, it takes a 
>> converted
>> COM idl file and generates a D wrapper which handles all the 
>> marshaling,
>> invoking, creation, etc.
>>
>> The problem is that DMD has started crashing when trying to 
>> compile the
>> complete set of interfaces. No error, warnings, etc. Just 
>> crashes.
>
> Interesting project. In Visual D I had to marshal a couple of 
> function calls manually, and that has been quite painful. 
> Automating that process would be great.
>

I think so too ;) If it works out well then it could really 
remove the COM headache. I'm not sure how robust it is though as 
I have only tested it with the photoshop interface, which is 
dynamic, but I guess all COM interfaces have a dynamic back(e.g. 
can use GetIDNames, Invoke, etc).


Most of the marshaling can be done completely automatic. The only 
thing I think that one can't get around is being able to extract 
the VARIANT type at runtime(because it is variant, of course 
;)... but maybe a helper template function will make those cases 
easier. I figure for those cases, one can extend the classes and 
overload the functions to take arguments one likes, if they plan 
on using it often. Only a slight bit of work.


> A couple of notes:
> - you can compile dmd as a 64-bit process using the VS solution 
> in src/vcbuild.

Yeah, I'm using it. Helps because this project gets over 2GB very 
quickly ;/

>
> - when built with this compiler the compile time generated 
> string is output using about 8GB of memory.

Which enum Interface is that? That is about what it uses for the 
small set.

> - it still crashes after that due to a stack overflow (seems to 
> be a compiler bug). You can avoid the recursion using the 
> -allinst command line option at the cost of slightly more 
> memory being used (should be very little in this project)
>

ok, I'm trying it and it seems to work.

> - compilation then continues but seems to be stalled. This is 
> caused by the huge function GenerateDCOMClasses, that is still 
> compiled into the object file even though it is just used at 
> compile time. dmd is pretty slow compiling large funtions, you 
> shouldn't dare enabling optimizations for these...
>
> - eventually (using about 11 GB of memory) the object file is 
> emitted with a size of 137 MB.
>
> If possible you should avoid huge functions.
>
> Would it be feasible to make the generation a runtime tool? If 
> I understand correctly, you run idl2d on the interface file 
> anyway, so integration with idl2d would make sense, too.


Yeah, possible. Seems it uses far less memory(hovering around 
~6GB so far when it would normally be at 12GB+ and eventually 
error out))

I simply removed the mixin code and pragma and allowed will have 
it write the string output to file like it already does. I did it 
the other way for development because it is faster but because of 
the memory issues I couldn't get very far initially). I didn't 
occur to me to do this, I guess because I wanted the compiler to 
just compile it as it was ;) Kinda bugs me that it can't handle 
it well.

Eventually it could be used as a tool to output from the idl2d 
the D wrapper classes. I've hacked up the idl2d output and it 
does expect the idl2d output to have a slightly format(hand 
edited). These can be modded though.

I have to tidy up some things and start testing it out in the 
real world before I'm satisfied, but at least the proof of 
concept seems to work.

Unfortunately I tried compiling again only the string output to 
file case and optilink crashed:

checkpoint(256)
Error: linker exited with status 1
with a dialog of cpu regs


Anyways, I'll play around with the new method, it should, in 
theory, work as it is just writing the large code string to file 
rather than trying to mix in it(which, shouldn't be a big problem 
either as it's just the same string that is output to a file then 
mixed in again).


Thanks.