App hangs, GC.collect() fixet it. Why?
Bastiaan at Veelo.net
Mon Sep 28 19:28:18 UTC 2020
On Monday, 28 September 2020 at 15:44:44 UTC, Steven
> On 9/28/20 8:57 AM, Bastiaan Veelo wrote:
>> I am glad to have found the cause of the breakage finally, but
>> it won't be easy to find a generic solution...
> Obviously, this isn't a real piece of code, but there is no way
> around this. You have to align your pointers. The other option
> is to not use the GC and use manual memory management.
> If this is a compatibility thing between D and Pascal, and you
> absolutely have to have the same layout, is there a way to
> adjust the structure in Pascal? Like put the elements that
> misalign the pointers at the end of the structure?
> Another totally drastic approach would be to supply your own
> even-more-conservative GC which will scan misaligned pointers.
> Probably going to hurt performance quite a bit. You might be
> able to get away with marking only certain blocks as having
> misaligned pointers, but you will have to scan all the stacks
> with this assumption.
> Some more information about the setup you are using might help
> (I'm assuming D and Pascal are using the same memory in the
> same process, otherwise this wouldn't be a problem). In
> particular, where does the data come from, and how malleable is
> it in your system? Are there times where references to the D
> data only exist in Pascal?
Thanks a lot for thinking with me. I’m not linking any Pascal
objects, so I don’t need to maintain binary compatibility in
memory; Only compatibility of data files. The problem arises when
those files are read using memory mapped files, from which
structs are memcpy’d over. This is of course the result of
machine translation of the current Pascal implementation.
Manual memory management is an option and would be
straightforward in principle, as we’ve done that for ages. The
only thing is that this memory cannot contain other allocations
on the GC heap, such as strings or other slices, unless they are
both aligned and their root is registered.
Fixing the alignment in Pascal is possible in principle, but any
old files would then need to first be processed by the last
Pascal version of the programs, which we then would need to keep
around indefinitely. There would also be issues when we port from
32 bit to 64 bit.
Another option could be to use 1-byte aligned structs for I/O,
and copy the members over in default aligned versions. But this
cannot be part of the automated transcompilation.
Thanks for suggesting a custom gc, which I had not thought of.
I’m leaning towards ditching the memory mapped I/O on the D end,
and replace it by regular serialisation/deserialisation. That
will be a manual rewrite though, which is a bit of bummer as
memory mapped files are widely used in our Pascal code. But this
will probably give the best end result.
More information about the Digitalmars-d-learn