[Issue 17448] Move semantics cause memory corruption and cryptic bugs

[d-bugmail at puremagic.com]

https://issues.dlang.org/show_bug.cgi?id=17448

--- Comment #31 from Shachar Shemesh <shachar at weka.io> ---
Here goes:

(In reply to Andrei Alexandrescu from comment #30)
> Indeed it seems we are not supporting registration by address with ease for
> D value types.

Make that: at all. I understand that there are cases where it is safe to do,
but I don't think we have a good guarantee about those cases. Without them, it
is simply impossible.

> That C++ ... still has a variety of safety,
> correctness, and efficiency issues (I am not exaggerating; all three
> problems are present)

In the interest of better understanding the trade-offs, can you please
elaborate? I'm asking because D's insistence on moving things around is one of
the language features I understand less. I don't see what the huge gains are,
and some of the actual compiler behaviors I've seen are downright
counter-productive. I've seen cases where the same object gets moved around so
much, and for no apparent good reason, that I seriously worry for the
performance bleed from this feature.

> (It should be noted that C++ has its own, distinct issues with
> self-registering objects, to which I dedicate several slides in
> http://erdani.com/index.php/training/mc1xd.)

Unless I've missed where in that link the actual slides are, not helping.

> 
> Allow me to make a few suggestions for workarounds:
> 
> * Avoid automatic/stack allocation and also return by value

This is a HUGE increase in the cost of allocating the struct. There are
efficient mechanisms for dynamic allocations (e.g. - maintaining a pool), but
they cost in other overheads (e.g. - predetermining the number of instances,
adding an initialization stage).

> As a perk, you avoid the
> creation, copying, and destruction of spurious objects - which comes along
> with calls to register/unregister, which I assume has a significant cost.

I don't think registering an object with a linked list is as big a cost as
dynamic allocation.

> * Use a "cookie", not an address, in the registration. A classic
> registration pattern returns a cookie/handle, usually an integer, which the
> object stores.

No, that's not correct. There is indeed a classic pattern where the registrar
returns a cookie, but it *stores* the pointer internally (or, in the Windows
way, obfuscates it into the handle). I don't think you can come up with a
scheme where the Linux kernel cannot translate an integer FD into a file
pointer without user-space's help.

Someone outside of the struct has to know the pointer, or the struct is
unlocatable without already knowing where it is. Needless to say, this would
defeat the purpose of registering it.

> If after exploring these and other solutions you come to the conclusion they
> are not satisfactory, I encourage you to create a DIP. Two possible lines of
> attack are:
> 
> (1) Allow specifying that an object can't be moved
> (2) Allow a type to intercept its move by means of a nothrow hook

(2) is what we've been asking for all along (e.g. - comment #15). Give us a way
to update the external reference when the address changes.

I'll try to phrase a DIP. It would be my first one, so help would be
appreciated.

--