Overriden method not detected ?

[Jonathan M Davis via Digitalmars-d-learn]

On Friday, June 03, 2016 13:27:41 chmike via Digitalmars-d-learn wrote:
> On Friday, 3 June 2016 at 12:41:39 UTC, Jonathan M Davis wrote:
> ...
>
> > On a side note, be warned that you almost certainly shouldn't
> > be using __gshared like this. It's intended for interacting
> > with C code not for D objects to be marked with D. As far as
> > the type system is concerned, __gshared isn't part of the type,
> > and the variable will be treated as thread-local by all of the
> > code that uses it, which can result in really nasty, subtle
> > bugs when the compiler starts doing stuff like optimizations.
> > If you want to be sharing D objects across threads, you really
> > should be using shared so that the compiler knows that it's
> > shared across threads and will treat it that way.
>
> Thanks to point this out. What does shared really do behind the
> scene ?
> Does it add synchronization instructions ?

shared does not add any synchronization instructions. It makes it so that
the object is on the shared heap instead of the thread-local heap; it makes
it so that that the compiler treats it as shared across threads (whereas if
it's not shared, the compiler assumes that it's on the thread-local heap and
that no other thread has access to it); and it makes it so that certain
things are illegal without using core.atomic (e.g. directly incrementing a
shared int isn't legal, because it's not thread-safe). If a variable is not
shared, then the compiler is free to assume that it's thread local and make
optimizations based on that fact.

So, if you have a shared object, you have to either make it use core.atomic
to do anything like incrementing values, or you have to cast away shared to
operate on the object (in which case, you need to have protected all access
to the object with a mutex or synchronized block just like you'd do with
C++). So, you end up with code like

shared MyObj myObj = getObjFromSomewhere();
synchronized(lockObj)
{
    // myObj is now protected by a lock; all other accesses to it
    // should be as well, or this isn't thread-safe.

    auto threadLocal = cast(MyObj)myObj;
    // do stuff with the thread-local reference

    // don't let any thread-local references to myObj escape
}
// the lock has been released, and the only references to myObj are
// shared.

It's basically the same thing that you'd do in languages like C++ or Java
except that you're forced to cast away shared, whereas in those languages,
everything is shared, and you can potentially have thread-related problems
pretty much anywhere in your code.

You can have also all of the functions on your struct/class be shared and
call them without locking, but you risk all of the normal race conditions if
you're not either protecting it via locks or using the atomic operations
from core.atomic, otherwise a number of operations on built-in types will be
considered illegal and result in compilation errors, since if operations
aren't atomic, they need to be done with the object being properly locked,
or they're not thread-safe.

In theory, we're supposed to get synchronized classes, which protect their
member variables via a lock for the whole class such that the outer layer of
shared can be stripped away within its member functions, but it's never been
fully implemented. Rather, right now, the closest we have is synchronized
functions, which is basically the same as using synchronized blocks, so it
doesn't have any of the properties that would allow the compiler to strip
away the outer layer of shared (since it can't guaranteed that there are no
other, unprotected references to the same data). So, if we had synchronized
classes, you could potentially avoid the ugliness of casting away shared,
but we don't have them yet.

The concurrency chapter from Andrei's book, "The D Programming Language" is
available for free online:

http://www.informit.com/articles/article.aspx?p=1609144

So, you can read that for more details, but it does talk like synchronized
classes are implemented, and we don't have synchronized functions (since
that was the plan when it was written but hasn't happened yet).

Another resource to look at would be Ali's book:

http://ddili.org/ders/d.en/concurrency_shared.html

shared does feel a bit clunky (particularly when you need to cast it away),
but it does allow you to segregate the code that involves threads, and it
helps protect you against race conditions, whereas if you use __gshared,
you're risking serious problems.

> In my case I really don't want the compiler to add
> synchronization instructions because the objects are immutable
> from the user perspective. This is enforced by the interface. Tho
> objects are fully instantiated in a private static this() {}
> function which shouldn't be affected by multi-threading since it
> is executed at startup.

If the object is really immutable, then just make it immutable. immutable is
implicitly shared across threads, but it doesn't have any of the locking
issues, because it can't ever be mutated.

Now, if what you mean is that the object is const such that a mutable
reference to that data could still change it, then that's different, and
you'll need to use shared to share across threads and do all of the
appropriate locking.

Also, be warned that if you're using static this with either shared or
immutable (or __gshared), then that static constructor needs to be marked
with shared as well, otherwise it will be run in every thread (the compiler
should prevent you from initializing static or module-level variables which
are shared or immutable in a non-shared, static constructor, but it doesn't
currently).

> The unpleasant side effect of shared is that I then have to use
> shared(Info) instead of the shorter type name Info.

Well, to some extent, that's on purpose in that the portion of your code
that deals with objects that are shared across threads should be very small
and segregated so that it's easy to manage - similar to how using @safe and
@trusted makes it so that you only have to look at @trusted or @system code
to figure out how something like a memory corruption happened, because as
long as @trusted is used properly, @safe code is guaranteed to never be able
to corrupt memory. And if shared is used correctly, then you're guaranteed
to never have threading issues with your thread-local objects.

> What are the subtle and nasty bugs you are referring to ?

The compiler is free to optimize based on the fact that an object is
thread-local if it's not shared. So, any optimizations that it ends up
making based on that will cause weird, subtle problems if the object is
actually shared across threads. What those problems are depends on what
optimizations the compiler does and how the object is being used across
threads.

- Jonathan M Davis