Difference between __gshared and shared.

[Jonathan M Davis via Digitalmars-d]

On Thursday, 9 July 2015 at 12:39:00 UTC, Márcio Martins wrote:
> On Wednesday, 8 July 2015 at 21:15:19 UTC, deadalnix wrote:
>> On Wednesday, 8 July 2015 at 12:08:37 UTC, Jonathan M Davis 
>> wrote:
>
>>> I know that there are a number of people who get frustrated 
>>> with shared and using __gshared instead, but unless you fully 
>>> understand what you're doing and how the language works, and 
>>> you're _really_ careful, you're going to shoot yourself in 
>>> the foot it subtle ways if you do that.
>>>
>>> - Jonathan M Davis
>>
>> Amen
>
> What sort of subtle ways? Can you give examples that are not 
> effectively the same subtle ways you would encounter with 
> pthreads in C/C++? I have been running with the assumption that 
> __gshared effectively bypasses TLS, which again, feels sort of 
> dirty to use a __ prefixed keyword for that, but, yeah...

Well, the compiler is free to assume that a variable that is not 
marked as shared is thread-local. So, it's free to make 
optimizations based on that. So, for instance, it can know for a 
fact that

auto foo = getFoo();
auto result1 = foo.constPureFunction(); // This function _cannot_ 
mutate foo
auto result2 = foo.constPureFunction(); // This function _cannot_ 
mutate foo
auto bar = foo;

So, it knows that the value of bar is identical to the value of 
foo and that result1 and result2 are guaranteed to be the same, 
because it knows that no other thread can possibly have mutated 
foo within this code, and there's no way that this code mutated 
foo even through another reference to the same data on the same 
thread. And it can know that thanks to how const, pure, and TLS 
all work. The compiler is free to optimize the code or make other 
alterations to it based on that knowledge, so if it makes an 
optimization based on that, and foo is actually shared across 
threads (either because the object it refers to was originally 
__gshared or because shared was cast away incorrectly), then 
you're going to have incorrect machine code. And what 
optimizations the compiler does with code like this could change 
over time. And unless you're an expert in the language and in the 
compiler, you're not going to know when the compiler is going to 
make optimizations where the fact that the variable is in TLS 
factors in. So, you're not going to know when the compiler might 
optimize your code in ways that won't work with __gshared, and 
what optimizations it does or doesn't do right now won't 
necessarily be the same ones that it does or doesn't do later.

You have the same problem with shared, but in that case, the 
compiler makes it so that you have to cast away shared to get 
into this mess. It protects against doing stuff like accidentally 
passing a shared object around into code that will treat is a 
thread-local. Heck, with a __gshared object, if you change its 
type, it could go from being a value type where passing it to 
other code works just fine because it's truly copied to being a 
reference type (or partial reference type) where it's not copied 
(or only partially copied), and the compiler won't be able to 
help you catch the points where you were doing a full copy before 
but aren't now. And if it's your coworker that changed the 
definition of the type of the variable that you marked as 
__gshared, you could be screwed without knowing it.

Really, we can't tell what subtle behavioral problems you're 
risking with __gshared, because that depends on what the compiler 
is currently able to do with the assumption that a variable is in 
TLS. You run into all of the problems that you risk with sharing 
variables in threads in C++ only worse, because the D compiler is 
free to assume that an object is thread-local unless it's marked 
as shared and thus can make optimizations based on that, whereas 
the C++ compiler can't. And you've thrown away all of the 
compiler's help by using __gshared. __gshared is intended 
specifically for use with interacting with C code where we don't 
really have a choice, and you have to be careful with it. For 
everything else, if you need to share data across threads, that's 
what shared is for, and the compiler then knows that it's shared, 
so it will optimize differently, and it'll yell at you when you 
misuse it. Ultimately, you still have the risk of screwing it up 
when you cast away shared when the object is protected by a lock, 
but then at least, even if you end up with a subtle bug, because 
a thread-local reference to the shared data escaped the lock, 
it's a lot easier to figure out where you've misused shared 
incorrectly in D than figuring out where you might have screwed 
it up in C++, because all of your shared objects are explicitly 
marked as such, and it's the points where you cast away shared 
that risk problems, so you have a lot less code to look at.

As annoying as it can be, shared is your friend. __gshared is not.

- Jonathan M Davis