Disallow thread local concepts to allow fibers move between threads @notl

Richard Andrew Cattermole (Rikki) richard at cattermole.co.nz
Fri Jun 14 17:30:27 UTC 2024


Over time it has increasingly become clear that stackless 
coroutines have won over stackfull coroutines (fibers), due to a 
number of reasons.

This proposal does not introduce a new type of coroutine, but 
provides a migration path for existing fiber users to be able to 
replace their eventloop implementation to using a more powerful 
functionality (such as IOCP which is considered the gold standard 
for its problem domain and is getting to be available everywhere 
in some form) without breaking their entire codebase.

Critically fibers must migrate between threads, the 
implementation isn't the problem, the problem is the language 
does not guarantee that this can occur safely, this was 
understood 12 years ago.
Unfortunately it is not possible for the compiler to understand 
where yields take place like with stackless coroutines, so we 
have to apply a very broad transitive guarantee that the unsafe 
things do not happen at this level.

Everything unsafe takes place with anything thread local. Thread 
local storage is the biggie, but also locks may not appreciate 
being unlocked in another thread and to make matters worse they 
are quite often (if not usually) reentrant upon the same thread.

The proposal for in the language is one new attribute ``@notl`` 
to disable thread local access, it is transitive.

The proposal for in core.attributes is two new attributes 
``@notlwhitelist`` and ``@notlblacklist``. These will only ever 
be recognized on the type, not on a variable.
I am not sold on the names, feel free to suggest others!

The reason why they only work on the type is to prevent working 
around this limitation for a type that genuinely isn't safe to do 
so.

The reason we need the white list is to make ``TaskLocal`` from 
vibe.d work, and the blacklist is to prevent ``__gshared`` or 
``shared`` which are not safe (like a mutex) from working here 
either.

I understand that people may be nervous at the prospect, that the 
intention here is to break all current fiber users code, but that 
isn't the case.
This is for migration purposes only, on the ``Fiber`` type an 
overload would be added that accepts the function with ``@notl`` 
on it. If it is supplied a flag is set that allows moving between 
threads that can be checked.

Do you need to be concerned about this? Unless you're working 
with eventloops and wanting to both be safe and to migrate away 
from a polling solution (which is limited in the number of 
handles each thread can take) probably not. But having (a) 
solution would be valuable to those that need it.

Walter asked me to do an ideas thread on this so here it is.

Destroy!


More information about the dip.ideas mailing list