Assertions getting corrupted

[Jonathan M Davis]

On Thursday, October 26, 2017 11:15:52 Shachar Shemesh via Digitalmars-d 
wrote:
> On 26/10/17 09:27, Jonathan M Davis wrote:
> > since almost no one ever derives from Throwable,
> > and I don't think it's really intended that anyone do so much as it is
> > possible (and I'm not sure why you would)
>
> Here's where we're doing it.
>
> Our product will often have several fibers essentially working for one
> common sub-component. Trouble is, with our product being distributed the
> way it is, it might be possible to simply kill that subcomponent. When
> that happens, we want to kill all the fibers that belong to that
> subcomponent.
>
> The solution is composed of several layers. First, we have a mechanism
> for injecting an exception into another fiber. This means that any
> function that may send a fiber to sleep, by definition, may also throw.
>
> We also have a FiberGroup, where we can assign fibers to the group. This
> way, when the component goes down, we just kill the group, and it sends
> an exception to all fibers that kills them.
>
> This leaves just one problem. What happens if someone catches that
> exception? We want all fibers to actually exit when we do that, and we
> rely on that happening. Obviously, this is not meant for malicious
> coders, so we /could/ mandate the following pattern:
>
> catch(FiberGroupExit ex) {
>    throw ex;
> } catch(Exception ex) {
>    ...
> }
>
> The problems with this pattern are huge. For one thing, you'd have to
> remember to put it anywhere you're catching an Exception. It's also
> quite verbose.
>
> Another solution is to have every other exception inherit not from
> Exception, but from some subclass of it. Problem is, this does not
> include exceptions defined in phobos, libraries, and pretty much
> anything else. It's also quite easy to forget.
>
> The solution we came up with was for FiberGroupExit to not inherit from
> Exception, but directly from Throwable. This means you can catch
> Exception as usual, but injecting a FiberGroupExit (or a ReactorExit)
> unconditionally terminates your fiber, while running all proper cleanups.
>
> Running the cleanups is important, as the process remains running. It
> makes no sense for scope(exit) not to clean up merely because a fiber is
> quitting.
>
> Hope this clears up a few things.

Hmmm. Well, that does make sense, but if cleanup is important, it's not a
great idea, because not only does the language definition not guarantee that
cleanup happens when anything other than an Exception is thrown, it _can't_
guarantee it, because nothrow specifically means that no Exception was
thrown, not that no Throwable was thrown, and when a function is nothrow,
at least some portion of the normal exception mechanism stuff is skipped for
performance.

With the current implementation, if you never use any nothrow functions, I
_think_ that it's the case that you're always going to get cleanup, but if
you ever get a nothrow function (and nothrow _could_ be inferred if
templates are involved), then potentially some cleanup will be skipped
(which is particularly bad if you rely on destructors for stuff like
reference counting). And while I _think_ that the current implementation
guarantees it when nothrow isn't involved, the spec most definitely does
not, and Walter has been adamant about that. So, the runtime could be
changed in the future so that no cleanup is done for anything that isn't
derived from Exception (I'm honestly a bit surprised that Walter has never
done it given how adamant he is that trying to do cleanup when an Error is
thrown is just making things worse). And if it is, then you won't get any
cleanup for your non-Exception Throwables.

But even if the you never purposefully use nothrow, and the runtime never
gets changed to skip cleanup for non-Exceptions, template inferrence could
really bite you if it ever decides that a function is nothrow. So, that
alone makes me think that your approach is risky.

It may very well be that what you've done is your best choice given the
options, but you're relying on behavior that the spec specifically does not
guarantee (i.e. cleanup when a non-Exception is thrown), and the fact that
it would be screwed up right now if nothrow ever gets involved means that
it's not even just a question of what the spec guarantees vs what the
runtime currently does. You could easily have a serious problem right now.
Unless you know for sure that you never throw one of your Throwables from a
nothrow function (which is hard to know unless you don't use templates in
any of the code using these Throwables), you could very well be missing some
cleanup in your code right now, causing subtle bugs that would not be there
if you were using Exceptions.

If you're very careful about templates and nothrow, you can certainly make
it work with the current runtime implementation, but if I were you, I'd use
an approach that didn't use Throwables that aren't derived from either
Exception or Error. Even if any other solution has other issues, at least
it's not going to involve cleanup code being skipped on you - though it
sounds like you may be stuck between the risk of cleanup code being skipped
(but fibers exiting when you want them to) and fibers not exiting when you
want them to (but cleanup code not being skipped).

Clearly, you have a bit of a thorny problem.

- Jonathan M Davis