Override assert handler

[Richard (Rikki) Andrew Cattermole]

On 20/08/2024 9:03 AM, Walter Bright wrote:
> I actually do understand how shared libraries work.
> 
> What happens by default when an assert failure happens is the function:
> 
> ```
> core.exception._d_assertp(immutable(char*) file, uint line);
> ```
> is called. That forwards the call to:
> 
> ```
> core.exception.onAssertError(string file, size_t line);
> ```
> which then forwards the call to:
> ```
> (*_assertHandler)(file,line,null);
> ```
> and core.exception._assertHandler is the pointer to the function.
> 
> The default behavior of _assertHandler is:
> ```
> throw staticError!AssertError(file, line);
> ```
> 
> Therefore, if you write your own _d_assertp function in the executable, 
> it will override the library version in your executable. For code in the 
> shared library, the shared library _d_assertp will be called.

Its not quite as simple as that. For Windows yes that's how it'll work 
without compiler & linker assistance.

"Symbols so introduced may duplicate symbols already defined by the 
program or previous dlopen() operations. To resolve the ambiguities such 
a situation might present, the resolution of a symbol reference to 
symbol definition is based on a symbol resolution order. Two such 
resolution orders are defined: load or dependency ordering. Load order 
establishes an ordering among symbol definitions, such that the 
definition first loaded (including definitions from the image file and 
any dependent objects loaded with it) has priority over objects added 
later (via dlopen()). Load ordering is used in relocation processing. 
Dependency ordering uses a breadth-first order starting with a given 
object, then all of its dependencies, then any dependents of those, 
iterating until all dependencies are satisfied. With the exception of 
the global symbol object obtained via a dlopen() operation on a file of 
0, dependency ordering is used by the dlsym() function. Load ordering is 
used in dlsym() operations upon the global symbol object."

https://pubs.opengroup.org/onlinepubs/009695399/functions/dlopen.html

If your ``_d_assertp`` was first seen it would apply to the entire 
process if the symbol was exported. If the shared library symbol was 
first seen, its the one that gets used no matter what image is being 
discussed as long as everything is exported.

Of course, the Posix behavior of setting the assert handler for the 
entire process is what you want. Not the Windows one where it is 
localized to the binary. After all it doesn't matter who errors out, you 
want to act upon it the same way consistently.

Which brings us back to what I already said, the function pointer design 
works best and doesn't limit you into a subset of desirable situations. 
It also covers the use case where you want to swap it out in a process 
that has been running for a month. Not to mention it's fully portable 
and won't change behavior based upon platform.

> ------
> 
> To understand assert error handling, it's necessary to understand:
> 
> ```
> AssertHandler
> assertHandler
> assertHandler  (yes, two of them!)
> _assertHandler
> onAssertError
> onAssertErrorMsg
> AssertError
> _d_assertp
> _d_assert_msg
> _d_assert
> ```
> 
> which is overly complex.

Agreed, there may be some simplification possible.