[Issue 16479] Missing substitution while mangling C++ template parameter for functions

[d-bugmail at puremagic.com]

https://issues.dlang.org/show_bug.cgi?id=16479

--- Comment #9 from github-bugzilla at puremagic.com ---
Commits pushed to master at https://github.com/dlang/dmd

https://github.com/dlang/dmd/commit/b75c9f110795109aebe610cfbf8f814cd1d6afee
Fix issue 16479: No namespace substitution for C++ mangling on POSIX

The C++ ABI used by POSIX is the Itanium C++ ABI.
Reference document available here:
https://itanium-cxx-abi.github.io/cxx-abi/abi.html#mangling

One important (and tricky) part of the ABI is the substitutions being done,
in order to reduce the bloat introduced by long symbol names,
a typical issue when using templates heavily (of which D was not exempt).

There are 2 kinds of substitutions: component substitution and template
parameter substitution.
Component substitution replaces repeated parts of the symbol with `S[X]_`,
template parameter substitution replaces occurences of template parameters with
`T[X]_`.
`X` represents a base36 index into the array of components or
template parameters already encountered so far, respectively.

This substitution is done on an identity basis, which means that the templated
function
`template<typename T> int foo()` instantiated with `int` will be mangled as
`_Z3fooIiE*i*v`
(asterisks are emphasis and not part of the mangling) while it would be mangled
as `_Z3fooIiE*T_*v`
if the definition was `template<typename T> T foo()`.

Moreover, experience with C++ compilers shows that component substitution is
prefered over
template parameter substitution, such as `template<typename T> T foo(T)` is
mangled as
`_Z3fooIiET_*S0_*` when instantiated with `int` and not `_Z3fooIiET_*T_*` as
would be the case
if template substitution was prefered.

This is just brushing the surface of the problem, since only template type
parameters have been
mentioned so far, but other kind (aliases, values) are also concerned.
Substitution also needs to happen if a template parameter is part of another
type,
such as the `template<typename T> array<T>* foo (T, int)`, which, when
instantiated with `int`,
is mangled as `_Z3fooIiEP5arrayIT_ES1_i`.

For more detailed test cases, see `test/compilable/cppmangle.d`.

The main issue encountered while implementing this in DMD is that there's no
easy way to know
if a type (which is part of the function's type, e.g. parameters and return
value)
was a template parameter or not, as DMD merges types, so in the previously
mentioned
`template<typename T> int foo()` vs `template<typename T> T foo()` the template
instantiation
will come with the same exact two pointer to the singleton `int` type.

Moreover, DMD does destructive semantic analysis, meaning that objects gets
mutated,
pointers get replaced, aliases get resolved, and information gets lost.

After different approaches where taken, the most practical and reliable
approach devised was to
provide a `visit` overload for non-resolved AST type `TypeIdentifier` and
`TypeInstance`,
and compare the identifier to that of the template definition.
Fallback to post-semantic type when it isn't found.

Note that no attempt has been made whatsoever to handle the mess that would
result from
expressions themselves being mangled. The reference doc for the ABI mentions
that
"[...] this mangling is quite similar to the source token stream. (C++ Standard
reference 14.5.5.1p5.)".
Original quote:
https://itanium-cxx-abi.github.io/cxx-abi/abi.html#expressions (5.1.6
Expressions)

https://github.com/dlang/dmd/commit/06d45325331a4c14a099da1d45fa4216c07ab2f6
Merge pull request #8455 from Geod24/cppmangle-fix-16479

Fix issue 16479: No namespace substitution for C++ mangling on POSIX
merged-on-behalf-of: Iain Buclaw <ibuclaw at gdcproject.org>

--