Postmortem: Template unittests are bad & you shouldn't catch Error

[H. S. Teoh]

On Thu, Oct 22, 2020 at 04:04:26AM +0000, Mathias LANG via Digitalmars-d wrote:
[...]
> First, why on god's green earth are we putting unittests with explicit
> template instantiation inside a *template* ? `unittest` inside
> templates are a terrible "feature", and I've never seen a correct
> usage of it, which would be a unittest that relies on the
> user-instantiated instance to test that its expectation still holds.
>
> I'm pretty sure the rationale is "for documentation", and we should
> really re-think this, because those unittests are being run from
> *every single module that instantiate Nullable*. Actually, they are
> being run for *every instantiation of Nullable*.
[...]

This is a known issue that has been brought up many times in the past.
Basically:

1) A unittest block inside a template is duplicated for *every*
instantiation of the template. To alleviate this, you'd need to do
something like this:

	template MyTemplate(Args...) {
		auto myFunc(...) { ... }
		static if (Args == ...)
		unittest { ... }
	}

This is not a fool-proof solution, however, because if that specific
combination of Args isn't actually instantiated, the unittest is
silently ignored (even if it would have triggered a failure due to some
bug). To alleviate this, you'd have to insert something like this
outside the template body:

	// Make sure unittests get instantiated.
	version(unittest) alias _ = MyTemplate!(...);

Which is, of course, extremely ugly. And definitely not something
newcomers would think of.

Another way to solve this is to move the unittest outside the template
body. However:

2) Ddoc's unittests require unittest blocks to be attached to the symbol
they're documenting. So you pretty much have no choice, unless you use
Phobos StdDdoc hack (ugh):

	struct PhobosTemplate {
		auto phobosFunc(...) { ... }

		version(StdDoc) // ugh
		///
		unittest { ... }
	}


Jonathan Davis & myself have proposed in the past an enhancement to
solve this problem: have some way of marking a unittest block inside a
template as single-instance only. Something like this:

	template MyTemplate(T)
	{
		auto myFunc(...) { ... }

		/// This is instantiated once per template instantiation
		unittest {
			pragma(msg, T.stringof);
		}

		/// This is instantiated only once, ever
		/// (The use of 'static' is hypothetical syntax, it can
		/// be anything else that marks the unittest as
		/// single-instance)
		static unittest {
			//pragma(msg, T.stringof); // Error: cannot reference T here

			// Have to explicitly instantiate the template
			// with the arguments you want to test for
			alias U = MyTemplate!int;
		}
	}

However, so far no action has been taken on this front besides the
proposal.


T

-- 
"Maybe" is a strange word.  When mom or dad says it it means "yes", but when my big brothers say it it means "no"! -- PJ jr.