druntime, templates and traits

[H. S. Teoh]

On Fri, Sep 13, 2013 at 03:20:10PM +0200, Maxim Fomin wrote:
> On Friday, 13 September 2013 at 10:26:35 UTC, monarch_dodra wrote:
> >
> >1. Performance (small issue). A lot of code that should be
> >otherwise fast is slower than it could be, since druntime will
> >check, at runtime, if postblit is needed, if destruction should be
> >done. For example, if you "dup" a string by hand, you can do it
> >50% faster than what druntime does.
> 
> Yes, this is pitfall of D design.

I don't see what this has to do with D design. It's just a quality of
implementation issue, isn't it?


> >2. Inference. This is an odd one. Currently, the function that
> >operate on arrays (reserve, dup, length) etc... may or may not be
> >@safe or pure. Or nothrow. The problem is that we really can't do
> >anything about it. duping a string is obviously nothrow, but if
> >you dup something with a postblit, then it might not be (nor
> >safe).
> 
> Yes, this is pitfall of D design.

It's just a matter of taking care of the details. Dup'ing an array
involves allocating a new array (pure @safe nothrow) and copying
elements over (purity/safety/nothrowness depends on postblit). So there
should be two versions of dup, reserve, etc., one for arrays whose
elements can be copied in a pure/safe/nothrow way, one for arrays where
the safety/etc. of copying elements depends on safety/etc. of the
element postblits.

The latter can be taken care of with attribute inference. So for
example, we could do something like this:

	T[] dup(T)(T[] array) pure @safe nothrow
		if (!hasElaborateCopyConstructor!(T[]))
	{
		// To reduce template bloat, forward to static function
		// that does bitwise copying.
		return simpleDupImpl(array.ptr, array.size, T.sizeof);
	}

	T[] dup(T)(T[] array) /* attributes inferred at compile-time */
		if (hasElaborateCopyConstructor!(T[]))
	{
		// This part is pure @safe nothrow
		T[] copy = simpleDupImpl(array.ptr, array.size, T.sizeof);

		// This part's purity/safety/nothrowness inherits from
		// postblit's characteristics.
		callPostblits!T(copy);
	}


> >These issues kind of keep popping up in phobos, and we will have
> >to address them sooner or later. The current state of affairs is a
> >kind of status quo of "its wrong, inconsistent, but maybe OK most
> >of the time".

Yeah this is one thing that irks me about D. It works superbly -- for
the most part. But when you hit something outside "the most part", then
it's a cascade of troubles and roadblocks, one leading to the other. We
really need to dig into all these details and straighten them out,
otherwise it's going to be a big source of frustration to potential D
adopters.


> They are systematically popping up because of systematical language
> features (D is statically typed language with little runtime
> supported).

And how do you propose we address that?


> >For example, "dup" is not nothrow, yet it is safe and pure.
> >Reserve is nothrow, yet assumeSafeAppend is not. Reserve may
> >actually call postblit, but not assumeSafeAppend.

Why is assumeSafeAppend not nothrow? If it were up to me, I'd say that
if for whatever reason safe append can't be done (i.e. the assumption is
invalid) and we detect it, we should assert(0), since the code is
obviously not prepared to deal with this case by virtue of calling
assumeSafeAppend in the first place. Throwing an Exception makes no
sense in this case -- it's not something you can handle.


> >========
> >
> >The solution (I think) would be to partially templatize d-runtime.
> >We can keep most of what is in rt/lifetime.d, but statically split
> >it into smaller chunks, which could be piloted with more accuracy
> >via a templated "manager".
> 
> It depends on how do you want to templatize it. Due to separate
> compilation model it is impossible to know beforehand (when you
> provide phobos library together with druntime) which types will be
> used.

Right, so we use templates for the "front-end" that interface directly
with user types, but the real work is done by static backend functions.
There will be a different backend function to deal with each different
category of cases, and the front-end templates will just forward to the
most appropriate one.


> >For now, if we could make a change as simple as "no postblit =>
> >safe pure nothrow, and fast guaranteed"/"postblit => not safe, not
> >pure, may throw", then it would already be a big win (IMO).
> >
> >========
> >
> >I had started toying around with this, and the main and immediate
> >challenge I ran into is the integration of templates. The 2 issues
> >are:
> >1. All templates must be in object.d, or the compiler won't see
> >it.

public import is your friend. :)


> >2. A simple (and needed) trait like "hasElaborateCopyConstructor"
> >requires a good third of both "typetuple.d" and "traits.d".
> >"functionAttributes" is pretty simple though (I think).
> 
> Yes, this is pitfall of D design.

How is this a problem with D design?

I'd argue that a large part of std.traits *should* be in druntime, not
Phobos, because they interface with compiler internals via __traits.
After "discovering" how parameter tuples work yesterday, I'm becoming
more convinced that __traits should not be used by user code at all, but
wrapped by more user-friendly functions in the runtime environment,
i.e., druntime.

One of the issues I ran into while trying to reimplement the built-in
AA's was that it needed some stuff from std.traits, but can't use it
because druntime isn't supposed to depend on Phobos. The duplication in
that case was relatively small, but if std.traits was in druntime in the
first place, this would never have been an issue.


> >I'm mostly looking to start a discussion on this, and for
> >guidance, ideas, on where to start.
> 
> I doubt that the problems you raised are solvable in general case
> with current runtime capabilities. While it still may be possible to
> tackle some of them, D design (separate compilation model, type
> system, templates, how pure, @safe, nothrow are defined) would
> prevent you from fixing these problems.

The upcoming 2.064 has attribute inference for pure/@safe/nothrow. I'm
pretty sure this will address most, if not all, of these issues, if
applied correctly.


T

-- 
Laissez-faire is a French term commonly interpreted by Conservatives to
mean 'lazy fairy,' which is the belief that if governments are lazy
enough, the Good Fairy will come down from heaven and do all their work
for them.