I've just fixed UFCS for the experimental type function branch

[H. S. Teoh]

On Thu, Sep 10, 2020 at 11:44:30PM +0000, Bruce Carneal via Digitalmars-d wrote:
> On Thursday, 10 September 2020 at 16:52:12 UTC, H. S. Teoh wrote:
> > On Thu, Sep 10, 2020 at 04:28:46PM +0000, Bruce Carneal via
> > Digitalmars-d wrote:
[...]
> > > Absolutely.  Functions are more readable, compose more readily,
> > > are easier to debug (better locality), and tickle fewer compiler
> > > problems than templates.  CTFE is a huge win overall, "it just
> > > works".
> > 
> > I wouldn't be so sure about that last part. The current CTFE
> > implementation is, shall we say, hackish at best? ...
> 
> When I said "it just works" I should have said "it just works as you'd
> expect any program to work".  The implementation may be lacking but
> the contract with the programmer is wonderfully straightforward.

If that's what you meant, then I agree.  One of the big wins of CTFE is
that it unifies compile-time code and runtime code into a single
interface (syntax), rather than require periphrasis in a separate
sub-language.  D templates win in this respect in the area of template
functions: template parameters are "merely" compile-time parameters,
rather than some odd distinct category of things with a different
syntax; this has been one of the big factors in the usability of D
templates.

However, D templates fail on this point when it comes to non-function
templates, e.g. you have to write a recursive template in order to
manipulate a list of types, and imperative-style type manipulation is
not possible.  This adds friction to usage (e.g., I have to re-think my
type sorting algorithm in terms of recursive templates rather than just
calling std.algorithm.sort) and induces boilerplate (I can't reuse an
existing sorting solution in std.algorithm.sort but have to rewrite
essentially the same logic in recursive template style). Ideally, this
redundant work should not be necessary; as long as I define an ordering
predicate on types, I ought to be able to reuse std.algorithm for
sorting or otherwise manipulating types.

Seen in this light, type functions are a major step in the right
direction.


[...]
> > To be fair, the problems really only arise with IFTI and a few other
> > isolated places in D's template system.  In other respects, D
> > templates are wonderfully nice to work with.  Definitely a
> > refreshing change from the horror show that is C++ templates.
> 
> Yes.  Of course in theory D templates are no more powerful than C++
> templates but anyone who has used both understands that simplicity in
> practical use trumps theoretical equivalence.  As you note, it's not
> even close.

Theoretical equivalence is really only useful in mathematical proofs; in
practice there's a huge difference between languages of equivalent
computing power. Lambda calculus can in theory express everything a D
program can, but nobody in his sane mind would want to write a
non-trivial program in lambda calculus. :-D  Hence the term "Turing
tarpit".


> It seems to me from forum postings and reports on the
> (un)maintainability and instability of large template heavy dlang code
> bases, that we're approaching the practical limits of our template
> capability.  At least we're approaching the "heroic efforts may be
> needed ongoing" threshold.

It really depends on what you're trying to do.  I'm a pretty heavy
template user (bring on those UFCS chains!), but IME it has not been a
problem.  The problems really only arise in specific usage patterns,
such as excessive use of recursive templates (which is where Stefan's
type functions come in), or excessive use of compile-time codegen with
CTFE and templates (e.g., std.regex, std.uni tables).

Or unreasonably-long UFCS chains: I have a non-trivial example in one of
my projects where almost the entire program logic from processing input
to outputting a PNG file exists in one gigantic UFCS chain. :-D  It led
to megabyte-long symbols that eventually spurred Rainer to implement the
symbol folding that we enjoy today. Eventually, I had to break the chain
down into 2-3 pieces just to get it to compile before running out of
memory.  :-D

For "normal" template usage, templates really aren't that big of a
problem.  Unfortunately, some of the "bad" usage patterns occur in
Phobos, so sometimes the unwary can trip up on them, which may be why
there's been a string of complaints about templates lately.


> So, what to do?  We can always add more tooling to try and help the
> situation: better error reporting, better logging, pattern resolution
> dependency graph visualizers, ...

I say we fix the compiler implementation so that we can use what the
language allows us to use. :-)


> We can also go the "intrinsics" route: "have something that's too hard
> to do with templates?  No problem, we'll add an intrinsic!".

I wouldn't add an intrinsic unless it provides some special
functionality not expressible with the normal language. I don't think we
have too many of those.

The current problems with templates is really a matter of quality of
implementation.  That, and the lack of more suitable ways of doing
certain things like type manipulation, so templates get pressed into
service where they are perhaps not really the best tool for the job.


> We can also go the template library route: "too tough for mere
> mortals?  No problem, my super-duper layer of template magic will make
> it all better!".

std.algorithm anybody? ;-)


> You'll note that not one of the above "solutions" actually reduces
> complexity, they just try to manage it.  Type functions, on the other
> hand, look like they would support real world simplification.  Much of
> that simplification comes from programmer familiarity and from the
> ability to "opt-in" to pattern/set operations rather than being forced
> to, awkwardly, opt-out.
[...]

Type functions will definitely be a major step towards unifying the meta
language with the regular language: the holy grail of metaprogramming. I
doubt we can really get all the way there, but the closer we get, the
more powerful D will become, and at the same time the more easy to use
D's metaprogramming features will become.  That's worthy of pursuit IMO.


T

-- 
Doubt is a self-fulfilling prophecy.