Clay language

[Steven Schveighoffer]

On Thu, 30 Dec 2010 12:52:32 -0500, Andrei Alexandrescu  
<SeeWebsiteForEmail at erdani.org> wrote:

> On 12/30/10 11:08 AM, Steven Schveighoffer wrote:
>> I'd have to see how it works. I also thought the new operator
>> overloading scheme was reasonable -- until I tried to use it.
>
> You mean until you tried to use it /once/.
>
>> Note this is even more bloated because you generate one function per
>> pair of types used in concatenation, vs. one function per class defined.
>
> That function is inlined and vanishes out of existence. I wish one day  
> we'd characterize this bloating issue more precisely. Right now anything  
> generic has the "bloated!!" alarm stuck to it indiscriminately.

Functions inline out of existence during runtime, but the function itself  
remains resident in the compiled binary.

I don't know that it's an important aspect to keep it in there or not, I  
just know it's kept.  There are a whole slew of improvements we can make  
in this regard, but I'm not sure they are possible, because I'm not a  
compiler writer.

One such nuisance in particular is the proliferation of types when you use  
something like isInputRange.  That invariably is *only* used at compile  
time, yet the type and its typeinfo are injected into the binary.

>> I mean bloated because you are generating template functions that just
>> forward to other functions. Those functions are compiled in and take up
>> space, even if they are inlined out.
>
> I think we can safely leave this matter to compiler technology.

I hope that can be done.  D already suffers from the 'hey what gives, how  
come hello world is 1MB?!!' syndrome.

>> Let's also realize that the mixin is going to be required *per
>> interface* and *per class*, meaning even more bloat.
>
> The bloating argument is a complete red herring in this case. I do agree  
> that generally it could be a concern and I also agree that the compiler  
> needs to be improved in that regard. But by and large I think we can  
> calmly and safely think that a simple short function is not a source of  
> worry.

Short template functions still have template mangled names.  I have found  
that template names with lots of parameters can slow down compilation, but  
I think Walter is working on fixing that.

If we can get to a point where language constructs such as this can be  
truly inlined out of existence, then I think we will be on another level  
 from other languages.  It's not an unimportant nuisance to be dealt with  
later.

And at that point, I can agree that the template solution is not bloated ;)

>> So I'd say, while my example is not proof that this is a disaster, I
>> think it shows the change in operator overloading cannot yet be declared
>> a success. One good example does not prove anything just like one bad
>> example does not prove anything.
>
> Many good examples do prove a ton though. Just off the top of my head:
>
> - complex numbers
>
> - checked integers
>
> - checked floating point numbers
>
> - ranged/constrained numbers
>
> - big int
>
> - big float
>
> - matrices and vectors
>
> - dimensional analysis (SI units)
>
> - rational numbers
>
> - fixed-point numbers
>
> If I agree with something is that opCat is an oddity here as it doesn't  
> usually group with others. Probably it would have helped if opCat would  
> have been left named (just like opEquals or opCmp) but then uniformity  
> has its advantages too. I don't think it's a disaster one way or  
> another, but I do understand how opCat in particular is annoying to your  
> case.

Probably the most common operator overload in D is opEquals, luckily that  
is not a template (even though it sadly does not work with interfaces yet).

It seems that operator overloads are in categories.  There are the numeric  
overloads, which I agree are generally overloaded in groups.  When I  
defined cursors to be more like C++ iterators in dcollections instead of  
small ranges, I used the ++ and -- overloads, which you typically define  
together.

When designing the mixin that allows you to define various operator  
overloads, I think it would be hugely beneficial to take into account  
these groupings and make the mixins modular.

>> I haven't had that experience. This is just me talking. Maybe others
>> believe it is good.
>>
>> I agree that the flexibility is good, I really think it should have that
>> kind of flexibility. Especially when we start talking about the whole
>> opAddAssign mess that was in D1. It also allows making wrapper types
>> easier.
>>
>> The problem with flexibility is that it comes with complexity. Most
>> programmers looking to understand how to overload operators in D are
>> going to be daunted by having to use both templates and template
>> constraints, and possibly mixins.
>
> Most programmers looking to understand how to overload operators in D  
> will need to bundle them (see the common case argument above) and will  
> go with the TDPL examples, which are clear, short, simple, and useful.

The code itself is simple, it's the "how does x + y match up with this  
template thingy" which is the problem I think.  We've already had several  
posts on d.learn ask how operator overloads work even after reading TDPL.

>> There once was a discussion on how to improve operators on the phobos
>> mailing list (don't have the history, because i think it was on
>> erdani.com). Essentially, the two things were:
>>
>> 1) let's make it possible to easily specify template constraints for
>> typed parameters (such as string) like this:
>>
>> auto opBinary("+")(Foo other)
>>
>> which would look far less complex and verbose than the current
>> incarnation. And simple to define when all you need is one or two
>> operators.
>
> I don't see this slight syntactic special case a net improvement over  
> what we have.

It's less intimidating.  Max pointed out opBinary(string op : "+"), which  
is close, but still has some seemingly superfluous syntax (why do I need  
string op? and what is that : for?)

Compare that to C++ operators:

operator+(Foo rhs)

I'd call that very simple to understand in the context of operator  
overloading.

Also, the proposal is a specialization of templates in general, not just  
for operator overloading.  It translates to the same thing as if you wrote:

opBinary(string $)(Foo other) if($ == "+")

where $ is an inaccessible symbol.  It basically optimizes out the parts  
you don't care about if you don't care about them.

>
>> 2) make template instantiations that provably evaluate to a single
>> instance virtual. Or have a way to designate they should be virtual.
>> e.g. the above operator syntax can only have one instantiation.
>
> This may be worth exploring, but since template constraints are  
> arbitrary expressions I fear it will become a mess of special cases  
> designed to avoid the Turing tarpit.

That's why I conditioned it as "provably" evaluate to single instance.  I  
meant provable by the compiler, so even something that may look obvious to  
a user as only instantiating to one instance may not be provable by the  
compiler.  It probably requires you to use specific constructs (like the  
one mentioned above) to help the compiler out.

>>> Using operator overloading in conjunction with class inheritance is  
>>> rare.
>>
>> I don't use operator overloads and class inheritance, but I do use
>> operator overloads with interfaces. I think rare is not the right term,
>> it's somewhat infrequent, but chances are if you do a lot of interfaces,
>> you will encounter it at least once. It certainly doesn't dominate the
>> API being defined.
>
> Maybe a more appropriate characterization is that you use catenation  
> with interfaces.

Concatenation, equality comparison, indexing, and assignment.  Out of  
those, only concatenation gives me headaches because it requires templates.

If you propose we remove concatenation from opBinary and give it its own  
form, then I thing that would solve the problem too.

>> Actually, the functionality almost exists in template this parameters.
>> At least, the reevaluation part is working. However, you still must
>> incur a performance penalty to cast to the derived type, plus the
>> template nature of it adds unnecessary bloat.
>
> Saw that. I have a suspicion that we'll see a solid solution from you  
> soon!

Alas, no solution is possible without templates being allowed in  
interfaces :(  But yes, I plan to use this technique as soon as it's  
possible.

-Steve