[OT] compiler optimisations

[via Digitalmars-d]

On Sunday, 26 April 2015 at 11:33:07 UTC, Laeeth Isharc wrote:
> Conceivable, but you can hardly control what people do with and 
> say about their use of a programming language, even of a closed 
> source commercial product.  I guess one can submit pull 
> requests that take the language in the direction one favours 
> though, and maybe you do this.

You are downplaying the underlying issue. When you design a 
language you have layers that build upon each other. The memory 
model is a foundational design layer (not implementation) that 
affects all layers above it. It's not a patchable entity.

> by what takes off.  The world is a big place and changing 
> rapidly.  If one has a set idea about what something should and 
> shouldn't do, one may find oneself eventually overcome by 
> Nature, who is more powerful - I have given up trying to fight 
> her.

That would take a paradigm shift. Which could happen, but not any 
time soon. (e.g. program synthesis). D/C++/Java are all built on 
Simula's model. Nothing surprisingly new.

And no, the uptake of programming languages over the past 30 
years have not been full of surprises... It's been surprisingly 
predictable. Installed base and critical mass across the board.

> available.  It's a funny thing I notice people do to pretend 
> that decisions about language adoption are based on the merits 
> of the pure language itself, when only for a subset (I suspect 
> a minority) is that truly the case.

As usual in these technological social dynamics you have:

1. early adopters

2. hubs (people/artifacts that connect to lots of new nodes in 
the network)

3. mainstream adoption (commercial project management)

At first you need the early adopters. Like independent game 
devs/enthusiast that pave the road by building attractive 
infrastructure, like state of the art frameworks. Then the 
frameworks/people (if they are good) will connect to the more 
pragmatic audience. But that means you need excellent 
programmers/designers in the first place, and that means you need 
something that excellent programmers/designers find attractive. 
If Rust overcomes the complexity of linear typing, then they are 
in a good spot (in terms of social dynamics), but that's a big 
"if".

The mainstream will go with what is perceived as low risk/reduced 
costs. And that takes a lot more than a downloadable compiler or 
three.

> [I am not sure if it is escapism to listen to your market and 
> do what you need to to address the biggest concerns].

You mean like fixing the language so that you can either have a 
fast GC, efficient ownership handling or both? Surely you need at 
least one?

> necessarily understand the needs of others.  Of course it is 
> frustrating if the world charges ahead in a direction that one 
> doesn't find interesting, but submitting code probably has more 
> influence than telling people they shouldn't do this.

Nope. As long as there are intelligent people involved informing 
is always the most efficient strategy, since you need people to 
pull in the same direction. If not, then forking is the only 
solution. Submitting code has very low effect when the key issues 
are design issues or knowledge related.

You now have @nogc and a modest effort on improving the GC. That 
came about as an outcome of persistent debate. Without persistent 
debate, nothing would have come in that direction.

> This whole scripting vs system language thing suffers from 
> reification of a distinction that once mapped to something 
> crisp in reality, but no longer does.

Uhm... This is completely wrong. It's like conflating perl and 
Ada.

> Is Go a scripting language or a systems language?

It is an application level programming language geared towards 
servers using a particular implementation strategy (CSP/GC). Go 
is a very opinionated language.

>  On the one hand, the D front page no longer positions D as a 
> systems language (which I think is the right move); on the 
> other, people are using it for low-level stuff.

Andrei and Walter both keep calling it a system level programming 
language. Walter repeatedly states that even 1% performance loss 
is a big issue for him. So obviously they will need to focus on 
backing that up, or make a statement that there is a shift in 
direction.

> Adam's technical contribution is large in itself, but the 
> effect of his inspiration on others may well be larger.

Probably, but workaround makes for unmaintainable code. Tricks 
are cute, but you should avoid using tricks in production.

> Have you written anything on what this should look like?

Could, not should. Read up on linear typing and you see what the 
perimeter looks like. Read up on C++ and you know what current 
practice looks like. Read up on dependent and behavioural typing 
and you may catch a glimpse of the horizon.

As far as I am concerned, C99/C++ semantics are good enough as a 
common ground (if made orthogonal). The main issues C++ struggles 
with are syntactical/textual. That doesn't mean you shouldn't try 
to do better, of course.

> Must it be the case that we are all in a battle to the death 
> for a share of a limited pie?

As long as the resources are limited, yes. As long as the 
programming language theory status quo is limited, yes.

The vast majority resources are not poured into the language 
development, it is put into the software written with it.

> expense of the rest of society? Actually, yes: Profits come out 
> of customers' wallets, and monopolies deserve their bad 
> reputation—but only in a world where nothing changes.

English speaking countries have huge savings on not maintaining a 
small scale language. The only advantage in speaking Norwegian is 
social dynamics and group borders, but those niche benefits have 
a high cost. In Norway we have 3 official languages unique to 
this country, with a rather small population. That's great for 
maintaining group identity, but there are costs involved and no 
measurable advantages, they are cultural.

You forget that imperative Simula-style languages are more or 
less the same, but the devil is in getting the details right. And 
CS is a field that gives you a lot of handholding to get the 
quirks out.

> But the world we live in is dynamic: We can invent new and 
> better things.

Language invention by and large happens in academia. Very little 
significant invention happens outside it. Neither C++/D or Java 
qualify as being inventive. That's ok, we are getting different 
blends of the same. Then you need to find the right blend, and CS 
and language history provides guidance.

> matters as a user of D isn't its market share, but whether one 
> can bet on it being around in 5-10 years, whether one can find
> people capable of helping one as one grows, whether its robust 
> enough for ones application (not all compiler bugs are equal), 
> power, efficiency, productivity, and the existence of and ease 
> of porting frameworks needed to accomplish one's ends.

What matters is that projects can pick the language because it 
reduces risks and saves costs. It is currently difficult to make 
a case for D/Rust/Nim without factoring in emotional factors 
(such as personal fun). Go might have a case in some scenarios.

> So as regards adoption, the following chart is much more 
> interesting to me than market share - and I think it should be 
> to you too, perhaps!

No, because retention is the only metric that matters. So github 
and stackoverflow presence is a more significant measure. Here's 
D's github presence:

https://github.com/trending?l=d&since=monthly

> need to learn a whole bunch of rules.  I personally found D 
> quicker to pick up to reach a level where I can be decently 
> productive than Python, and I don't think that would have been 
> true of C++.

C++ suffers from a lot of textual noise. Python's model is quite 
simple, much simpler than D. What matters long-term is the core 
language. E.g. the semantical language that you are dealing with 
after removing all the sugar.

> Plus I personally wouldn't be short (bet against) quality of 
> tooling if you look at how things have developed and what is in 
> the pipeline.

There is no language spec.

>> However the "winner takes it all" effect has become a lot 
>> stronger now that you have so many excellent free libraries.
>
> I thought languages weren't judged by libraries ;)  But I take 
> your point, and agree with it - which is why C++ interop is so

Languages should not be judged by libraries, but libraries are 
judged by project managers.

> Empirically though, as someone else here said, the number of 
> languages in decently wide usage does not seem to fit your view 
> of a winner take all phenomenon (I am not sure if you intend 
> this to apply to all languages, or just to systems languages).

It applies to all general programming languages.

> That's what Knuth called for in the talk I posted from decades 
> back - he said that because use cases differ and because of the 
> diversity of cognitive styles, one size fits all in language 
> design was not the right way.

I sincerely doubt he was arguing for having a dozen different 
Simula descendants with a little bit of ad-hoc generic 
programming bolted on.

He probably referred to different programming paradigms. C++ and 
D are the same paradigm.

> don't win by beating your opponent head-on in the area where he 
> is strongest - that's just suicidal.  So in my view it's 
> totally irrelevant to speak about what would be needed to get a 
> core heavy C++ guy to switch - you want to persuade the guy who 
> has unusual needs, who is unhappy with his existing options, 
> who has more freedom to try things, and the like.  Then if you 
> achieve big wins in small areas, other people will slowly take 
> notice.

But then you need to excel in at least one area?! Competing 
compilers are free, the price point can't be lowered. So your 
only option is to compete on quality, marketing or bundling (e.g. 
swift/C#).

> can do that very well in many cases, and not quintessentially 
> exclusively a systems language.

But _A_ systems language. And that comes with lot of requirements 
if it is to be taken seriously.

> pin on it ?  It might be more constructive to say: here is the 
> problem I am grappling with in building this embedded 
> system/doing this audio processing/etc, and there are these 
> little things in the runtime/library etc that get in the way.

If D was mature and competitive yes, but I already know the 
answer. C++ is currently the better option, even if it is a 
syntactical mess. That does not mean that I favour C++, or would 
do hobby projects in it. It means that I don't find D a rational 
choice for commercial use in system level programming.

>> area are distributed cloud solutions. Like Google Big Query 
>> that AFAIK can do brute force SQLish quries over very large 
>> datasets fast (using some kind of built in query optimization).
>
> There is a lot inbetween the area where Python chokes, and 
> where you want to have the complexity, hassle and expense of a 
> managed cluster

BigQuery is on demand and $5/TB for processing. BigQuery and 
Google Cloud Dataflow look like reasonable alternatives to 
evaluate IMO.

>  Also, in this environment, it's so much easier to prototype 
> something where one doesn't require justifying a budget than to 
> have to do it in an industrial style at vast scale from day one.

Maybe. An external query engine sounds like the better solution 
for prototyping to me, though.

> I'll leave it there as per Andrei's request about focusing on 
> the hackathon.

Oh well, I don't think these debates have any effect on that. 
People probably have an ability to set their own priorities 
whether it is family or hackatons.