[OT] compiler optimisations

[via Digitalmars-d]

On Saturday, 25 April 2015 at 14:48:41 UTC, Laeeth Isharc wrote:
>> I find it worrying that the evangelical D users are perceiving 
>> D as a compiled scripting language and claim it is similar to 
>> Python... D semantics are not at all like Python. That can't 
>> win.
>
> Why does it worry you?  What bad things will happen?

Bad things that could happen is that D never can be like Python 
and if you try to make it such you no longer have a system 
programming contender.

> questions, but I think your argument would be more effective if 
> you explained why shipping vibe.d somehow detracts from D's

Because it shifts the focus towards an application area where D 
will have trouble to gain significant ground. That means the 
language will be evaluated up to that application area.

There is a limit in the market as new projects will gravitate 
towards the most promising language in their application area. 
And there are many languages pitching in the web domain.

> Really?  You have a man with the expertise and experience of 
> Walter Bright devoting his time to rewriting string processing 
> parts of the standard library himself, in service of the goal

Which essentially is escapism from a language development point 
of view. Languages are not judged by their libraries, unless they 
lack functionality due to flaws in language semantics.

> needs are ignored?  (Not that games need strings, but that 
> either a library is GC free or it isn't, and this is something 
> games people seem to care about).

@nogc was a good addition. If D is the best option in an 
application domain where you have long running projects people 
will build frameworks for it that covers the ground that the 
existing libraries are missing.

I have no concerns related to Phobos whatsoever. It is 
inconsequential.

This is different in a scripting language which often is used in 
contexts where you cannot predict your needs ahead of time. I.e. 
you are prototyping and are exploring new directions or are just 
covering your needs day by day. If you are doing that in a long 
running predictable project you are in a bad shape (aka fire 
fighting).

> Plus all the work on refcounting etc.  I am sure there are many 
> other aspects, and games themselves don't interest me, but that 
> doesn't strike me as a balanced perspective.

Games is just something that is being brought up because people 
interested in it come to D looking for something less tedious 
than C++. It is just an exemplar of system level programming 
(when the games run after loading). You could say the same things 
about interactive audio software, embedded programming, memory 
constrained high throughput servers etc.

> It's odd to mention D's role as a systems language without 
> discussing its use in embedded systems.  The pioneer who spoke 
> at dconf a couple years back undertook a valiant effort, but it 
> was too much for him to manage in one go.  (And of course Adam 
> Ruppe's highly entertaining presentation on bare metal 
> programming).

Adam is a great guy, but he is probably more patient than most 
with figuring out workarounds ;-).

> Similarly the work on ARM/Android/iOS, which seems to be coming 
> along.

Maybe, I do iOS work and it is very convenient to just use 
Objective-C++ everywhere I need something that cannot be done in 
C++. Add to this that Apple keeps mutating their libraries and 
Apples IDE becomes kind of irreplacable. You need something a lot 
better than C++ to encourage a switch there...

>> There is a need to move towards something beautiful, and 
>> that's not in Andrei's vision, but in the original D1 vision + 
>> the improvements proposed by Bearophile, Timon Gehr and others.
>
> I appreciate you may not have time, but if you had any links to 
> stuff if they are gathered in documents rather than myriad 
> fragments, I would be curious to see.

I don't think so, but it is mostly a fairly standard stance about 
programming language ideals. (Which C does not adhere to, and D 
leans heavily on C.)

> It's not for me to talk about strategy.  But it strikes me that 
> you are calling for a further massive shift, when people have 
> their plates too full already.

Not at all. I've argued that D2 should stay with the GC, and 
focus on doing what it does really well, basically catering to 
the market I think you are in. Changing the semantics slightly so 
that you only touch the cachelines that need to be traced when 
scanning live objects.

Then rework the memory model, which is a lot of work if done 
well, to a D3 version of the language. Fudging it with reference 
counting hacks makes D not very attractive beyond "compiled 
scripting", but "compiled scripting" is better off with a good GC 
than unmanaged memory handling and ref counting by default...

So the proposed solutions have a very low potential for 
increasing market share. In fact some of the proposed changes 
would probably make the language hard to analyze which has a bad 
effect on future tooling and a programmer's ability to keep a 
sane model of the language in his head. C++ combats this with 
good IDEs (that complain when you forgot to add "typename" or 
".template" in a templated method call (rather ugly). That's 
obviously just an emergency solution for a language that's beyond 
"cleaning up". And it's where D is heading, not in one go, but 
drip, drip, drip... like C++.

> circulation of the elites.  In other words the top dog is not 
> static - this applies to income of a relatively free-market

I takes ~10 years for a language to get big, so it is not like we 
will be overrun by surprises anytime soon...

However the "winner takes it all" effect has become a lot 
stronger now that you have so many excellent free libraries. When 
you had to purchase your libraries the situation was way 
different. In the 80s you could fund a decent dev environment, 
but that gets harder and harder unless you own the platform (like 
Apple). Critical mass is a very strong phenomenon in this domain. 
To the extent that the commercial sector has more or less given 
up and sell IDEs only. If this was not so, the commercial sector 
would be much more active in language development (inventing new 
languages).

> You might as well have said To Honda, Toyota, Nissan, Hyundai 
> etc that "it's a winner takes all game" when their products

Nah, there is plenty of commercial activity in the car industry.

> this used to cost $100k pa plus.  So data sets keep growing - 
> CPU performance continues to improve, but in a less convenient 
> way, but as I understand it memory perf lags.  Which means in 
> the future you may be increasingly irritated by people speaking 
> of using D for scripting purposes...

I'm not irritated by it. It just does not represent system level 
programming, so unless D stops claiming to be a system level 
programming language (like Go) it should not be the primary long 
term target. Pervasive reference counting is a scripting language 
solution.

System level programming means you control memory layout, memory 
usage etc. For instance, in my current project it would perhaps 
be easier to use ref counting, but since I generate/load arrays 
that could easily consume 40% of the memory I better be sure that 
the memory is released before loading the next set. Otherwise 
real time performance will suffer (audio playback).

That level of control is overkill for processing historical data, 
but the most promising solution for that application area are 
distributed cloud solutions. Like Google Big Query that AFAIK can 
do brute force SQLish quries over very large datasets fast (using 
some kind of built in query optimization).

Cheers!