DConf 2013 Closing Keynote: Quo Vadis by Andrei Alexandrescu

[Joakim]

On Wednesday, 26 June 2013 at 11:08:17 UTC, Leandro Lucarella 
wrote:
> Joakim, el 25 de June a las 23:37 me escribiste:
>> I don't know the views of the key contributors, but I wonder 
>> if they
>> would have such a knee-jerk reaction against any paid/closed 
>> work.
>
> Against being paid no, against being closed YES. Please don't 
> even think
> about it. It was a hell of a ride trying to make D more open to 
> step back now.
I suggest you read my original post more carefully.  I have not 
suggested closing up the entire D toolchain, as you seem to 
imply.  I have suggested working on optimization patches in a 
closed-source manner and providing two versions of the D 
compiler: one that is faster, closed, and paid, with these 
optimization patches, another that is slower, open, and free, 
without the optimization patches.

Over time, the optimization patches are merged back to the free 
branch, so that the funding from the closed compiler makes even 
the free compiler faster, but only after some delay so that users 
who value performance will actually pay for the closed compiler.  
There can be a hard time limit, say nine months, so that you know 
any closed patches from nine months back will be opened and 
applied to the free compiler.  I suspect that the money will be 
good enough so that any bugfixes or features added by the closed 
developers will be added to the free compiler right away, with no 
delay.

> What we need is companies paying to people to improve the
> compiler and toolchain. This is slowly starting to happen, in
> Sociomantic we are already 2 people dedicating some time to 
> improve D as
> part of our job (Don and me).
Thanks for the work that you and Don have done with Sociomantic.  
Why do you think more companies don't do this?  My point is that 
if there were money coming in from a paid compiler, Walter could 
fund even more such work.

> We need more of this, and to get this, we need companies to 
> start using
> D, and to get this, we need professionalism (I agree 100% with 
> Andrei on
> this one). Is a bootstrap effort, and is not like volunteers 
> need more
> time to be professional, is just that you have to want to make 
> the jump.
I think this ignores the decades-long history we have with open 
source software by now.  It is not merely "wanting to make the 
jump," most volunteers simply do not want to do painful tasks 
like writing documentation or cannot put as much time into 
development when no money is coming in.  Simply saying "We have 
to try harder to be professional" seems naive to me.

> I think is way better to do less stuff but with higher quality, 
> nobody
> is asking people for more time, is just changing the focus a 
> bit, at
> least for some time. Again, this is only bootstrapping, and is 
> always
> hard and painful. We need to make the jump to make companies 
> comfortable
> using D, then things will start rolling by themselves.
If I understand your story right, the volunteers need to put a 
lot of effort into "bootstrapping" the project to be more 
professional, companies will see this and jump in, then they fund 
development from then on out?  It's possible, but is there any 
example you have in mind?  The languages that go this completely 
FOSS route tend not to have as much adoption as those with closed 
implementations, like C++.

> First of all, your examples are completely wrong. The projects 
> you are
> mentioning are 100% free, with no closed components (except for
> components done by third-party).
You are misstating what I said: I said "commercial," not 
"closed," and gave different examples of commercial models.  But 
lets look at them.

> Your examples are just reinforcing what
> I say above. Linux is completely GPL, so it's not even only 
> open source.
> Is Free Software, meaning the license if more restrictive than, 
> for
> example, phobos. This means is harder to adopt by companies and 
> you
> can't possibly change it in a closed way if you want to 
> distribute
> a binary.
And yet the linux kernel ships with many binary blobs, almost all 
the time.  I don't know how they legally do it, considering the 
GPL, yet it is much more common to run a kernel with binary blobs 
than a purely FOSS version.  The vast majority of linux installs 
are due to Android and every single one has significant binary 
blobs and closed-source modifications to the Android source, 
which is allowed since most of Android is under the more liberal 
Apache license, with only the linux kernel under the GPL.

Again, I don't know how they get away with all the binary drivers 
in the kernel, perhaps that is a grey area with the GPL.  For 
example, even the most open source Android devices, the Nexus 
devices sold directly by Google and running stock Android, have 
many binary blobs:

https://developers.google.com/android/nexus/drivers

Other than Android, linux is really only popular on servers, 
where you can "change it in a closed way" because you are not 
"distributing a binary."  Google takes advantage of this to run 
linux on a million servers powering their search engine, but does 
not release the proprietary patches for their linux kernel.

So if one looks at linux in any detail, hybrid models are more 
the norm than the exception, even with the GPL. :)

> Same for C++, which is not a project, is a standards, but the
> most successful and widespread compiler, GCC, not only is free, 
> is the
> battle horse of free software, of the GNU project and created 
> by the
> most extremist free software advocate ever.
D is not just a project but a standard also.  I wouldn't say gcc 
is the "most successful and widespread compiler," that's probably 
Microsoft's compiler, since Windows market share is still much 
more than linux.  But yes, gcc is a very popular open-source 
implementation: I didn't say there wouldn't be an open-source D 
compiler also.  But I don't think C++ would be where it is today 
if only open-source implementations like gcc supported it, which 
is the case today with D.

> Android might be the only
> valid case (but I'm not really familiar with Android model), 
> but the
> kernel, since is based on Linux, has to have the source code 
> when
> released. Maybe the drivers are closed source.
As I said earlier, most devices' drivers are almost always closed 
and the non-GPL parts of Android, which are the majority, are 
usually customized and the source is usually not released, 
because most of Android is Apache-licensed.  I think this closed 
option is a key reason for the success of Android.  Hell, the 
hardware vendors would never have adopted Android if not for 
this, as Google well knew.

> You are missing more closely related projects, like Python, 
> Haskel,
> Ruby, Perl, and probably 90% of the newish programming 
> languages, which
> are all 100% open source. And very successful I might say. The 
> key is
> always breaking into the corporate ground and make those 
> corporations
> contribute.
I believe all of these projects have commercial implementations, 
with the possible exception of Haskell.  Still, all of them 
combined have much less market share than C++, possibly because 
they use the weaker consulting/support commercial model most of 
the time.  One of the main reasons C++ is much more popular is 
that it has very high-performance closed implementations, do you 
disagree?  I'm suggesting D will need something similar to get as 
popular.

> There are valid examples of project using hybrid models but 
> they are
> usually software as a service models, not very applicable to
> a compiler/language, like Wordpress, or other web applications. 
> Other
> valid examples are MySQL, or QT I think used an hybrid model at 
> least
> once. Lots of them died and were resurrected as 100% free 
> projects, like
> StarOffice -> OpenOffice -> LibreOffice.
There are all kinds of hybrid models out there, some would work 
for compilers also.  I think it's instructive that you are 
listing some of the largest and most successful, mostly-OSS 
projects in this list. :)

> And finally making the *optimizer* (or some optimizations) 
> closed will
> be hardly a good business, being that there are 2 other 
> backends out
> there that usually kicks DMD backend ass already, so people 
> needing more
> speed will probably just switch to gdc or ldc.
Let me turn this argument around on you: if there is always 
competition from ldc and gdc, why are you so scared of another 
option of a slightly-closed, paid compiler?  If it's not "a good 
business," it will fail and go away.  I think it would be very 
successful.

> As in breaking into the commercial world? Then agreed. If you 
> imply
> commercial == closing some parts of the source, then I think 
> you are WAY
> OFF.
OK, so it looks like you are fine with commercial models that 
keep all the source open, but not with those that close _any_ of 
the source.

The problem is that your favored consulting or support models are 
much weaker business models than a product model, which is much 
of the reason why Microsoft still makes almost two orders of 
magnitude more revenue with their software products than Red Hat 
makes with their consulting/support model.

I am suggesting a unique hybrid product model because I think it 
will bring in the most money for the least discomfort.  That 
ratio is one that D developers often talk about optimizing in 
technical terms, I'm suggesting the same in business terms. :)

>> It is amazing how far D has gotten with no business model: 
>> money
>> certainly isn't everything.  But it is probably impossible to 
>> get to
>> a million users or offer professionalism without commercial
>> implementations.
>
> Yeah, right, probably Python and Ruby have only 5k users...
>
> This argument is BS.
First off, they both have commercial implementations.  Second, 
they still only have a small fraction of the share as C++: part 
of this is probably because they don't have as many closed, 
performant implementations as C++ does.

I realize this is a religious issue for some people and they 
cannot be convinced.  In a complex, emerging field like this, it 
is easy to claim that if OSS projects just try harder, they can 
succeed.  But after two decades, it has never happened, without 
stepping back and employing a hybrid model.

I have examined the evidence and presented arguments for those 
who are willing to listen, as I'm just about pragmatically using 
whatever model works best.  I think recent history has shown that 
hybrid models work very well, possibly the best. :)