Patches, bottlenecks, OpenSource

[Spacen Jasset]

bearophile wrote:
> After the last posts about patches, I can write something myself about this topic :-) I am far from being an expert of big software projects, but I think I am now able to understand some things of the D project.
> 
> I like D, it's an alive project, but from what I've seen D so far is not having a so significant success. I think lot of people (especially young programmers) don't want "a better C++", they want no stinking C++ at all. They want something quite different. (On the other hand now I am not so sure Scala will have a widespread success, its type system makes it not easy to learn).
> 
> A C++-class language compiler is a big project that requires quite different skill sets: in the D compiler there is work to do on the vector operations, lent/uniqueness, multi-core, 64 bit, work for keep it well updated on Mac/Linux/Win and well packaged, development of Phobos and its data structures, work on the back-end to use the SSE registers and future CPUs or GPUs, tuning of the associative arrays, future improvements of the core language, improvements in the unit test system, large improvement in the D web site and the documentation, big improvements needed for the GC to turn it into something more modern, and so on. Such things have to be refined, debugged, efficient, cared of, polished.
> 
> So as D develops and grows it will get hard for a single person to write all patches to the language and other parts. As D grows this will become a bottleneck, and from what I've seen it already is. The disadvantage of allowing other people the rights to patch the front-end is some lost control, but Walter can review patches after they are already applied & working (in a big project having a director is positive). This can speed up the patching process itself. The compiler can become more like a patchwork of different programming styles, and Walter has to ask to other people how some parts not written by him work. This can be doable if the front-end becomes more modular. LLVM shows a more modular design that can be copied. This is a lot of refractoring work.
> 
> Increasing openess can increase the influx of devs and their willingness to help the project. This is quite positive and its importance can't be underestimated.
> 
> Pyrex (http://www.cosc.canterbury.ac.nz/greg.ewing/python/Pyrex/ ) was forked in Cython (http://www.cython.org/ ) because Pyrex author was too much slow in patching. A fork will not happen in D soon because there are not enough people yet that care and work for D enough to make a fork viable. D is not a project as alive as Pyrex. Cython is a tiny rib of the Python language circus.
> 
> I suggest to give patching rights to Don, it can be one more step forward toward Open Sourcing D development style :-) In future such right can be given to other few good people that have shown to deserve enough trust.
> 
> I am not sure the future of D is in the dmd back-end, llvm looks in better shape for this purpose. LLVM is able to support 99.9% of the features of D2 (and I presume llvm devs are willing to add the few missing things if someone gives them patches; from the unladen-swallow project I have seen they are open enough), and it supports a large number of good features currently not done/present by/in the back-end of dmd. Currently ldc is a compiler much better than dmd (try it if you don't believe me!), but its devs are not appreciating D2 language yet much. I don't like to see the Walter-locomotive so detached from the present nice ldc D1 compiler, and from the future of LDC and llvm. I'd still like ldc to become the official D2 compiler, developed and cared of :-)
> 
> Bye,
> bearophile

It's all true. I also think we should use a DVCS too so that Walter can 
apply patches easily and we can upload patch branches directly to 
somewhere that can be merged and or reconciled by the original authors 
in case of conflict. It's all good. The history of the patch itself can 
also be tracked even after merging. I suggest Bazaar but I am deeply 
biased on the matter.

Further, Don et al. could act as gatekeeper to prevent Walter getting 
bogged down with too many patches. And then things will be all good and 
I may find the time to get interested in D again.