D: A language without focus

[gabe]

I've been reading these posts for a while now, and going over the documentation,
and discovering some of the tools, and a few things have jumped out at me about
the D language and the D community:

Arrow One: The Message
The reason one language succeeds and another language fails has everything to do
with the marketing of the language. Look at Java -- there's a language that
didn't work for s!#& when it first came out -- it was clunky, slow, and
uncomfortable for people who knew what a computer did and wanted the computer to
do just that (and only that). And yet, Java has succeeded, not only through
their bankroll, but through the packaging of what they do -- providing a unified
and coherent source of information, documentation, tutorials, compiler,
debugger, build tools, packaging tools, networking tools, and system analysis
tools. Each one of these things is a singular effort, unremarkable on the whole,
but packaged together, they provide a compelling and remarkable interface for
programmers desperately looking to combine functionality with ease of use. On a
similar note, look at Python: there's a language with decent libraries,
excellent syntactical clarity, tons of documentation and tutorials, and an open
source license that pleases the hackers while not scaring away businesses with
implementation worries -- if something breaks, they know they have the community
and their own resources to fix it.

None of those things can really be said for D. First, you don't have a fully
functional, stress-tested compiler. Second, you have poor documentation, and
what documentation you do have is either written tersely, poorly, or scattered
to hell-and-gone. Third, you have absolutely no focus: there are dozens of
projects out there planning to build the 'next big utility/tool/ide/whatever',
but hardly any of them seem to be terribly concerned with the fact that D's
libraries rest on pretty shakey ground. In short, the D library is a joke. Have
any of you perused the glibc recently, or the java.* packages? Notice anything
interesting about these projects? If you thought something like 'this has a
stunning wealth of library functionality and documentation', you'll probably see
where I'm going with this. We're not talking about advanced JavaBeans and
ServerFaces, gtkhtml+, or any one of a dozen other amazingly diverse and
specialized fields -- we're talking about the bare essentials here. And nothing
is more important for a language's success than those primary libraries. You can
go over to the 'Ares' forum, where I posted a comment on how a potential library
should be structured. There has to be an interface for programmers, and it's up
to the D community to provide those fundamental interpretations and guidelines.

Arrow Two: Liscenses.
It's all about the liscenses. Take a look around the programming world and look
at the most active developer communities -- the Linux kernel, the Apache
project, the Eclipse project. Each one of these communities shares one startling
thing in common: they are all, from front to back, from first line to last, open
source. Please, focus on this point, because I cannot stress enough the
importance of what this kind of community support can do for a language (PHP, as
people have noted, went from a poor backwater to an IBM flagship in a matter of
3 years through just this same kind of open source initiative). The problem with
D, as I see it, is the ambiguous and somewhat offputting liscensing terms of the
D libraries and compilers.

One of the things that I think a lot of people miss about these open source
projects -- where their strength really lies -- is their appeal to those kinds
of people that adopt whole heartedly the notion of freedom. Freedom to do what
you want when you want with the source code that you have. This kind of freedom
appeals to a huge mass of hackers out there -- hackers who have learned and
weaned themselves on the open source projects of Linux, GNU, and Apache. I think
that's something that most people coming from the Windows side miss when
thinking about this issue. Microsoft, whatever your opinion of it, has trained a
raft of people into beleiving that the only way you make money on something is
by keeping everything to yourself and trying to kill your opponent through
technical starvation. It has led to a 'race condition', if you will, where
everyone wants access but everybody has to wait for the BigGuy to say it's ok. I
think that the point of the D language should be instead to spread it as much as
possible, as freely as possible, to everyone, so that they can see for
themselves how great it really is.

Don't get me wrong -- I don't necessarily hate Microsoft -- but I do think that
a lot of people who use Microsoft have been bamboozled into thinking that the
only way that they're going to acheive commercial success is by assuring
everyone that you're working with that everything will remain strictly
confidential. Secrecy and the ability to deny innovation become the order of the
day. I would merely like to point out that companies like RedHat and Novell, who
have clearly adopted the OSI/FSF mentality, are thriving companies, not least of
all because they acknowledge that sharing is by far the quickest route to
success.

Arrow Third: Solidarity and conformance.

Part of any projects success is it's ability to institude order among a chaotic
group of competing individuals and ideals. This is no less true in D, where a
number of libraries have developed vague interdependancies and marginal or
somewhat disparate coding methodologies. One of the successes of many projects
can be linked back to their development to a uniform interface to outside
consumers. Take Ruby as an example. Initially, it wasn't a very successful
language, on the whole. It struggled in relative obscurity for nearly 7 years
before books and articles started to be written about it. Then, all of a sudden,
it seemed like, something shifted and the community started to amass new
members, contributions increased, and it managed to get over the versioning hump
into 1.8. How can we explain this success? There are a number of explanations, I
believe, but the one that I think speaks best to D's current situation is it's
development of coherent documentation and coding style guidelines. Look at the
Ruby specification for 1.6, and then look at it for 1.8. If you notice anything,
you should notice that the technical guidelines for what was and was not
acceptable coding style became clear -- in essence, language purists began
exercising control over not only the development but the day-to-day use of the
language. Perlisms began to gradually drop out of the language, to be replaced
with more 'idomatic ruby' (i.e., generators and iterators). Also, the coding of
documentation went through the roof -- take a look at www.ruby-doc.org and see
what proper documentation of methods actually looks like. They have everything
that a person could need -- not only an API and documentation, but EXAMPLES OF
USAGE that make how to use the API absolutely clear. Hard as I try, I can't
really say that I've seen the same in D. (For the record, you should also note
Python, started out as a teaching language, and had from the beginning excellent
documentation and EXAMPLES of usage, examples which only got better and more
advanced as the libraries themselves were refined.) Now, all of this isn't to
say that you should write a dozen tutorials per method when you're writing
classes, but there needs to be a quick and coherent way to organize your
information so that other people have more than just method declarations to use.
This leads me to my biggest and most overarching concern as to why D will never
take off as a language -- nobody knows how to write good documentation. And it
doesn't help that the D documentation guidelines aren't that good. Frankly, it
should be much clearer what constitutes a comment and what doesn't -- as well as
what should be included where when creating documentation files. The whole hash
pair of "XXX: xxx" doesn't really work for me -- or for a lexical parser in
creating complex and properly formatted documents, really. If you want to make D
a more palatable language, it's time that it grow up a little and start
accepting the complexities of coding 'in the round', as it were. We need to
embrace the Unicode roots of D source files and begin to plan for XML document
creation. We need to find/create a parser that will work on XML-D documents and
create the appropriate documentation -- be they xhtml docs, pdfs, dvis --
whatever. D needs to learn the cardinal rule of building a language library: if
people can't understand and easily assimilate exactly what it is you want to do,
they're not going to care.

Exclamation Solutions:
I would like to get involved in the D community in developing the tools that
will make D a truly great language. To make that possible, however, we need to
add a centralizing force and focus to the disparate goals of the project. First,
we need to design some kind of mission statement -- something that uniquely
defines our goals as a community and as a group of programmers devoted to making
D a better language for everyone to use. Second, we need to decide on some goals
-- goals that will help us realize our mission by making it easer for people to
come to D from other languages (or even there first language!). Third, we need
to organize ourselves into groups that can work on aspects of the mission and
can communicate easily to one another. This means that we need redundancy in
projects (so that if someone leaves, someone else can take over),
democratization across the group (so that API's and tools can have community
input), and distributied responsibilites for all members (so that each person
feels that they belong through to the group -- not merely because they say they
do, but because they have earned the respect and the trust of their fellow
members). In short, D should be a language where the community that bands around
it is one of openness, respect, and excited collaboration.

I would like to post some more ideas about this, and I'd like to start
organizing those people who are interested in joining together. I'd like to hear
back from the community about this. I'd like to hear back from people excited
about making D the best possible language it can be -- and making programming in
D an absolute joy.

I look forward to hearing from you! 
_________________
New to the land of D, but I like the look of the place.
-Gabe