D: A language without focus

[gabe]

In article <e2pgfg$4s3$1 at digitaldaemon.com>, nick says...
>
>Sounds to me as though you are really eager to have D1.0 right this
>moment and you are frustrated with all the things that aren't working
>just the way you like them.
>

Well, you're only partially right about that.  I do have some dissapointment
that things don't exactly 'work'.  I think that anything that doesn't just
'work' for people coming to the language (or even working in it for a long
time), tends to make D look inflexible and amaturish compared to other efforts.
That is not to say that D is in any way either of those things -- I think it has
the potential to be very robust and quite user-friendly, but in order for the
language to really succeed, you're going to have to convince more than just me.
In order to succeed, you're going to have to convince all the other beginning,
10 cent hackers out there that D is a language of the future -- that it HAS a
future -- at the center of a stable and growing community.  All this feeds into
the notions that I outlined in my first message.  Design a system -- not just
part of the language -- to foster growth and stability, and people will search
YOU out, and not the other way around.  The keys to that system revolves around
finding what is necessary within the community and then designing toward those
goals.

So, with that in mind, let me offer up some ideas that I have been ruminating
upon.

First: The Build Mechanism
I've tried to play around a little with Build, and I think to a certain extent
it's a fine tool.  What it lacks, I feel is not so much the ability to compile a
series of programs but to really bring the end product together for users.  To
clarify: what build doesn't really seem to do is build in user-friendliness into
the libraries.  To a large extent, I think this more the fault of documentation
than of Build.  In my idea of a build tool, the documentation is far more
succinct and far more powerful; it leverages the native unicode support of D
against XML.  Now, I know XML is everyone's favorite language to hate, as it
quickly becomes quite complicated if not done well, but I feel that by using a
basic and standardized syntax, D could quickly develop a usable (online)
documentation system that exists in real-time with the development code itself.
Allow me to elaborate with the following, very rough example

---D source---
/++doc.xml
<doc lang="en_us">
<author name="Danger Duck" date="01.12.07" liscense="GPLv3" />
<method>
<param>long sec_uid</param>
<desc>A fake variable passed to secure this message against flame</desc>
<return>A <link lib="util.fire.safety">FlameRetardantSuit</link> to protect user
against singing</return>
</method>
<example><![CDATA[[
{
Suit s = Suit.getRetardantSuit(my_long_security_id); /* pass in unique id*/
s.buttonUpItsGonnaGetHotInHere(); /* Lock yourself in the vault */
s.waitForAllClear(clear_handler); /* Whew!, saved my tail feathers again! */
s.divest(); /* release the lock */
}
]]>
</example>
<tutorial><![CDATA[
You have to know what you're doing when you use this object.  Put it on too
soon, and you may overheat and cook your own goose.  Put it on too late, and
those henpecked farmers may get your gizzard.  As you can see from this example
<example ref="inline" /> the feathers are really flying.  Use it carefully,
though, and it'll all be like water off a duck's back.

I apologize for all the fowl phrases.
]]>
</tutorial>
</doc>
<doc lang="fr_fr">
<method nom_de_poulet="Danger Duck" date="01.12.07" permet="GPLv3">
<param>long sec_uid</param>
<usage>Parlez le francais mieux de moi</usage>
<rentre>Quelquechose de 'honk'</rentre>
<exemple>
..etc., etc., etc., ...
</doc>
+/
public FlameRetardantSuit getFlameRetardantSuit(long sec_uid) {
...
}

--- end D ---
As you can see, it's possible to implement documentation at a level that can be
directly transformed into any number of kinds of documentation.   You could
publish the API and the examples on the Web; you could publish a library
reference in PDF; you could distribute a localized tutorial in every language
imaginable.  The possibilities, frankly, for brining in people form around the
world are virtually limitless.  I know that this probably seems like way too
much documentation for a single method, but I think the idea is essentially
there -- it just has to be mined efffectively.

Second:
I've heard back from a few people about having a small core library.  I can
certainly respect that: coming from Java, I know what it feels like to have a
glut of options.  On the other hand, one of the undeniable draws for a language
is a substantial library of well-tested code.  The functionality of a language
is, in this sense, directly reflected in the composition of its libraries.  To
that end, I would propose a two-tiered solution.  On the first tier, there sits
two basic libraries: 'lang' and 'sys'.  Unlike Java, which hides much of the
implementation details of the system, D users should be allowed the freedom to
deal directly with the hardware and the underlying system when needed.  For this
reason, the sys and the lang packages will supply all of the basic functionality
needed for D to run in any environment.  This should especially appeal to
systems programmers and library implementers, as one can hide a good deal of
existing c code behind some carefully constructed interfaces or superior
packaging.  The 'lang' package, in particular, needs the 'sys' libs in order to
deliver all of the low-level coding required by each operating system, thus, I
believe, releiving even some of the lower details from language implementors.
It should be the case that these two packages are automatically searched and
compiled into any running program, though the top level DPATH may only refer to
'/usr/include/d/', or whatnot.  

The second part of this teir delivers everything else that the core library
lacks, including complicated IO, data structures, utilities, ui components, and
parsers.  I've given a rough outline of what the library might looklike below.
Of course, much of my model liberally samples (i.e. outright steals) from the
Java and C# library implementations.  This is the way I feel that it should be
-- each language should stand on the shoulders of the one that came before it --
and I think each of those languages are quite good at packaging together
disparate pieces.  Of course, we wouldn't import from d.xxx -- we can drop the
'd.' syntax entirely once the compiler supports native library builds; I always
did resent the whole 'import java....' fiasco.  This seems to be an ok solution
as long as we make sure that we don't conflict with URL domains in these
top-level directories.  

d/
------ BRL - Basic Runtime Library ------
lang
gc
exception
thread
primitives
array
bitarray
string
box
hashmap
compiler
..

sys 
console (for some basic io)
memory
call (hiding system calls, I thought)
lib  (Windows lib, glibc, ... what have you)
..

-------------------------------------------

----- ERL - Extended Runtime Library ------

ext
c
stdio
math
fenv
process
stdarg
stddef
stdlib
string
time
..
c++
..

data (collections/standard data structures, probably via templates)
ArrayList
Stack
Queue
LinkedList
DoublyLinkedList
CircularlyLinkedList
BinaryTree
Map
..

util
compress (protocols for bzip, gzip, tar etc.)
logging
date
random
..

os (or 'kaos', for nameing conflicts [kernel and o/s] -- hsa mythic overtones)
posix
c
windows
c
..
com
..
dotnet
linux
c
..
kernel (???)
..
userland (???)
titan
minix
..

security
crypt
md5
sha1
..
dgp (a port of gnu pgp might be very helpful, if signing mail, for instance)
invoke (secure calls to the system? 'Titan' might find this useful)
ssl
key

math
complex
fourier
boolean
..

ui 
curses
thea (D language graphical system based on the qt framework)
opengl
window
..
..

text
encoding
unicode
ansi
codepage
locale
format
regex
parsers
CVS
yaml
yacc (?)
..

sql
dbi (database interface abstraction)
vendor (specific vendor APIs)
slumber (hibernate or ado api?)
..

io  (or 'dio', to avoid any conflicts)
file
buffer
pipe
stream
..

inet (to deal with local imports like net.chickenplucker.geetar.blah)
socket
server
http
udp
ftp
mail
encoding
protocols
ident
url
uri
..
reactor
svn
..

xml
parsers
validators
xpath
xslt
rpc
soap
..

..

----------------------------------------

One last thing before I slip off to bed: I really want to rally around an open
source license for this project, particularly one that strives for complete
transparency.  Not only does this really promote a sense of sharing and
community, but I feel that it also gives newcomers and old hackers some ideals
to really identify with.  To that end I would suggest the GPL as the starting
point -- possibly even liscenesed under some kind of dual GPL/LGPL liscense.
This kind of transparency, I feel, really invites people to the table in an
egalitarian and communal way.  (Not to mention the fact that putting it under
the GPL opens up the floodworks for linking into other GPL code -- and there's
tons of it.)

I'm sure that I've forgotten a few things in all this, and I think that there's
so much more to say, but I'm tired so I wish you all a fond goodnight and I look
forward to reading your responses in the morning.

Quack!
Gabe