Is all this Invarient **** er... stuff, premature optimisation?

[Sean Kelly]

== Quote from Me Here (p9e883002 at sneakemail.com)'s article
> Sean Kelly wrote:
> > == Quote from Me Here (p9e883002 at sneakemail.com)'s article
> > > Phobos vs. Tango
> > > I definitely don't want the dead weight of pointless OO wrappers or deeply
> > > nested hierarchies. Nor the "everything must be OO" philosophy.
> > > Once I regain access to std.string for my char[]s, (and a simple,
> > > expectation conformant rand() function :), I'll be happy.
> >
> > Please don't discount Tango based on what has been said about it in this
> > forum.  I know for a fact that Walter, for example, has never even looked
> > at Tango (or he hadn't as of a few weeks ago anyway).  In truth, the percentage
> > of classes to functions in Tango is roughly the same as in Phobos... Tango is
> > just a much larger library.  If you're interested in algorithms and string operations,
> > I suggest looking at tango.core.Array and tango.text.*.  The former is basically
> > C++'s <algorithm> retooled for D arrays, and the latter holds all the string-specific
> > routines in Tango.
> >
> The primary basis of my immediate decision regarding Tango was it incompatibility
> with Phobos as outlined in
>     http://www.d-riven.com/index.cgi?tango-phobos
> (And several other first page hits when googling for "D Tango Phobos")

For what it's worth, the "Tangobos" project is a port of Phobos to the Tango runtime,
and a pre-packaged version is available on the Tango website.  If you compare the
source code with Phobos itself, you'll find that there are precious few diffs anywhere
in the entire package, and the few that exist are mostly in std.thread.  So this may be
an option if you'd like to use both together.

> Beyond that, I'm in favour of OO when only when it truly benefits me. That is,
> when it manages state that I would otherwise *have* to manage myself.
> That, for example, does not mean simply substituting an object handle for an OS handle.
> Nor caching of derived values unless their derivation is truly expensive.
> Nor the use of getters and setters to avoid direct manipulation of attributes,
> unless there is some genuine value-add from doing so.

That's the basic philosophy behind Tango.  In fact, the bulk of the objects in Tango are
in the IO package, with much of the remainder being in places where polymorphism is
desirable (localization, for example).  If you find an object in Tango that has no actual
state information, it's generally packages as a class for this reason.

> OO-dogma that they will isolate the library from speculative future changes in the
> underlying OS calls (that have been fixed in stone for 1 or4 decades or more)
> do not cut much ice with me.

Me either.  However, since Tango is portable across Win32 and Posix systems (currently),
I do think an argument could be made for some level of abstraction.  But the C API headers
are available as well if you really want to use them.

> I'm also not fond of all-in-one library packaging. Seems to me that there is enough
> information in the source code to allow libraries to be packaged as discrete dlls/sos
> and to only statically link against those required. But that may be a tool chain problem
> rather than anything to do with Tango.

This was actually driven by fairly vocal user request.  The original conception was for Tango
to be a lightweight, modular framework to be extended by users rather than a monolithic
library.  In fact, we didn't even distribute an all-in-one prebuilt library for Tango until
sometime last summer.  Before that we expected that a tool such as Bud or Rebuild would
be used.  This is still quite possible however, and the modular design in terms of code
dependency is still in place.  If you choose to toss the tango-user lib altogether and find
you want even more modularity, I suggest looking at this page:

http://dsource.org/projects/tango/wiki/TopicAdvancedConfiguration

As far as I know, I'm the only one that has actually read it so it's a bit out of date (the library
names are wrong), but the basic concept still applies.  That is, the choice of a GC can be made
at link-time with Tango, and other portions of the runtime are easily replaceable as well.  Some
kernel projects have found this useful in the past.

> It should be possible to substitute one implementation of a std.* library for another,
> without it being an all or nothing change. I should be able mix'n'match between
> implementations of std.* packages.
> For example, with the std.string problem I've been having. If I use
>     import std.string;
>     char[] a = readln();
>     a.toupper();
> it should work. If I do
>     import std.string.immutable;
> it wouldn't.

Agreed.  My biggest complaint here is having to maintain two essentially identical packages,
assuming I were to do such a thing.  This is why I find Steven's "scoped const" proposal so
attractive.

> One of the things that force me to go away from D a couple of years ago was the ever changing state of the
> libraries. Not the internal, implementations or occasional bugs, but the constantly changing interface definitions.
> It becomes impractical to develop a major project when you're constantly rewriting major chunks of code to accommodate
> the latest set of group think on the best way to package the OS and "C lib" functionality.
> Back then, I put it down to the necessary gestation of a new language, and moved away to get my project done.
> I've now come back and find that the same situation exists. The answer to an essentially trivial problem is to write
> and entire new library. Or rather, since the library I need was already a part of Phobos with D 0.-something
> resurrect and old library.
> And that's the most worrying thing of all. The removal of the existing library from Phobos because the main proponents
> suddenly drank the cool aid of Invariant strings--especially for reasoning that I still find entirely specious--does not bode well
> for ongoing stability

The lack of responsiveness of the Phobos maintainers (ie. Walter) is what drove us to create
Tango in the first place.  I'll freely admit that the design of Tango has changed here and there
as we've moved through beta, but it's largely solidified now and will be frozen once we hit 1.0.
Neither I nor the other Tango developers have any desire to maintain deprecated code and the
like, so we've been doing our utmost to find a design that we hope will last.  Also, that there
is at least one commercial project based on Tango (I think there are actually more, but I don't
keep track of this very closely) says a lot about the library's stability and its support.

> I had hoped that during my two years away, that at least the interfaces would have become standardised,
> even if the implementations varied from release to release. But if whole chunks of functionality can suddenly
> disappear from the library, at the same time as major new chunks of very desirable functionality are added to the language,
> on the whim of 1 or 4 major players getting religion, then I'm really not sure that D is, or will ever be,
> ready for anything other than academic exploration of compiler technology.

To be honest, I actually feel much the same.  However, I also feel that D 1.0 is a fantastically
designed language overall, and I would choose it in a second over C or C++ (I'm a systems
programmer so those are really the only other choices available).  So at the end of the day, I
will be disappointed of the "future of D" takes a hard left-hand turn towards somewhere
I have no interest in going, but since I'm really quite happy with D 1.0 I won't shed too many
tears over it.  This perhaps doesn't bode terribly well for my use of D in the long-term, but
that's a bridge I'll jump off if and when I come to it.

> Reading that back. the independence of Tango begins to be more attractive, even if I have a distaste for the
> "everything must be OO" philosophy that (apparently) underlies it. Maybe I'll pull a copy and look for myself.
> For my current needs, I'm just looking for C speed with having to manage my own memory
> For the project I went away from D for 2 years ago, and came back hoping for stability, my own personal
> research project come memorial folly to be, I don't think D is yet ready for that. Maybe D1 if it doesn't
> becomes completely unsupported.

If it's speed you're looking for, Tango is it ;-)  The IO subsystem trounces pretty much everything
I've seen it compared to, for example.  In practice, I think you'll find that one reason for this is that
no hidden allocations take place anywhere in Tango.  This tends to conflict with convenient use in
some cases for simple apps however, so as things stand now I do think that some users would
benefit from convenience wrappers.  I tend to do this sort of thing myself for my own projects,
but a third-party package would be nice for those less inclined.  If you're interested in direct
performance comparisons however, I suggest reading the "benchmarks" links on this page:

http://dsource.org/projects/tango/wiki/Documentation

The XML tests in particular are pretty astounding (I feel comfortable saying that because I had
nothing to do with the development of that particular package :).

> In the interim I've "done the rounds" of an amazing variety of languages. From the functional brigade,
> Haskell, OCaml, Mozart/Oz, Erlang  et al. and various of the newer dynamic languages. They all have their
> attractions, but most are spoilt by some level of dogma. Haskell with is purity. Python with the whole
> significant whitespace thing. P6 with unix-first, and non-delivery.
> Mostly, the dynamics lack the low-level control and performance I need. I've been seriously working with
> structured assembler to achieve the low level control and performance I want, but doing everything yourself
> just takes you off into far to many interesting side projects. Implementing your own memory management
> could occupy a lifetime; especially if you consider the possibility and advantages of using (wait for it) a
> segmented architecture. Most older programmers memory of segmented memory stems from the 16-bit Intel
> days and they (almost) universally eschew any notions of it now a 32-bit (and 64-bit) flat memory models are available.
> But there are some very interesting possibilities in combining 32-bit segments and virtual memory.
> D is my last best hope of avoiding the assembler route and trying to do it all myself. Walter's pragmatism stood out
> in my early experience of both the language and library design--al be it that they kept changing;)--but I was really
> expecting (hoping) for greater stability by this point.

Personally, the combination I find most attractive for my work right now is Erlang backed by C or D for the
performance-critical work.  That gives me the easy parallelization I want, IPC, etc, plus an easy way of
optimizing the heck out of trouble spots... or simply sidestepping the strict functional model when data
sharing is actually needed.  I've actually come to feel that having a language separation here is a good thing
as well, because it prevents "bleed through" of concepts which I feel risks poisoning the efficacy of each
approach.

As for the rest, Kris, one of other Tango developers, has done a lot of work in the embedded space and
pushed very hard in the past for D to better support this style of programming.  He wasn't terribly successful
insofar as language/compiler development was concerned (there has been a lot of talk in the past about TypeInfo
in particular), but Tango, at least, was designed with embedded development in mind.  The lack of hidden DMA,
for example.  I can't say whether Tango will suit your needs, but it does seem to at least match your general
goals with D.

> Ooh. Did I write all that? Still. It has persuaded me to at least go look at Phobos, even if it is done with a jaundiced eye.
> A stable, even if philosophically distasteful, implementation of the staples is better than a philosophically desirable but
> whimsically changing one.
> Cheers for prompting me to re-think my blanket dismissal. b.

Thank you for reconsidering :-)  D may be a young language, but there has really been quite a bit of drama
surrounding it in newsgroup discussion.  I think it can sometimes be difficult to look past all this and take the
time to decide for oneself.  If nothing else, doing so takes time, and even I tend to employ the "30 second rule"
when it comes to new technology.


Sean