API changing, possible low lying fruit, random thought

[Chad J]

Hi folks,

First, I'm shooting from the hip, so if you don't have time to sit and 
sift through these ramblings, maybe you shouldn't try.  You have been 
warned.

So I was driving to college this weekend while recalling some d ng 
conversation about memory management and how it was a "low lying fruit" 
which was plucked by writing a garbage collector, and how cool it would 
be if D could find another such "low lying fruit" to leverage on.  At 
about the same time, I was thinking about how much it freaking sucks 
that libraries and APIs in general are one-shot deals and can't have 
their interfaces rewritten at a later date without severe consequences. 
  It made me think - perhaps API interfaces are such a low lying fruit.

There are a couple problems with API interfaces:
1.)  People learn them.  There is a time investment.
2.)  Interface changes can cause code to need to be rewritten.  A LOT of 
code.

D addresses (1) a little bit.  There is the deprecated keyword.  It 
helps a little, but all it does is tell the lib user that a function, 
class, member, or whatnot is deprecated.  There is no way to tell the 
lib user what to use instead of why this thing was deprecated.  So there 
needs to be some way to deprecate old interface junk and give meaningful 
information to someone who is just now trying to compile with the new 
version of a library.  It would be impossible to infuse the user with 
all sorts of new knowledge at no cost, but when an API change should 
happen it would be nice if the user gets the information they need, when 
they need it, and as easily as possible.

It seems that D doesn't address (2).  I don't think very many languages, 
if any, address (2) at all.  But it can be handled, and perhaps using 
mostly language features that already exist in D.  It can be handled by 
creating some intuitive way for library/API writers to define changes or 
patches to their interface.  These changes would do stuff like rewrite 
writefln( "Hello World!" ); to Stdout( "Hello World!" ).newline;.  Even 
if this only handles trivial cases like rewriting function calls, it 
would catch a large portion of API changes.  It would be really nice 
though, if it could also edit the user's code, as in modify/add/remove, 
so that some levels of refactoring could be performed automatically. 
The new code (user's source tree) might be placed in a new directory or 
some such; an implementation detail.  I imagine the import mixin type 
stuff could be used to grab the user source code for analysis, and then 
you just need some way to write the result somewhere else.  As for the 
analysis itself, I'm not so sure about this.  I know it should be 
possible already, and talked about proposed features like AST macro type 
things seem to be pointed in this direction.

OK so let's assume we do the above two things.  That might still leave 
some issues.  Now we have library scripts rummaging through our code and 
changing various bits and pieces without our knowledge.  Maybe a little 
scary.  This could be handled by pragma(msg,"...")'ing some sort of 
description about changes that happened during a code update or patch. 
But it would also help to know that libraries aren't going to rewrite 
any user code until they are explicitly asked to do so, which might just 
mean versioning off of a compile switch.  Even better if, upon 
compilation, the library runs a script that checks for new versions of 
the library, and mentions to the user when those things are available. 
Perhaps even more practical would be to have the compiler keep a list of 
what versions of libraries are in use, and when a new one is tried it 
prompts to start the patching process.

Then there is a problem when refactoring requires some sort of decision 
from the library user.  This is where some compiler/language magic might 
be needed, in the form of better compile-time io.  Just some sort of 
scanf to pause compilation and retrieve user input.

All in all, I envision library users seeing a kind of installer like 
user interface whenever they try to upgrade to a new library version, 
and they have to do very little, if any, code refactoring of their own.

That's all I have for now.
- Chad