RFC: Pay-as-you-go, Portable D Runtime for Microcontrollers (and maybe more)

[Johannes Pfau via Digitalmars-d]

Am Thu, 07 May 2015 11:52:29 +0000
schrieb "Mike" <none at none.com>:

> On Tuesday, 5 May 2015 at 17:38:38 UTC, Johannes Pfau wrote:
> 
> >
> > I would probably split the runtime into at least two, probably 
> > three
> > parts:
> >
> > * Compiler support library (object.d, gcc/*.d, exception
> >   implementation) (module rt/ no module name)
> > * Higher level library (portable, micro)
> > * Hardware specific library (not portable, avr/ stm/)
> >
> > I think some basic portability between microcontrollers is 
> > useful. The
> > compiler support library should be only a few 100 lines of code 
> > and
> > should be without external dependencies. This way it's also 
> > useful for
> > very small platforms (8bit) or use-cases where you inject code 
> > into
> > other processes. It's trivial to port: we should have an ansi-c 
> > port
> > and native ports should be implementable in 1-2 hours.
> >
> >
> > The higher level library should contain stuff like 
> > 'core.time.Duration',
> > emplace, allocator framework, Volatile!T, register wrapper 
> > types, etc.
> > The implementation could be hardware specific, but it should 
> > provide a
> > common interface (e.g. a delay!Duration function has got a 
> > common
> > interface but different implementation, malloc is similar).
> > Namespace micro => import micro.time; import micro.memory;
> > So everything in the micro namespace is portable.
> >
> >
> > The hardware library the provides access to hardware specific
> > peripherals. In reality the high-level-library might require the
> > hardware library or they could actually be merged. The 
> > important part
> > is portable vs platform specific module API for user code.
> 
> Sorry for the late reply, but I'm still chewing and digesting 
> this.  How do you propose delegating implementation down the 
> supply chain?  I suppose you may have answered that with your git 
> idea below, but did you have something else in mind?
> 

Mostly the git idea. It makes sense to keep the 'customization' points
to a minimum and have well-defined hooks like in newlib and like you
suggested. But the git branch way also allows modifying code
everywhere, in case the generic hooks don't fit one platform for some
reason.

> I think that's really what I'm trying to work out with this 
> experiment:  How to delegate the implementation so it's obvious 
> what porters need to do without a lot of explanation, and they 
> can implement just the features they need.

Let me explain the git idea:

core library; branch 'common':
--------------------------------------------
object.d:
class Object
{
    
}

void traceLine(string msg)
{
    static assert(false, "Not implemented");
}
--------------------------------------------
branch avr
--------------------------------------------
object.d:
class Object
{
    
}

version(none) // Not implemented for AVRs right now
{
void traceLine(string msg)
{
    static assert(false, "Not implemented");
}
}
--------------------------------------------
branch nintendo-ds
--------------------------------------------
object.d:
class Object
{
    //Lots of memory, let's add some useful methods
    void toString(scope delegate(const(char)[] buf))
    {
    ...
    }
}

void traceLine(string msg)
{
    asm (print to port xyz...)
}
--------------------------------------------

Porters simply grep for '"Not implemented"' and implement the function
or comment/remove it.
If we now change the interface in the common branch:
void traceLine(string msg) => void traceLine(string msg, int level = 0)

Both branches will get the changes or a merge conflict by simple merging
the common branch. This way we can keep a kinda common interface and
have lots of customization possibilities.

Note 1: The core lib might be a bad example. It should be mostly
portable and have little user-facing code so keeping a common interface
is not important here. It's more useful for a high-level library (e.g.
keep emplace/allocator interface compatible while still allowing custom
malloc or other allocator implementations)

Note 2: Even a simple 'traceLine' function depends on board
definitions. So a port targeting many boards could not even implement a
'traceLine' in the corlib. It would have to delegate the implementation
to the hardware library* effectively tying the core library to the
hardware library. I think this is sometimes unavoidable for low-level
code but this should probably be a decision porters make. The 'common'
corlib should not depend on other libraries.

*or even user code.

Note 3: It could still make sense to keep all the 'to-be-implemented'
functions/hooks in a special module or package. But instead of having
ports/arm/ ports/avr we could have one port/*.d with template files as
described above and the actual platform specific implementations in
branches.

> >
> > A radically different approach instead of using ports 
> > directories is
> > using git features: Have a base repository with a master branch 
> > which
> > only includes the interfaces, probably with static assert 
> > wherever
> > platform specific code is necessary. Then have AVR/STM32/LPC/...
> > branches where you simply implement these functions. (We could 
> > also
> > have different repositories instead of branches)
> >
> > + you can modify all code
> > + you already start with a common interface
> > + changes are easy to compare by comparing git branches
> > + changes have descriptions (in their git commits)
> > + it's easy to merge further generic changes
> 
> I'll have to think about this and give it a try.  I fear, 
> however, that it might be a little too radical for what people 
> are used to, but maybe not.

Many newlib ports are not part of official newlib but custom forks
AFAIK. There's no big difference here (instead of branches we could
also use different repos). By using git we make maintaining custom
changes much simpler. But it is indeed a rather radical approach and
there could certainly be some drawbacks as well. 

(Even if we define formal hooks and a port/ directory structure porters
could of course still fork&modify the code instead. They could probably
use the techniques described here without our explicit support. But if
we decide to go this way, we can simplify _our_ code. No need to have
lots of hooks or version(ARM) else version(AVR) else version() or a
port/ directory structure. Every port is in its own branch)