[D-runtime] Supporting different C libraries and architectures

[Brad Roberts]

It's not much less of a problem for DMD than any other compiler given that many, if not all, of the same problems crop
between windows, linux, freebsd, and osx -- even if they are all x86 at the moment.  Only one of those is a glibc based
platform.  Among the non-windows platforms, all of them strive for some level of posix compatibility.  The major
differences are more "what subset of posix" not "I did big parts of the posix apis differently".  I believe that
exposing completely separate trees for each combination of platform and libc implementation is likely to lead to major
pain down the road.  For what it's worth, I'm not extrapolating based purely on dmd's current state.  I've worked with
nearly every flavor of unix at one point in time or another, porting large code bases among them.

I do agree that parts of druntime (and almost certainly phobos too) conflate issues and use the occasional api that
isn't part of posix.  Those are more the exception than the rule and are likely fairly easy to clean up.  The classic
example is indeed backtrace.  It's use is both non-portable (though man it's a nice api to have) across libc's but also
very well isolated to one call back function.  Additionally, where it doesn't exist, it almost certainly would be better
to implement behind a similarly named function call rather than a version block per platform right there in the
exception handling code.

My 2 cents,
Brad

On 2/25/2012 9:31 AM, Johannes Pfau wrote:
> This is probably less a problem for DMDs druntime, but we hit this issue with gdc and the android port now, so I thought
> I'll also post this message here, maybe someone has a good idea.
> 
> Android uses a custom C library (Bionic) which is not 100% Posix compatible. With the current approach of supporting
> different systems using version() blocks, this will lead to lots of changes in all the core/* files. And it will get
> much worse once we try to support other C libraries (newlib for DS/Wii homebrew, uclibc/dietlibc/eglibc for embedded
> linux, etc). The current approach is already broken for Android, as druntime assumes version(linux) --> glibc which is
> not true. One such broken example is the usage of the glibc "backtrace" function.
> 
> Here's the original message sent to the D.gnu newsgroup:
> 
> ------------------------------------------------------------------------------------------------------------------------
> As recently discussed in a pull request it would be great if we could
> make it easier to port druntime to different architectures. Special
> casing every C library, every architecture and every OS using version
> blocks could lead to difficult to maintain code, so we need a better
> solution.
> 
> As far as I can see, the biggest differences are caused by different C
> libraries, not by different architectures (except 32bit vs 64 bit
> differences). For example glibc headers seem to be very similar for arm
> and x86, but bionic vs. glibc brings more differences. Bionic initially
> didn't support anything wchar_t related, so core.stdc.wchar_ and
> core.stdc.wctype couldn't work with android at all. And C libraries
> vary even more in the subset of posix functionality they support.
> 
> We need a directory scheme to support
> * Different C libraries
>   * on different OS (glibc/bsd glibc/linux)
>   * on the same OS (glibc on linux, bionic on linux, uclibc on
>     linux, ...)
> * Different architectures (ARM, X86, MIPS, PPC, SH4, ...)
> 
> I propose the following directory layout, but of course
> better solutions are appreciated:
> 
> druntime
> |-core
> |---stdc
> |---sync
> |---sys
> |-----posix
> |-gc
> |-gcstub
> |-rt
> |-gcc
> |---arch
> |-----glibc
> |-------core
> |---------stdc
> |---------sync
> |---------sys
> |-----------posix
> |-----bionic
> |-------stdc
> |-------sync
> |-------core
> |---------sys
> |-----------posix
> |-----newlib
> |-------stdc
> |-------sync
> |-------core
> |---------sys
> |-----------posix
> 
> Code in the gc, gcstub and rt directories shouldn't depend on special C
> libraries, small differences can be solved using version() blocks.
> 
> Every C library get's it's  own bindings in the gcc.arch package. I'm
> not sure if those bindings should only provide functionality included
> in core, or whether these bindings should include additional, C library
> specific bindings.
> 
> The files in core.* would reference the corresponding files in
> gcc.arch.core.*
> 
> The downside of that approach is that we'll have some code duplication,
> especially for similar C libraries. And merging changes is therefore
> not that easy as well. However, as the C and Posix headers don't change
> often, that's probably neglectable.
> 
> Here are some examples for that directory scheme:
> 
> core/stdc/config.d (Should be shared with dmd druntime?)
> -----------------------------------------------
> enum LibC
> {
>     GlibC,
>     uClibc,
>     dietlibc,
>     newlib,
>     dmc, //digitalmars, windows
>     msvcrt,
>     ...,
>     unknown
> }
> 
> //Need some way to define this correctly
> enum libC = LibC.GlibC;
> enum string libCVersion = ""; //Free form version number
> -----------------------------------------------
> 
> core/stdc/math.d
> -----------------------------------------------
> import core.stdc.config;
> 
> static if(libC == LibC.GlibC)
> {
>     public import gcc.arch.glibc.core.stdc.math;
> }
> else
> {
>     static assert(false, "C library '" ~ libC ~ "' not supported in " ~
>     __FILE__);
> }
> 
> //Maybe put generic parts here?
> -----------------------------------------------
> 
> core/sys/posix/dirent.d
> -----------------------------------------------
> import core.stdc.config;
> 
> static if(libC == LibC.GlibC)
> {
>     /*
>      * I admit this looks horrible. We can probably remove the core
>      * part, maybe even gcc, so arch.glibc.sys.posix.dirent?
>      * or glibc[.core].sys.posix.dirent?
>      */
>     public import gcc.arch.glibc.core.sys.posix.dirent;
> }
> else static if(libC == LibC.GlibC)
> {
>     public import gcc.arch.glibc.core.sys.posix.dirent;
> }
> else
> {
>     static assert(false, "C library '" ~ libC ~ "' not supported in " ~
>     __FILE__);
> }
> -----------------------------------------------
> 
> Maybe we could use pragma(msg) warnings instead of static asserts, so
> it'd be possible to import the modules in any case and use
> traits(__compiles) to check for specific symbols/functions.
> 
> -- 
> Johannes Pfau
> 
> 
> 
> _______________________________________________
> D-runtime mailing list
> D-runtime at puremagic.com
> http://lists.puremagic.com/mailman/listinfo/d-runtime