[D-runtime] Supporting different C libraries and architectures

[Iain Buclaw]

On 27 February 2012 09:16, Johannes Pfau <johannespfau at googlemail.com> wrote:
> Am 26.02.2012 18:10, schrieb Iain Buclaw:
>
>> On 25 February 2012 17:31, Johannes Pfau<johannespfau at googlemail.com>
>>  wrote:
>>>
>>> This is probably less a problem for DMDs druntime, but we hit this issue
>>> with gdc and the android port now, so I thought I'll also post this
>>> message
>>> here, maybe someone has a good idea.
>>>
>>> Android uses a custom C library (Bionic) which is not 100% Posix
>>> compatible.
>>> With the current approach of supporting different systems using version()
>>> blocks, this will lead to lots of changes in all the core/* files. And it
>>> will get much worse once we try to support other C libraries (newlib for
>>> DS/Wii homebrew, uclibc/dietlibc/eglibc for embedded linux, etc). The
>>> current aproach is already broken for Android, as druntime assumes
>>> version(linux) -->  glibc which is not true. One such broken example is
>>> the
>>> usage of the glibc "backtrace" function.
>>>
>>> Here's the original message sent to the D.gnu newsgroup:
>>>
>>> ________________________________
>>> As recently discussed in a pull request it would be great if we could
>>> make it easier to port druntime to different architectures. Special
>>> casing every C library, every architecture and every OS using version
>>> blocks could lead to difficult to maintain code, so we need a better
>>> solution.
>>>
>>> As far as I can see, the biggest differences are caused by different C
>>> libraries, not by different architectures (except 32bit vs 64 bit
>>> differences). For example glibc headers seem to be very similar for arm
>>> and x86, but bionic vs. glibc brings more differences. Bionic initially
>>> didn't support anything wchar_t related, so core.stdc.wchar_ and
>>> core.stdc.wctype couldn't work with android at all. And C libraries
>>> vary even more in the subset of posix functionality they support.
>>>
>>> We need a directory scheme to support
>>> * Different C libraries
>>>   * on different OS (glibc/bsd glibc/linux)
>>>   * on the same OS (glibc on linux, bionic on linux, uclibc on
>>>     linux, ...)
>>> * Different architectures (ARM, X86, MIPS, PPC, SH4, ...)
>>>
>>> I propose the following directory layout, but of course
>>> better solutions are appreciated:
>>>
>>> druntime
>>> |-core
>>> |---stdc
>>> |---sync
>>> |---sys
>>> |-----posix
>>> |-gc
>>> |-gcstub
>>> |-rt
>>> |-gcc
>>> |---arch
>>> |-----glibc
>>> |-------core
>>> |---------stdc
>>> |---------sync
>>> |---------sys
>>> |-----------posix
>>> |-----bionic
>>> |-------stdc
>>> |-------sync
>>> |-------core
>>> |---------sys
>>> |-----------posix
>>> |-----newlib
>>> |-------stdc
>>> |-------sync
>>> |-------core
>>> |---------sys
>>> |-----------posix
>>>
>> So I gather from what your proposing that ie: core.stdc.stdio will
>> look like the following?
>>
>> module core.stdc.stdio;
>>
>> version (GLibc)
>>   public import gcc.glibc.core.stdc.stdio;
>> else version (Bionic)
>>   public import gcc.bionic.core.stdc.stdio;
>> // etc.
>>
>>
> Yep, that was my initial idea. It's maybe too complicated though. The other
> solution is to remove the toplevel core/stdc and core/sys files and create
> those modules in a subdirectory for every c library. We could then just
> compile in the correct directory:
>
> druntime
> |-core
> |---atomic.d (generic files)
> |---bitop.d
> |---[...]
> |-gc
> |-gcstub
> |-rt
> |-libdc
> |---glibc
> |-----core
> |-------sys
> |-------stdc
> |---bionic
> |-----core
> |-------sys
> |-------stdc
> |---newlib
> |-----core
> |-------sys
> |-------stdc
>
> --
> Johannes Pfau
>

I think that is fine for the structure of the build directories,
however I feel strongly that they should not be installed in such a
structure.

To take your example:

druntime
|-core
|---atomic.d (generic files)
|---bitop.d
|---[...]
|-gc
|-gcstub
|-rt

For the file structure below -  when building for glibc, the make file
would have -I libdc/glibc in it's DFLAGS (or compile all relevant
sources in one go) - so libdc/glibc/core/stdc/stdio.d would still be
declared as  'module core.stdc.stdio;'

When running 'make install' - the build gets the correct sources and
puts them in their true locations.

|-libdc
|---glibc
|-----core      // Installed in /usr/include/d2/core
|-------sys     // Installed in /usr/include/d2/core/sys
|-------stdc    // Installed in /usr/include/d2/core/stdc
|---bionic
|-----core      // Installed in /usr/include/d2/core
|-------sys     // Installed in /usr/include/d2/core/sys
|-------stdc    // Installed in /usr/include/d2/core/stdc
|---newlib
|-----core      // Installed in /usr/include/d2/core
|-------sys     // Installed in /usr/include/d2/core/sys
|-------stdc    // Installed in /usr/include/d2/core/stdc


The idea being that bionic/newlib packages are not shipped with glibc releases.


The complexity here would be a minor alternation to the way the
current build process is done, and will require *no change* in either
existing user code or how the druntime library is written.


Regards

-- 
Iain Buclaw

*(p < e ? p++ : p) = (c & 0x0f) + '0';