Cross compilation idea.

[Jacob Carlborg]

On 2020-08-01 12:26, Rel wrote:
> Lately I've been looking at Zig programming language. Besides other 
> interesting features like error handling and automatically importing C 
> headers, one feature made me really excited. Zig includes a minified 
> version of musl's, glibc's and mingw's CRT, so it can cross compile 
> stuff very easy. And for whatever reason it includes a C cross compiler 
> as well (I guess they already had libclang bundled for importing C 
> headers, so it was kinda easy to use it to compile C).

I've been thinking of the same thing myself. I've talked with kinke (via 
private email) about this. He wasn't sold on the idea. But I've started 
to work on a project for this [1].

> As far as I understand there are few things that forbids LDC from cross 
> compiling as easy as Zig:
> 1) Cross linker. Zig uses LLVM's linker which is called LLD. Can LDC use 
> LLD to target all suppored operating systems?

There are some things that are missing for LLD to work for D 
applications for targeting Apple platforms.

My idea was to use the official Apple linker (ld64). Someone has already 
ported ld64 to other Posix platforms, including MinGW. I've setup a CI 
job to build ld64 for Linux [2].

> 2) Cross libc. Zig bundles minified version of musl, glibc and mingw (I 
> didn't quite understood if it can cross compile to macosx). Do you think 
> LDC could do the same or reuse the work done by the Zig team?

It can cross-compiler to macOS, by using, in my opinion, an ugly hack. 
The Zig compiler adds a flag to the linker to ignore undefined symbols. 
Since the whole SDK is dynamically linked on Apple platforms, the only 
thing you need the SDK for, is to verify that the symbols the 
application is using exist. It cannot cross-compiler C/C++ code to macOS 
because it doesn't include the header files.

My solution to this problem was to generate SDK in a CI job. These days 
the libraries in the SDK don't contain any code, since everything is 
dynamically linked. Instead, the SDK consists of .tbd files, which are 
basically YAML files. It more or less only contains the symbol names. 
Apple provides a tool to generate these files based on the actual 
library files. I've built a tool [3] that automatically generates the 
SDK. Then I have a separate CI job that uses that tool to generate the 
SDK [4] and publish it on GitHub. I'm not sure what the Xcode license 
say about this. But I created the files myself and not copied any 
existing files.

> 3) Cross druntime and phobos. I guess LDC could distribute prebuilt 
> versions of druntime and phobos for most used operating system. Or just 
> build and cache it the first time the user tries to cross compile to new 
> target the first time. What do you think?

It could, but my idea was to have more or less an archive for each 
target available online. The first time the target is used it will 
automatically download the archive, unpack it and install it so it can 
be used.

[1] https://github.com/d-cross-compiler

[2] 
https://github.com/d-cross-compiler/cctools-port/releases/tag/cctools-949.0.1-ld64-530

[3] https://github.com/d-cross-compiler/apple-sdk-generator

[4] https://github.com/d-cross-compiler/sdk-apple/releases/tag/v0.0.1

-- 
/Jacob Carlborg