What is the FreeBSD situation?

[Jonathan M Davis]

On Thursday, November 02, 2017 11:33:39 Steven Schveighoffer via 
Digitalmars-d wrote:
> On 11/2/17 10:56 AM, Andrei Alexandrescu wrote:
> > On 11/02/2017 08:46 AM, Steven Schveighoffer wrote:
> >> Well, that is only header compatibility, not binary compatibility. In
> >> other words, that still means if you use C or C++, you need to
> >> recompile if you are on FreeBSD 12. You can't have one binary that
> >> supports both platforms.
> >
> > Thanks, this is the kind of hard data we're looking for. Do you have a
> > link to support this?
>
> Sorry, I don't have any experience with FreeBSD. I was reacting to
> Jonathan's claim that "They did a lot of work to make that compatible so
> that programs didn't just break, but the compatibility stuff that they
> did was for C/C++, and we don't use C/C++ headers"
>
> That lead me to believe that marking the headers differently is all they
> did. As I said, this is not binary compatibility.
>
> Sure, we can do this too, we don't use C/C++ headers, but we can port
> our headers. The question to answer is, "can we create a binary that
> runs on both versions of FreeBSD?".
>
> The answer to that question, I don't know. Is it possible on FreeBSD to
> detect this at runtime and call the right binding? Not sure. It might
> not even be detectable (see codephantom's post on how FreeBSD 12 may be
> compiled with the older binding anyway!)

The bug report I linked to has an example of how it might be fixed with a
wrapper function that calls a different version of stat depending on what
version of FreeBSD it looks like is running at runtime. But I have no idea
how complete that is. I keep meaning to find the time to look into this in
detail but haven't gotten around to it yet. I believe that what
codephantom's post means is that with that build option, the compatibility
stuff is in place which allows the 32-bit inode stuff to be used but in
spite of the fact that they've now switched to 64-bit inodes (certainly,
that's my understanding of what the compatibility stuff does), but I haven't
dug deep enough to figure out the details.

Given that this issue hasn't been sorted out yet and that if it's not sorted
out by next month, I'm going to be stuck between not upgrading my desktop
and not having a working D setup (due to the TrueOS-stable update to a newer
version of FreeBSD 12), it's looking like I'm going to have to find the time
to dig deeper.

> Another possible option, if it existed, would be to simply call a
> function that does the right thing. Perhaps a .so that can be swapped
> out based on which version of the OS you are on?
>
> But the only thing I can think of currently, that will definitely
> "solve" the problem, is to treat each version as a different OS with a
> new version statement. This doesn't scale well, so I'm not sure if
> that's the road we want to go down.

Well, in the case of FreeBSD, it's not terrible, especially since it doesn't
make much sense to support versions of FreeBSD that the FreeBSD guys don't
support. And in that case, the most you'd get is probably 3 versions
(currently FreeBSD10, FreeBSD11, and FreeBSD12), and they would shift over
time, so while there would be more in the compiler if they weren't removed,
druntime or Phobos wouldn't get an ever growing number of them. But from
what I can tell, the API changes rarely enough that it usually wouldn't even
be needed. I think that it's usually a greater risk that you'd need to build
for a newer system because of ABI changes than that the API changed in a way
that requires versioning druntime or Phobos. This inode situation is an
extreme case, and they've been working for years on getting it done in a way
that minimizes the level of disruption and breakage that it causes.

The bigger problem with separate versions is that the autotester would then
really need to be running at least a FreeBSD 10.x and FreeBSD 11.x box (and
later 11.x and 12.x once 12 is released), and we'd need to release different
packages for each (though we could arguably just rely on the ports tree for
the releases so long as someone is maintaining the FreeBSD port). The fact
that we don't test multiple versions of FreeBSD has occasionally been a
problem already, but the bigger problem has been that the auto-testers just
haven't gotten updated often (IIRC they're on 8.x and not even 9.x, whereas
10.3 is currently the oldest supported release), and now several things have
needed to get fixed towards being able to update them, because several
things broke.

If we _can_ continue to do just do one release of dmd for FreeBSD that works
across several versions, then that's ideal, but the way that FreeBSD is
released, I think that it's normally expected that you're going to have to
rebuild stuff for each major release even if APIs don't change, and if we're
not testing multiple versions, then we're going to miss it when we manage to
break something.

In any case, we need a clear picture of exactly what needs to be done for
the inode64 breakage to know whether we absolutely need to have separate
builds or whether we can get away with playing tricks at runtime to work
around it. The bug report makes it sound like we _might_ be able to work
around this with runtime tricks.

- Jonathan M Davis