Moving druntime into the DMD repository

[Steven Schveighoffer]

On 7/31/18 4:48 PM, Seb wrote:
> On Tuesday, 31 July 2018 at 19:54:20 UTC, Steven Schveighoffer wrote:
>> On 7/31/18 2:24 PM, Walter Bright wrote:
>>> Since DMD and Druntime require each other, it is the right thing to do.
>>
>> I think this is an incorrect relationship.
>>
>> DMD does NOT require Druntime, *testing* DMD requires Druntime.
> 
> Well DMD ships just happens to ship with Druntime ;-)

What I mean is that one can build and use DMD without druntime (see Mike 
Franklin's project from 2013). The test suite requires it because we are 
testing the interactions between the DMD binary and the runtime that it 
uses. This whole thread is really about testing.

>> In fact, I thought the whole point of the -betterC feature was to 
>> eliminate any dependency on druntime.
> 
> That's only partially true. Even with -betterC, `object.d` will still be 
> imported by default and actually many language features that work in 
> -betterC will be lowered to druntime (even in betterC):
> 
> One simple example: https://run.dlang.io/is/41vvoO

True, you can probably say that most of object.d belongs to the 
compiler. I wouldn't have any problem moving that.

>> As we move to more and more library-provided hooks and away from 
>> "compiler magic", this coupling will become less and less prevalent.
> 
> On the contrary, DMD will depend more and more on druntime as the magic 
> is now in druntime and not in the dmd source code.

Not really. There is a confusion here about what is a dependency, and 
what is the interface. The agreed-upon interface between compiler and 
runtime is really the only thing that requires careful updates and 
breaks the CI. As long as the rest is independent, it can be done on one 
side or the other.

What I meant by the statement above is, the runtime is going to be 
deciding more of what actually happens instead of the compiler. When 
these decisions are moved out of the compiler and into the runtime (in 
the form of templates), the compiler becomes more streamlined and less 
crowded with "code generating code". The hooks become simpler and 
naturally more stable, because the place where you "play with" the 
functionality is in the library. This means you need to change those 
hooks less often, and breaking projects becomes less common.

A glaring example is the AA library. If the compiler simply lowered any 
Value[Key] to AssociativeArray!(Key, Value), and let the runtime define 
*everything*, when would you have to change the compiler again after 
that? But of course, until then, the compiler and AA runtime have to be 
altered in lock-step.

>> In fact, most of the changes that require both projects to be 
>> simultaneously to be updated are these magic-removing ones.
> 
> Yes and no. Whenever you want to update a library hooks (which are 
> implementation-defined and not part of the specification), it's rather 
> complicated to do with keeping all CIs happy.

Yes, exactly my point! The removal of magic means removal of the 
complicated hooks.

> 
>> If anything makes sense, it would be to remove the 
>> compiler-dependencies out of druntime into a smaller runtime library 
>> that is included in the dmd project. Most of druntime is dependencies 
>> for the platform, not the compiler.
> 
> Good idea!

I think object.d can probably go to the compiler, but some stuff in 
there should probably be moved into a druntime-specific module. I'd have 
to look more deeply at the rt package to see what makes sense, but 
everything else should stay in druntime (core, gc).

-Steve