DMD Source Archive - Why?

[Steven Schveighoffer]

On Sunday, 14 April 2024 at 06:04:02 UTC, Walter Bright wrote:
> On 4/11/2024 8:28 AM, Steven Schveighoffer wrote:
>> Think of why we use standard object formats instead of our own 
>> format (which would allow much more tight integration with the 
>> language).
>
> We use standard object formats because we don't have a linker. 
> I've spent a lot of time trying to understand their byzantine 
> structure. It's not fun work.

Exactly, we don't need to be responsible for all the things. 
Using standard object format means we don't have to write our own 
linker.

> I mentioned that the archive support can be pluggable. It's 
> only two functions with a generic interface to them. If we 
> aren't going to move forward with source archives, it would be 
> a giant waste of time to learn .tar and all its variations.

Fair point. If this doesn't fly, then learning all the variations 
of tar might not be applicable (though I can say I personally 
"learned" tar in about 15 minutes, it's really simple).

> I chose to invent the .sar format because it's 20 lines of code 
> to read them, and about the same to write them. Even doing a 
> survey of the top 10 archive formats would have taken more time 
> than the entire PR, let alone the time spent debating them.

This misses the point. It's not that it's easy to add to the 
compiler. Both are easy, both are straightforward, one might be 
easier than the other, but it's probably a wash (maybe 2 hours vs 
4 hours?)

The problem is *all the other tools* that people might want to 
use. And specifically, I'm talking about IDEs. You have a 20 line 
solution in D, how does that help an IDE written in Java? 
However, Java has `tar` support that is tried and tested, and 
probably already in the IDE codebase itself.

Writing 20 lines of code isn't "mission accomplished". We now 
have to ask all IDE providers to support this for symbol lookup. 
That's what I'm talking about.

> The source archive PR is a proof of concept. The actual archive 
> format is irrelevant.

This is good, and I understand what you are trying to say. As 
long as it remains PoC, with the expectation that if it turns out 
to be useful, we address these ecosystem issues, then I have no 
objections.

> > or we could indicate they are vendor-specific extensions
>
> Wouldn't that defeat the purpose of being a .tar format?

No, vendor-specific sections are in the spec. Existing tar 
programs would still read these just fine.

But even if we wanted to avoid that, adding an index can be done 
by including a specific filename that the D compiler recognizes 
as the index.

> > It should be a high bar to add new file formats to the
> toolchain, as this affects all tools that people depend on and 
> use.
>
> Using a .tar format would affect all the dlang source code 
> tools just as much as using the .sar format would.

Yes, of course. It's just, will there be a ready-made library 
available for whatever IDEs are using for language/libraries? 
With .sar, the answer is no (it hasn't been invented yet). With 
.tar, it's likely yes.

-Steve