Linking is the slowest part of D's compilation process– can we use mold to speed it up?

[H. S. Teoh]

On Wed, Feb 24, 2021 at 11:53:34PM +0000, deadalnix via Digitalmars-d wrote:
> On Wednesday, 24 February 2021 at 22:12:46 UTC, Atila Neves wrote:
> > On Tuesday, 23 February 2021 at 14:41:59 UTC, James Lu wrote:
> > The linker can be a bottleneck, yes, especially since it doesn't do
> > work in parallel. But in my experience, if the linker takes a while,
> > compiling took a lot longer still. Of course, any improvements in
> > this area are welcome, and I hope mold is production-ready as soon
> > as possible.
> 
> This is true for a fresh build, but often not the case for incremental
> builds, which dev often have to go through. This is because the work
> you have to do for sources grows with the size of the changeset, while
> the work you have to do link grows with the size of the project as a
> whole, changed or not. On large projects, it is very common that
> linking dominates incremental builds.
[...]

This is very interesting.  I wonder if there's a way to incrementally
update the executable, instead of starting from scratch each time?

E.g., hypothetically, if the linker emitted not only the executable but
also some kind of map file describing the various parts that compose the
executable, together with some extra information about offsets/addresses
that depend on each other between parts, then in theory, if we change n
object files (where n is significantly less than the total number N of
all object files), we ought to be able to regenerate the executable by
copying most of its current data, move a few sections around, and patch
up some references.

If the executable format is flexible enough (I think ELF is, don't know
about PE), we could also pad the executable with some extra unused space
between sections to allow for growth of individual sections up to some
limit. Then we might be able patch in updated object files in-place,
along with updating some references as needed, as long as said object
files don't grow beyond the size of the extra space.

This could significantly speed up the code-compile-run cycle during
development.  For releases, of course, you'd want to compact the
executable, but generally it's expected that release builds are OK to
take longer.


T

-- 
Государство делает вид, что платит нам зарплату, а мы делаем вид, что работаем.