Compilation is taking a ton of memory

[H. S. Teoh]

On Thu, Jun 28, 2018 at 06:13:45PM +0000, Eugene Wissner via Digitalmars-d wrote:
> On Thursday, 28 June 2018 at 16:24:07 UTC, H. S. Teoh wrote:
> > On Thu, Jun 28, 2018 at 04:11:57PM +0000, crimaniak via Digitalmars-d
> > wrote: [...]
> > > The problem is aggravated by the fact that DUB compiles all the
> > > sources in one DMD launch.
> > 
> > Doesn't dub have an option to compile packages (i.e. subdirs)
> > separately? Or does that only apply to dub packages, not to subdirs
> > within a single project?
> 
> As far as I can see dub builds static libraries from dub dependencies.
> But it passes all source files from one dub project together to build
> final static library/executable. You can actually use --verbose to see
> what commands a called.

Someone should file an enhancement ticket for this.


[...]
> I compile every source file to an object file, put object files from
> dependencies to its own static libraries and link everything together
> at the end.

That's the traditional C/C++ approach, which is stable, if not the most
suitable for D.  I forget the details now, but I believe there are some
savings in compiling template code if you compile multiple source files
at once (assuming, of course, that they share some number of template
instantiations).  So personally I'd group source files by subdir and
compile them in batches that way, then pass the resulting static libs to
the parent dirs, and so on.


> But I should also admit, I'm not using dmd at all, but build
> everything with gdc/gcc. The only problem is handling dependencies and
> dependencies of dependencies if you have a lot of them.
[...]

Unless you're doing imports using mixins, which is a questionable
practice to begin with, it should be pretty easy to, say, grep for
'import' lines and build your dependency tree that way.

Or use the tup approach of instrumenting the compiler with LD_LIBRARY_*
so that any dependencies are automatically caught, and you don't have to
worry about it (though that only accounts for dependencies on a specific
run, it won't pick up imports across static-if branches, for example,
though it's unlikely to be an actual problem in real-life code).


T

-- 
You have to expect the unexpected. -- RL