Proposed improvements to the separate compilation model

Vladimir Panteleev vladimir at thecybershadow.net
Sat Jul 23 15:34:51 PDT 2011


On Sun, 24 Jul 2011 00:54:57 +0300, Andrei Alexandrescu  
<SeeWebsiteForEmail at erdani.org> wrote:

> On 7/23/11 4:01 PM, Vladimir Panteleev wrote:
>> On Sat, 23 Jul 2011 23:16:20 +0300, Andrei Alexandrescu
>> <SeeWebsiteForEmail at erdani.org> wrote:
>>
>>> On 7/23/11 1:53 PM, Andrej Mitrovic wrote:
>>>> Isn't the biggest issue of large D projects the problems with
>>>> incremental compilation (e.g.
>>>> https://bitbucket.org/h3r3tic/xfbuild/issue/7/make-incremental-building-reliable),
>>>>
>>>> optlink, and the toolchain?
>>>
>>> The proposed improvement would mark a step forward in the toolchain
>>> and generally in the development of large programs. In particular, it
>>> would provide a simple means to decouple compilation of modules used
>>> together. It's not easy for me to figure how people don't get it's a
>>> net step forward from the current situation.
>>
>> Then you don't understand what I'm ranting about.
>
> That's a bit assuming.

OK, that implies that you weren't talking about me above.

> I thought about it for a little and concluded that I'd do good to  
> explain the current state of affairs a bit.
>
> [snip]

There was no need to go in such great detail; I think the basics are well  
understood, I was asking for arguments and counter-arguments.

> Ultimately the programmers figure there's no way to keep files separate  
> without establishing a build mechanism that e.g. generates a.di from  
> a.d, compares it against the existing a.di, and complains if the two  
> aren't identical. Upon such a build failure, a senior engineer would  
> figure out what action to take.

I was going to suggest something like this, but without creating the  
dependency on a 3rd-party build tool. I mentioned in another thread how  
DMD shouldn't touch the .di file's mtime. A better idea is to not attempt  
to overwrite the file at all if the generated .di is identical to the old  
one. With this behavior, you can simply make .di files read-only - the  
compiler will bail out when it tries to save a new version of the .di file.

Yes, this is a hack, but it's not the only solution. Aside writing a build  
tool, as you mentioned, I believe many organizations include automatic  
tests coupled with source control, which could easily detect check-ins  
that change .di files.

Yes, neither of the above are "proper" solutions. But, unless I've lost  
track of something, you're trying to justify a solid amount of work on the  
compiler to implement the "proper" solution, when the above alternatives  
are much simpler in practice. (If you have more counter-arguments, I'd  
like to hear them.)

> But wait, there's less. The programmers don't have the option of  
> grouping method implementations in a hierarchy by functionality (which  
> is common in visitation patterns - even dmd does so). They must define  
> one class with everything in one place, and there's no way out of that.

Sorry, I don't understand this part. Could you elaborate?

> My understanding is that the scenarios above are of no value to you, and  
> if the language would accept them you'd consider that a degradation of  
> the status quo.

I'm not trying to argue for my personal opinion and the way I use D. I was  
trying to point out that your suggestion seems less efficient in terms of  
benefit per work-unit for all users of D.

> Given that the status quo includes a fair amount of impossible to detect  
> failures and tenuous mechanisms, I disagree. Let me also play a card I  
> wish I hadn't - I've worked on numerous large projects and I can tell  
> from direct experience that the inherent problems are... well, odd.  
> Engineers embarked on such projects need all the help they could get and  
> would be willing to explore options that seem ridiculous for projects  
> one fraction the size. Improved .di generation would be of great help.  
> Enabling other options would be even better.

[snip]

>> 1) We don't have an infinity of programmer-hours. I'm saying that the
>> time would likely be better spent at improving .di generation, which
>> should have a much greater overall benefit per required work unit - and
>> for all I can tell, you don't even want to seriously consider this  
>> option.
>
> Generation of .di files does not compete with the proposed feature.

Again, I'm not saying that this is a bad direction, just not the best one.

[off-topic]

>> 2) Once manually-maintained .di files are usable, they will be used as
>> an excuse to shoo away people working on large projects (people
>> complaining about compilation speed will be told to just manually write
>> .di files for their 100KLoC projects).
>
> Your ability to predict future is much better than mine.

I didn't say who'll say that... It might not be you or Walter, but can you  
account for all users of D on IRC, Reddit, StackOverflow, etc.?

Good enough is the enemy of better, etc.

[/off-topic]

P.S. I appreciate you taking the time for this discussion.

-- 
Best regards,
  Vladimir                            mailto:vladimir at thecybershadow.net


More information about the Digitalmars-d mailing list