DIP10005: Dependency-Carrying Declarations is now available for community feedback

[Timothee Cour via Digitalmars-d]

> I just tried compiling phobos and its unittests for dmd 2.066.1 and
2.067.1

I think it's worth considering compile time for partial recompilation as
opposed to full compilation. The benifit of this DIP should be more
pronounced there since there'll be more opportunities to skip parsing
modules in that case. Partial recompilation is what matters most during
`edit compile debug cycle` anyways

On Thu, Dec 22, 2016 at 2:17 AM, Joakim via Digitalmars-d <
digitalmars-d at puremagic.com> wrote:

> On Wednesday, 21 December 2016 at 15:16:34 UTC, Andrei Alexandrescu wrote:
>
>> On 12/20/2016 11:32 PM, Joakim wrote:
>>
>>> Second, as I noted above, most top-level imports have not been made
>>> selective yet, because of the symbol leak bug that was recently fixed by
>>> Martin.  You will see in my PRs that I only list those symbols as a
>>> comment, because I could not turn those into selective imports yet.  If
>>> the compiler is doing its job right, selective imports should greatly
>>> reduce the cost of importing a module, even if your metric would still
>>> show the module being imported.
>>>
>>
>> That is not relevant to this section, which discusses the effectiveness
>> of using local imports with the current compilation technology. Per the
>> section's opening sentence:
>>
>> A legitimate question to ask is whether consistent use of local
>>> imports wherever possible would be an appropriate approximation of
>>> the Dependency-Carrying Declarations goal with no change in the
>>> language at all.
>>>
>>
> It is relevant because it could further reduce the cost from module-scope
> imports.  Whether your section only chooses to focus on locally scoped
> imports and ignore the impact of selective imports is irrelevant to me.
>
> Finally, while it's nice to know the extent of the dependency graph,
>>> what really matters is the _cost_ of each link of the graph, which is
>>> what I keep hammering on.  If the cost of links is small, it doesn't
>>> matter how entangled it is.  If minimizing the dependency graph through
>>> scoping alone, ie without implementing this DIP, removes most of the
>>> cost, that's all I care about.
>>>
>>
>> In first approximation, whether a file gets opened or not makes a
>> difference (filesystem operations (possibly including networking),
>> necessity to rebuild if dependent code is changed. The analysis shows there
>> is significant overhead remaining, on average 10.5 additional files per
>> unused import.
>>
>
> Opening a file or 10 is extremely cheap compared to all the other costs of
> the compiler.  Purely from a technical cost perspective, I'm not sure even
> scoped imports were worth it, as my simple investigation below suggests.
>
> If the current document could be clearer in explaining costs, please let
>> me know.
>>
>
> Yes, please explain what significant "overhead" was reduced by scoping
> imports and would be futher reduced by this DIP.  Opening files doesn't cut
> it.
>
> My point is that the dependency graph matters, but now that we're
>>> getting down to the last entanglements, we need to know the cost of
>>> those last links.  Your dependency analysis gives us some quantitative
>>> idea of the size of the remaining graph, but tells us nothing about the
>>> cost of those links. That's what I'm looking for.
>>>
>>> I will spend some time now investigating those costs with sample code.
>>> My request all along has been that you give us some idea of those costs,
>>> if you know the answer already.
>>>
>>
>> I don't know how to make matters much clearer than the current document.
>> Any suggestions are welcome. The section "Workaround: Are Local Imports
>> Good Enough?" discusses the material cost in terms of extra files that need
>> to be opened and parsed (some unnecessarily) in order to complete a
>> compilation. The "Rationale" part of the document discusses the costs in
>> terms of maintainability, clarity, and documentation.
>>
>
> I don't know how to make it clearer that that's not good enough.  You seem
> to understand that I want more justification than hand-waving about
> "scalable" and "overhead," which is why you now give the cost of opening
> files as justification, but you don't seem to have anything more
> substantive than that flimsy claim.
>
> I just tried compiling phobos and its unittests for dmd 2.066.1 and
> 2.067.1, the dmd releases from right before and right after Ilya's PRs
> linked above (compiling phobos from the older release with the newer dmd
> didn't work and I wasn't interested in porting it).  I found they took
> about the same amount of time to compile their respective phobos and the
> later version consistently took 15 seconds longer to compile the unittests
> on a single core.
>
> This suggests that there has been essentially no technical benefit to
> scoped imports, that it is largely superficial.  That's fine, I think it's
> worth it just from the standpoint of understanding where most dependencies
> are coming from.  I don't think an additional syntax change is necessary
> for the remaining few module-scope dependencies for template constraints.
>
> Note on investigation: Of course, some other relevant factors could have
> changed in dmd and phobos between those two releases, so the result is
> certainly not conclusive.  But the fact that there was no decrease in
> compile times while phobos took around the same amount of time is highly
> suggestive.
>
> I'm uninterested in investigating further given the consistent hand-waving
> justifications for this DIP.  If somebody else had submitted this DIP, it
> would have been quickly shot down, and rightly so.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.puremagic.com/pipermail/digitalmars-d/attachments/20161222/87334888/attachment.html>