DIP10005: Dependency-Carrying Declarations is now available for community feedback

Joakim via Digitalmars-d digitalmars-d at puremagic.com
Thu Dec 22 02:17:33 PST 2016


On Wednesday, 21 December 2016 at 15:16:34 UTC, Andrei 
Alexandrescu wrote:
> On 12/20/2016 11:32 PM, Joakim wrote:
>> Second, as I noted above, most top-level imports have not been 
>> made
>> selective yet, because of the symbol leak bug that was 
>> recently fixed by
>> Martin.  You will see in my PRs that I only list those symbols 
>> as a
>> comment, because I could not turn those into selective imports 
>> yet.  If
>> the compiler is doing its job right, selective imports should 
>> greatly
>> reduce the cost of importing a module, even if your metric 
>> would still
>> show the module being imported.
>
> That is not relevant to this section, which discusses the 
> effectiveness of using local imports with the current 
> compilation technology. Per the section's opening sentence:
>
>> A legitimate question to ask is whether consistent use of local
>> imports wherever possible would be an appropriate 
>> approximation of
>> the Dependency-Carrying Declarations goal with no change in the
>> language at all.

It is relevant because it could further reduce the cost from 
module-scope imports.  Whether your section only chooses to focus 
on locally scoped imports and ignore the impact of selective 
imports is irrelevant to me.

>> Finally, while it's nice to know the extent of the dependency 
>> graph,
>> what really matters is the _cost_ of each link of the graph, 
>> which is
>> what I keep hammering on.  If the cost of links is small, it 
>> doesn't
>> matter how entangled it is.  If minimizing the dependency 
>> graph through
>> scoping alone, ie without implementing this DIP, removes most 
>> of the
>> cost, that's all I care about.
>
> In first approximation, whether a file gets opened or not makes 
> a difference (filesystem operations (possibly including 
> networking), necessity to rebuild if dependent code is changed. 
> The analysis shows there is significant overhead remaining, on 
> average 10.5 additional files per unused import.

Opening a file or 10 is extremely cheap compared to all the other 
costs of the compiler.  Purely from a technical cost perspective, 
I'm not sure even scoped imports were worth it, as my simple 
investigation below suggests.

> If the current document could be clearer in explaining costs, 
> please let me know.

Yes, please explain what significant "overhead" was reduced by 
scoping imports and would be futher reduced by this DIP.  Opening 
files doesn't cut it.

>> My point is that the dependency graph matters, but now that 
>> we're
>> getting down to the last entanglements, we need to know the 
>> cost of
>> those last links.  Your dependency analysis gives us some 
>> quantitative
>> idea of the size of the remaining graph, but tells us nothing 
>> about the
>> cost of those links. That's what I'm looking for.
>>
>> I will spend some time now investigating those costs with 
>> sample code.
>> My request all along has been that you give us some idea of 
>> those costs,
>> if you know the answer already.
>
> I don't know how to make matters much clearer than the current 
> document. Any suggestions are welcome. The section "Workaround: 
> Are Local Imports Good Enough?" discusses the material cost in 
> terms of extra files that need to be opened and parsed (some 
> unnecessarily) in order to complete a compilation. The 
> "Rationale" part of the document discusses the costs in terms 
> of maintainability, clarity, and documentation.

I don't know how to make it clearer that that's not good enough.  
You seem to understand that I want more justification than 
hand-waving about "scalable" and "overhead," which is why you now 
give the cost of opening files as justification, but you don't 
seem to have anything more substantive than that flimsy claim.

I just tried compiling phobos and its unittests for dmd 2.066.1 
and 2.067.1, the dmd releases from right before and right after 
Ilya's PRs linked above (compiling phobos from the older release 
with the newer dmd didn't work and I wasn't interested in porting 
it).  I found they took about the same amount of time to compile 
their respective phobos and the later version consistently took 
15 seconds longer to compile the unittests on a single core.

This suggests that there has been essentially no technical 
benefit to scoped imports, that it is largely superficial.  
That's fine, I think it's worth it just from the standpoint of 
understanding where most dependencies are coming from.  I don't 
think an additional syntax change is necessary for the remaining 
few module-scope dependencies for template constraints.

Note on investigation: Of course, some other relevant factors 
could have changed in dmd and phobos between those two releases, 
so the result is certainly not conclusive.  But the fact that 
there was no decrease in compile times while phobos took around 
the same amount of time is highly suggestive.

I'm uninterested in investigating further given the consistent 
hand-waving justifications for this DIP.  If somebody else had 
submitted this DIP, it would have been quickly shot down, and 
rightly so.


More information about the Digitalmars-d mailing list