DIP10005: Dependency-Carrying Declarations is now available for community feedback

Fri Dec 23 18:23:53 PST 2016

tldr: the performance impact of this DIP would be too small to easily 
measure and only impacts 26 declarations referencing 14 templates in the 
standard library anyway.

On Fri, 23 Dec 2016 18:55:25 -0500, Andrei Alexandrescu wrote:
>> I grant that everyone uses phobos, and phobos uses template constraints
>> a lot. If it's *just* a problem inside phobos, though, there's another
>> obvious solution: split up modules that tend to house a lot of template
>> constraints. Split up the modules that use a wide variety of template
>> constraints.
> 
> This point is discussed carefully in DIP1005. Please let me know if
> anything needs to be added.

An estimate for the actual impact on phobos, since that's your primary 
driver for the change -- both under the status quo and if we try to split 
modules.

The comparison to mach.d is a strawman. When I thought this might be a 
problem within phobos, I thought we'd probably split std.traits and maybe 
std.meta up, probably into 2-5 modules each. Not 150 lines per module; 
more like 1500 to 4000 lines per module.

Then I looked at the code.

Phobos has 26 templates that would use this special syntax, referencing 
14 distinct templates and CTFE functions in three modules.

If you kept the same ratios as are found in mach.d, you'd have one file 
for every template used as a constraint outside its own module, one for 
everything else, and as many files again with nothing in them.

The templates that have to be parsed are std.traits, std.meta, and 
std.range.primitives. I wrote a test program that imported them without 
using them. It compiled in too little time to accurately measure -- 
`time` reports 0.01s wall time.

You want to add new language syntax to save me ten milliseconds during 
compilation.

You hope that template constraints become more common. If they do, this 
change relies on:

* projects defining their own constraints (not just using std.traits)
* those constraints being defined in modules other than where they're used
* those constraints being defined in very large modules
* my code not depending on anything in those constraint-defining modules
* my code depending on something in a file with a template with one of 
those constraints
* this whole situation being common enough to give me a death by a 
thousand papercuts

If any project aside from the standard library has a 7k line module 
defining things mainly used in template constraints, something is 
seriously weird. But on the plus side, it would only cost me 10 
milliseconds. Now, if *dozens* of projects did that, well, I'd be running 
to the relative simplicity of Java APIs long before I worried about 
compilation speed.

> Lazier compilation is indeed a projected benefit of this DIP. I did not
> want to dilute the thrust of the proposal with a remote promise.

Lazier compilation would *obviate* this DIP. Lazy compilation of 
selective and static imports would not require any parser changes and 
would make a lot of code faster (at the cost of allowing some things that 
don't compile to start compiling, as does your proposal). You can't get 
any performance advantages outside templates without implementing lazy 
imports.

Appendix A: templates that would receive 'with(import foo)' in Phobos.

I used a relatively simple regex to look for this. If someone put more 
than one line between a template and its constraints, or comments with 
certain formatting, I may have missed it. However, that would violate the 
phobos style guide.

This doesn't include templates that use constraints defined in their own 
modules, because that module must already be parsed and no efficiency 
gains could be realized.

It also omits a few cases where the module has a strong reason to import 
the dependency aside from template constraints. I believe this was only 
two constraints defined in std.digest, used in one or two other modules.

algorithm/comparison.d:98:template among(values...)
experimental/typecons.d:82:private template implementsInterface(Source, 
Targets...)
experimental/typecons.d:94:    template implementsInterface()
experimental/typecons.d:126:private template implementsInterface(Source, 
Targets...)
experimental/typecons.d:184:template wrap(Targets...)
experimental/typecons.d:237:        template wrap(Source)
range/package.d:2069:template Take(R)
range/package.d:3501:template Cycle(R)
range/interfaces.d:277:template MostDerivedInputRange(R)
range/interfaces.d:336:template InputRangeObject(R)
numeric.d:678:template FPTemporary(F)
conv.d:3894:template octal(alias decimalInteger)
utf.d:1136:package template codeUnitLimit(S)
typecons.d:1779:private mixin template RebindableCommon(T, U, alias This)
typecons.d:1838:template Rebindable(T)
typecons.d:4239:template wrap(Targets...)
typecons.d:4252:    template wrap(Source)
typecons.d:4412:template wrap(Targets...)
typecons.d:4429:template unwrap(Target)
typecons.d:4461:template unwrap(Target)
format.d:657:template FormatSpec(Char)
algorithm/iteration.d:1055:template filter(alias predicate) if (is(typeof
(unaryFun!predicate)))
internal/math/biguintcore.d:81:template maxBigDigits(T) if (isIntegral!T)
meta.d:248:package template OldAlias(T) if (!isAggregateType!T || is
(Unqual!T == T))
meta.d:254:package template OldAlias(T) if (isAggregateType!T && !is
(Unqual!T == T))
utf.d:3542:template byUTF(C) if (isSomeChar!C)

Appendix B: templates that would need to be extracted out in phobos, if 
parsing their modules cost a non-negligible amount of time.

std.meta:
    allSatisfy
    anySatisfy
    ApplyLeft

std.range.primitives:
    hasSlicing
    isInputRange
    isInfinite

std.traits:
    isAggregateType
    isAssociativeArray
    isDynamicArray
    isFloatingPoint
    isImplicitlyConvertible
    isIntegral
    isMutable
    isSomeChar