Reducing the inter-dependencies (in Phobos and at large)gei

[renariko at yahoo.com]

On Wednesday, 24 April 2013 at 12:03:52 UTC, Dmitry Olshansky 
wrote:
> Recently I've struggled again to integrate a module in Phobos 
> and the amount of curious forward reference bugs made think 
> about a related but not equivalent problem.
>
> Intro
>
> Basically an import graph of Phobos is a rat's nest of mutual 
> imports.
> With the most of modules being template "toolkits" you 
> shouldn't pay for what you don't use. Yet it isn't true in 
> general as the module may drag in other modules and with that 
> all of static constructors and/or globals related.
>
> Adding even more fun to this stuff - to get a "constraint 
> checker" you have to import all other stuff from that module 
> (and transitively from all imported by it modules, see 
> std.range example below).
>
> One motivating example (though I believe there are much more 
> beyond this one):
>
> As some might know regular expression engine can be used for 
> both generating sequences for a known pattern and 
> matching/finding pieces that do match it in some data.
>
> In fact I have such a functionality in std.regex hidden from 
> public interface (not yet complete, wasn't sure of the kind of 
> API etc.).
>
> Now the *key* fact:
>
> auto generate(RegEx, rng) (RegEx re, Random rng)
> 	if(isRegex!RegEx && isUniformRNG!Random)
> {
> ...
> }
>
> Now given that innocent signature you have dependency on the 
> whole of std.random *including* seeding the default random 
> number generator at the program start!
>
> And recall that generating strings while neat is arguably more 
> rare need then pattern matching.
>
> Same thing with std.datetime - to get an ability to accept any 
> kind of Date as a template parameter (in your API) you have to 
> get the whole std.datetime *even if the user never ever calls 
> this function* of your module API.
>
> And everyone and their granny depends on full version of 
> std.range
> that in turn depends on std.algorithm that depends on std.conv 
> that depends on std.format(!) and that incidentally on std.uni.
> BTW std.uni turns out to be a kind of sink everybody imports 
> sooner or later if only for unittests, sadly it's mostly 
> imported unconditionally.
>
> And that skipping a full rat's nest to preserve the brains of 
> the reader.
>
> After a couple of frustrating evenings dealing with it 
> (multiplied by the bogus dmd) I've come up with the idea below.
>
>
> Solution
>
> First of all no compiler magic required (phew-ew!) :)
>
> Not to mention the 2 obvious facts - smaller modules might 
> help, as would guarding by version(unittest) imports used only 
> for unit tests.
>
> What we need is to re-arrange the module hierarchy (and we need 
> that anyway) so that we split off the "concept" part of modules 
> to a separate package.
>
> That would allow modules that need this to use these Duck-typed 
> entities (IFF the user ever passes such an entity) can stick 
> with importing only the concept part.
>
> Applying that to the current layout would look like:
> std.concept.range
> std.concept.random
> std.concept.* //every other module with any useful isXYZ 
> constraint
> std.* // stays as is
>
> Any module that has "concept" part then looks like this:
>
> module std.xyz;
> import std.concept.xyz;
> ... //the rest
>
> And then other weakly-dependent modules (i.e. these that are 
> satisfied with traits and duck-typed interfaces) can safely 
> import std.concept.xyz instead of std.xyz. E.g. std.regex would 
> import std.concept.random to get isUniformRNG and rely on duck 
> typing thusly described to use it correctly.
>
> The change is backwards compatible and introduces no breakage.
> Only clean sugar-free interdependence of modules in Phobos.
>
> Later people (mostly library writers) can use e.g. 
> std.concept.range
> to avoid pulling full dependency tree in case only constraints 
> are needed. The technique can be touted as coding guideline for 
> template and duck-type heavy libraries.
>
> Thoughts? Other ideas?