Is package.d a good idea?

[Jonathan M Davis]

On Sunday, July 01, 2018 14:23:36 Yuxuan Shui via Digitalmars-d wrote:
> On Sunday, 1 July 2018 at 11:55:17 UTC, Jonathan M Davis wrote:
> > On Sunday, July 01, 2018 11:36:51 Yuxuan Shui via Digitalmars-d
> >
> > wrote:
> >> [...]
> >
> > The entire reason that package.d was added as a feature was so
> > that modules could be split into packages without breaking
> > code, and it's still valuable for that.
> >
> > [...]
>
> I was suggesting we do what Rust did. i.e. 'import foo', imports
> foo.d, which can in turn do 'import foo.bar', which will import
> foo/bar.d.

So, basically, we'd implicitly have a package.d for every package where each
implicit package.d publicly imported every module in its package (including
the implicit package.d from any sub-packages)? Well, for starters, that's
going in pretty much the opposite direction of where best practice in D has
been going. Local and scoped imports have become best practice, which means
writing stuff like

import std.conv : to;

inside the function that's using std.conv.to rather than

import std.conv;

at the top of the file, let alone

import std;

So, the suggestion that you could just import any package and get everything
in it recursively, doesn't really fit with where we've been going.
Regardless of that though, I'm not even sure that it works with D's import
model.

As things stand, D only ever imports exactly the modules that you tell it to
import. It doesn't even have the concept of importing a package. package.d
is a file like any other. It's just that the compiler has the little bit of
magic to know that if it's told to import a.b that if a/b/package.d exists
and has the module declaration for a.b, then it should be treated as being
module a.b. The only reason that package.d then ends up importing the entire
package is because the programmer explicitly pubicly imports each module
from the package. They could do something else, and in some cases, it makes
good sense to do stuff like include modules from elsewhere or to exclude
modules from within the package. Your suggestion gives no such control,
which could be a serious downside.

But aside from the control issue, the fact of the matter is that D currently
only imports what it's told to import. It doesn't have to know or care what
all of the modules in a package are. It just goes and grabs what it's told
to grab, whereas if it has to search recursively, then that probably
negatively impacts the performance of imports, and it fundamentally changes
how importing works, because then instead of just grabbing what it's told to
grab, it has to compile the list based on what it finds - which may or may
not be the correct list. If any modules are missing for some reason, or if
the compiler's import paths are misconfigured, then you could think that
you're importing a module when you're not, which could lead to function
hijacking when you try to use the function. In most cases, it would just be
a compilation error, because the symbol is missing, but if another symbol
which isn't missing happens to match, then instead of getting a symbol
conflict, it would call the wrong function. It's the sort of thing that's a
lot more likely to happen when installs and upgrades go wrong than during
your typical development, but it does make the situation more error-prone
than it is now.

It also wouldn't work well if anyone did something like split a package
across libraries (which is a questionable practice but perfectly legal).
When you compiled one library, you'd get one set of modules when importing
the package, and when you imported another module, you'd get a different set
(whereas right now, you just get the modules that you explicitly imported).
What you'd get when using both libraries would then be highly dependent on
how you compile your code. The idea that you're going to recursively find
all of the modules in a package simply doesn't work well with a separate
compilation model. It's not impossible to make work, but it's error-prone.

And of course, I have no clue how any of this interacts with modules where
the file name doesn't match the module declaration (which is unfortunately
very purposefully legal). It's something that I never do (and doesn't tend
to work well with some build tools), so I don't understand the details very
well, but it's the sort of detail that makes the whole situation much more
complicated.

If someone wants to write a DIP on this, then they can, but the whole idea
seems to fly in the face of current best practice in D, and honestly, the
whole idea just seems error-prone. It would probably work a decent
percentage of the time, but it looks like it has some nasty corner cases,
and it would almost certainly hurt compilation speed.

As it is, Walter approved package.d precisely because it added almost
nothing. It just used the pre-existing import rules with the change that the
compiler would know to check for the existance of package.d. The import
rules themselves didn't change at all. So, the whole thing was simple and
clean. It also was never done with the intention of making it common
practice to import entire packages at once but simply as a way to aid in
refactoring a module without breaking existing code. So, the goal which got
it approved was very different.

- Jonathan M Davis