Git, the D package manager

[Atila Neves via Digitalmars-d]

I have to say that the directory naming issue bit me; I had to 
move all of my code to "source" in order to use dub, and then I 
end up with "cerealed/source/cerealed/*.d" which is kind of silly.

rdmd is awesome, I use dub when the D program I'm writing has dub 
package dependencies. For vibe.d, for instance, I really don't 
want to have to figure out how to build it myself, and rdmd won't 
cut it there, at least not without major contortions.

But both rdmd and dub fall short with anything remotely 
complicated. I've used a lot of CMake features that I simply 
don't have with either of them.

Atila

On Monday, 2 February 2015 at 08:09:39 UTC, Vladimir Panteleev 
wrote:
> On Monday, 2 February 2015 at 05:23:52 UTC, Daniel Murphy wrote:
>> "Vladimir Panteleev"  wrote in message 
>> news:viqwfixznbdbdwvhavuk at forum.dlang.org...
>>
>>> I don't use Dub
>>
>> You really should!  I put it off for months and months but I'm 
>> quite happy with it now.
>
> Even if I had faith that dub was a perfectly polished piece of 
> software, it doesn't solve any problems I have with building D 
> programs, and in fact would make said task more complicated. 
> Here's why.
>
> 1. rdmd
>
> rdmd is a simple and effective way to build D programs, and I'm 
> sad to see its use declining.
>
> rdmd leverages two things: D's module system, and the 
> compiler's import search path. Dub's design seems to ignore 
> both of the above.
>
> 1a. rdmd and D's module system:
>
> When you run `dmd -o- program.d`, the compiler will 
> automatically read all modules imported by your program, and 
> their imports, and so on. It does so by searching the 
> filesystem across its search path for matches which correspond 
> with D's module system, and only reads those files that are 
> needed.
>
> rdmd leverages this by collecting (and caching) the list of 
> modules used in the program, and passing that list to the 
> compiler. The compiler will then compile no more and no less 
> than the exact set of modules transitively imported by the 
> program.
>
> In contrast, Dub's default modus operandi is to blindly send to 
> the compiler all *.d files found in the "src" folder, whether 
> they're actually used or not. Not only can this be slower if 
> not all modules are always used, but it also won't work if the 
> source code contains multiple entry points, forcing you to 
> write complicated configuration files (i.e. do the computer's 
> work for it).
>
> 1b. rdmd and D's search path
>
> rdmd does not have any additional parameters to set up for 
> where it needs to look for source files, because it relies on 
> the compiler's search mechanism. Thus, if you can build your 
> program with rdmd, "dmd -o- program" will succeed, and usually 
> vice versa.
>
> In contrast, Dub builds its own search path using its JSON 
> configuration files, and has no equivalent of "dmd -o-".
>
> There is no simple way to syntax-check just one file in a 
> project when using Dub. I rely on this a lot in my workflow - I 
> configured a syntax-check key in my editor, which I use almost 
> as often as I save. A syntax check (dmd -o-) is much faster 
> than a full build, as it skips parsing other parts of the 
> project, code generation, and linking.
>
> 2. git
>
> I've found that git's submodule feature is an excellent way to 
> manage dependencies across D libraries, and fits really well 
> with D's package system. For clarity to readers not too 
> familiar with Git, I'll explain by example:
>
> 2a. Directory structure
>
> If you have a library "fruit" with the modules "fruit.apple" 
> and "fruit.banana", create a git repository (in a directory 
> called "fruit") with the files apple.d and banana.d.
>
> Now, in your project (e.g. "orchard"), add a symlink called 
> "fruit" pointing to the repository:
>
> ~/
> |- fruit/
> |  |- .git/
> |  |- apple.d
> |  '- banana.d
> '- orchard/
>    |- .git/
>    |- fruit -> ~/fruit/
>    '- tree.d
>
> tree.d can now import the "fruit.apple" module without any 
> additional compiler switches. `rdmd tree` will just work. This 
> is because the modules of the "fruit" library are placed at the 
> repository root, and the repository's directory name matches 
> with the library's package name, so when the compiler will look 
> for the "fruit.apple" module, it will find it at 
> "fruit/apple.d".
>
> 2b. Distribution
>
> Upload the "fruit" repository to GitHub. In your "orchard" 
> project, delete the symlink, and run:
>
> git submodule add https://github.com/You/fruit fruit
>
> This will clone the repository from GitHub, create a 
> .gitmodules file, and add an entry to the git index. If you 
> commit all these, and put "orchard" on GitHub too, someone can 
> run the command:
>
> git clone --recursive https://github.com/You/orchard
>
> ... to get the orchard project, and its dependencies (the fruit 
> library).
>
> 2c. Versioning
>
> Git stores the exact commit of each submodule in the parent 
> repository.
>
> So, if you make a commit in orchard/fruit/, and run `git 
> status` in orchard/, git will show you that the "fruit" 
> submodule has been modified.
>
> Most importantly, this means that any breaking change in the 
> "fruit" library will never affect existing projects which use 
> it, since they will continue to use the registered commit which 
> was known to work.
>
> This gives you dependency versioning with commit granularity - 
> you can hardly ask for something better - all without messing 
> with configuration files.
>
> ---
>
> The one thing I wish git had is symlink support for Windows. 
> Windows has symlinks and directory junctions, but msysgit still 
> doesn't support them.
>
> For a practical instance of the above example, see Digger:
>
> https://github.com/CyberShadow/Digger
>
> Note the simple build instructions:
>
> https://github.com/CyberShadow/Digger#building
>
> 2d. Git vs. Dub
>
> Unfortunately, the above-described approach is not compatible 
> with Dub:
>
> - Dub packages are generally expected to have their source code 
> in a "src" subdirectory, although you can set the source 
> directory to "." in the configuration file.
>
> - When cloning repositories, dub does not preserve the 
> repository's directory name (so e.g. fruit will be cloned to 
> ~/.dub/fruit-1.0.0/).
>
> Somebody has created a Dub package for my library (excluding 
> certain packages, due to point 1a above), and the other day 
> someone complained that it doesn't work with Dscanner, because 
> of the above issue - the module path "ae.dir.module" does not 
> correspond to the filesystem path "ae-1.0.1/dir/module.d".
>
> So, in order to start using Dub, I'd need to:
>
> - restructure the directory layout of my library (breaking 
> change)
> - update all projects which use this library to use Dub instead
> - give up quick syntax checking
> - give up commit-granularity versioning
> - begin maintaining JSON configuration files
> - begin versioning libraries by hand
> - install Dub on all my computers, servers, and virtual machines
>
> No thanks.
>
> I could invest time in improving Dub to fix or ameliorate some 
> of the above points, but I don't see a compelling reason to. In 
> fact, I think we should integrate rdmd into dmd - dmd clearly 
> already knows which source files participate in compilation, as 
> all rdmd does is basically take dmd's output and feed it back 
> to it. This will greatly speed up compilation, too.
>
> Change my view.