Compiler as dll

[grauzone]

Alexander Pánek wrote:
> grauzone wrote:
>> John Reimer wrote:
>>> ddl does not work for memory sharing like normal dll's, where 
>>> multiple applications have access to a single dll at runtime.  It 
>>> appears that such support would be quite difficult to implement and 
>>> moves in the direction of operating system features.
>>
>> Couldn't this be achieved by simply mmap()-ing the file contents into 
>> memory? mmap() normally shared the memory pages with other processes.
>>
>> Of course, this wouldn't work if the code both isn't position 
>> independent, and needs to be relocated to a different base address. 
>> But that's also the case with operating system supported dynamic 
>> shared objects.
>>
>>> It does do runtime linking, however, which is extremely useful for 
>>> certain situations... specifically any sort of application that needs 
>>> a plugin architecture for D (ie.. it can link with libraries and 
>>> object files at runtime) that is gc and exception friendly.  
>>
>> I never understood why this is needed. Can't they simply compile the 
>> plugins into the main program?
> 
> Well, compiling them directly into the main program kinda defeats the 
> purpose of runtime-pluggable plugins, wouldn’t it?

But why do you want them in the first place? For Open Source projects, 
this seems to be completely pointless to me.

>> When it's a commercial program, the DLL plugin approach probably 
>> wouldn't work anyway: in order to enable others to compile plugins, 
>> you would need to expose your internal "headers" (D modules). Note 
>> that unlike in languages like C/C++, this would cause internal modules 
>> to be exposed too, even if they are not strictly needed. What would 
>> you do to avoid this? Maintain a separate set of import modules?
> 
> Make use of .di files. You don’t have to distribute code.

D interface files (.di) files are what I meant by "import module", sorry 
about this. They are compiler specific, and the only intended purpose is 
speeding up the compilation. Quoting from the D site:

 > D interface files bear some analogous similarities to C++ header
 > files. But they are not required in the way that C++ header files are,
 > and they are not part of the D language. They are a feature of the
 > compiler, and serve only as an optimization of the build process.

http://www.digitalmars.com/d/2.0/dmd-linux.html#interface_files

I looked at the Tango .di files, which are (I think) automatically 
generated by the D compiler. I noticed several things:

- Private symbols are included, even private imports or private class 
members => they are exposed to the public, and changing them might break 
ABI compatibility under circumstances.

- All transitive imports seem to be included => you either expose your 
internal modules as interface file, or your public modules must not 
(transitively) import private modules. Note that this forbids direct use 
of any private type or function. You'll probably have to program around 
using indirections like interfaces or abstract base classes. Also note 
that this is way harder as in C: unlike in D, there are incomplete 
types, and the implementation (.c files) is completely separate and can 
import any private headers.

- Sometimes, the full code for methods or functions is included, 
although they are not templated in any way. I guess it's even possible 
that plugins will inline those functions. This means changing these 
functions could randomly break already compiled plugins. Of course, this 
can be fixed, but nobody has bothered yet. It probably shows that .di 
files weren't designed with ABI compatibility in mind.

It seems that dynamic linking with D code is extremely fragile, and it 
requires serious extra effort to maintain ABI compatibility. Please 
correct me if I'm wrong.