Compiler as dll

[Don]

grauzone wrote:
> Alexander Pánek wrote:
>> grauzone wrote:
>>> John Reimer wrote:
>>>> ddl does not work for memory sharing like normal dll's, where 
>>>> multiple applications have access to a single dll at runtime.  It 
>>>> appears that such support would be quite difficult to implement and 
>>>> moves in the direction of operating system features.
>>>
>>> Couldn't this be achieved by simply mmap()-ing the file contents into 
>>> memory? mmap() normally shared the memory pages with other processes.
>>>
>>> Of course, this wouldn't work if the code both isn't position 
>>> independent, and needs to be relocated to a different base address. 
>>> But that's also the case with operating system supported dynamic 
>>> shared objects.
>>>
>>>> It does do runtime linking, however, which is extremely useful for 
>>>> certain situations... specifically any sort of application that 
>>>> needs a plugin architecture for D (ie.. it can link with libraries 
>>>> and object files at runtime) that is gc and exception friendly.  
>>>
>>> I never understood why this is needed. Can't they simply compile the 
>>> plugins into the main program?
>>
>> Well, compiling them directly into the main program kinda defeats the 
>> purpose of runtime-pluggable plugins, wouldn’t it?
> 
> But why do you want them in the first place? For Open Source projects, 
> this seems to be completely pointless to me.
> 
>>> When it's a commercial program, the DLL plugin approach probably 
>>> wouldn't work anyway: in order to enable others to compile plugins, 
>>> you would need to expose your internal "headers" (D modules). Note 
>>> that unlike in languages like C/C++, this would cause internal 
>>> modules to be exposed too, even if they are not strictly needed. What 
>>> would you do to avoid this? Maintain a separate set of import modules?
>>
>> Make use of .di files. You don’t have to distribute code.
> 
> D interface files (.di) files are what I meant by "import module", sorry 
> about this. They are compiler specific, and the only intended purpose is 
> speeding up the compilation. Quoting from the D site:
> 
>  > D interface files bear some analogous similarities to C++ header
>  > files. But they are not required in the way that C++ header files are,
>  > and they are not part of the D language. They are a feature of the
>  > compiler, and serve only as an optimization of the build process.
> 
> http://www.digitalmars.com/d/2.0/dmd-linux.html#interface_files

They were also intended to act as header files, to conceal 
implementation details.

> I looked at the Tango .di files, which are (I think) automatically 
> generated by the D compiler. I noticed several things:
> 
> - Private symbols are included, even private imports or private class 
> members => they are exposed to the public, and changing them might break 
> ABI compatibility under circumstances.
> 
> - All transitive imports seem to be included => you either expose your 
> internal modules as interface file, or your public modules must not 
> (transitively) import private modules. Note that this forbids direct use
> of any private type or function. You'll probably have to program around 
> using indirections like interfaces or abstract base classes. Also note 
> that this is way harder as in C: unlike in D, there are incomplete 
> types, and the implementation (.c files) is completely separate and can 
> import any private headers.
> 
> - Sometimes, the full code for methods or functions is included, 
> although they are not templated in any way. I guess it's even possible 
> that plugins will inline those functions.

Yes, I think that's why they're included.

  This means changing these
> functions could randomly break already compiled plugins. Of course, this 
> can be fixed, but nobody has bothered yet. It probably shows that .di 
> files weren't designed with ABI compatibility in mind.

> 
> It seems that dynamic linking with D code is extremely fragile, and it 
> requires serious extra effort to maintain ABI compatibility. Please 
> correct me if I'm wrong.