CTFE and DI: The Crossroads of D

Adam Wilson flyboynw at gmail.com
Wed May 9 12:27:19 PDT 2012


Hello Everyone,

I am afraid that we as a community have reached an inflection point. The  
crossroads of CTFE and DI.

I recently completed my work on a patch for the DI generation system. In  
the process I solicited the feedback of the community about what should  
and should not be in a DI file. The most agreed up point was that all  
functions that can loose their implementations should. In the communities  
opinion that means only auto-functions and template-functions should  
retain their implementations.

The problem is thus: CTFE requires that any function that it could  
possibly evaluated by CTFE, must retain it's implementation.  
Unfortunately, there is simply no way for the DI generation system to know  
which functions are capable of being called by CTFE and which ones  
actually are.

This limitation is due to the fact that DI generation must be run before  
Semantic Analysis because said analysis may perform significant rewrites  
of the AST. There is even a large (for DMD) comment in the main function  
of DMD explaining that DI generation should not be moved from where it is  
due to the inconsistencies that could arise.

The patch I created currently fails in the autotester because the template  
function dur() in the druntime is called via CTFE from Phobos. Per the  
community agreed upon DI rules, the function implementation of the  
Duration constructor that is called by the dur() function is stripped away  
and CTFE fails.

We as a community need to decide how important these two features are.  
Here are the Pro's of each feature as I see it. I encourage you to add to  
this list and debate the merits of each.

Pro's for DI:
Shared libraries are useless without proper header-style interfaces to the  
code.
Can reduce compile time.
Required by business so as not to share the entire code-base of their  
product.

Pro's of CTFE:
Makes writing certain types of otherwise complicated code simple.
Very useful to systems programmers.

By my view of it, lack of DI is a major blocker to any business looking to  
use D, including mine; and that CTFE is just "nice-to-have". And I would  
argue that if D wants to see any real usage above the 0.3% it got on the  
May TIOBE index, it needs serious business investment in using the  
language. My company would love to use D, but we can't because we don't  
want to release our entire code-base; hence my work on the DI generation  
patch. I would suggest to you that almost every business looking at D is  
going to find the current DI situation ... untenable.


A Potential Solution:

In my estimation there is a solution that allows both features to retain  
their full functionality and only requires minimal rewrites to Phobos.  
However this solution CANNOT be statically enforced by the compiler,  
except though a better, more explicit, error message(s).

The core of the solution is to disallow externally dependent CTFE in all  
modules accepted into Phobos. This would also require a clear and concise  
explanation accompanying the CTFE documentation stating that calling  
external code via CTFE will most likely result in a compiler error.

The reason I am proposing this solution is that I would argue that the  
author of std.datetime choose to utilize CTFE against an external module  
(in this case, the DRuntime) when Walter has (to the best of my knowledge)  
explicitly stated that DI files in the future would not contain unneeded  
implementations and that the current DI implementation was essentially a  
hack to get *something* working. Looking at the DI generation code, I can  
tell you, it is definitely not meant to be permanent, it is filled with  
hard-coded idiosyncrasies (like it's ridiculous indenting) that had to be  
fixed.

In essence I am arguing that it is their usage of CTFE that is incorrect,  
and not the fault of DI generation. As such, those modules should be  
rewritten in light of said assumptions changing. It is my opinion that

A longer term solution would be to give CTFE the ability to work on  
functions without the source code, so long as it mets the other rules of  
CTFE and can run the required code. However, there are some serious  
security issues that would need to be cleared up before this could work.


I apologize to the author of std.datetime for the harshness of my words  
and for singling you out and to any other Phobos module authors I may have  
implicated. I mean no ill-will towards you and I am grateful for the work  
you have done in improving Phobos.

Questions? Comments? Rants? Raves?
What do you think?

-- 
Adam Wilson
IRC: LightBender
Project Coordinator
The Horizon Project
http://www.thehorizonproject.org/


More information about the Digitalmars-d mailing list