Discussion on static reflection syntax in C++

[Paul Backus]

On Tuesday, 23 February 2021 at 14:14:13 UTC, Stefan Koch wrote:
> On Monday, 22 February 2021 at 23:44:45 UTC, Paul Backus wrote:
>>
>> Aren't macros in Lisp just regular Lisp code, too? :)
>>
>> As far as I'm concerned, a procedural macro is a function that
>>
>> 1. runs at compile time,
>> 2. takes AST nodes as input, and
>> 3. produces AST nodes as output.
>>
>> Type functions fit those criteria perfectly. Currently, they 
>> only work on a subset of AST nodes (types), so they're not a 
>> *complete* implementation of procedural macros, but generalize 
>> them enough and that's what you'll get.
>
> They don't exactly take AST nodes.
> They take a constrained (stabilized) datastructure which 
> reflects a subset of the ast.
> At least conceptually.
> They don't create ast nodes, but regular values, which may be 
> converted to ast-nodes in the case where they are statically 
> available. (currently only types)

This is also how macros work in some Lisp implementations. For 
example, in Racket, the AST nodes (called "syntax objects") are 
converted to S-expressions by the function `syntax->datum`, the 
S-expressions are manipulated by CTFE, and the result is 
converted back into an AST node by `datum->syntax`. [1]

> My currently exploration leads me to a path where you can 
> _extend_ open scopes within typefunction by using special 
> functions which map provide a stable mapping to compiler 
> intrinsics.
> such as __struct* __add_struct(string name, __symbol[] members);
> the __struct* will not be a proper struct type unless
> __type__ __add_to_scope(__scope Scope, __struct* s);
> has been called.

I haven't fully thought this through, but would it be possible to 
overload `mixin` to do this? E.g.,

     import core.ast: Struct, Type;

     // some type function
     Struct pair(Type t) { ... }

     alias PairOfInts = mixin(pair(int));

Conceptually, `mixin` already means "take this stuff and inject 
it into the current scope," and grammatically it's allowed to 
appear as both an expression and a type.

> At which point any modification of the __struct* is an error 
> and will abort compilation.

If __struct is only the "user-space" representation, and not the 
real AST node, I don't see why this is necessary, since modifying 
the __struct after mixing it in wouldn't have any affect on the 
actual AST.

> I am still searching for a solution which is more declarative, 
> but I fear for heavy usage (10_000 new types and declarations 
> added) scenarios the procedural imperative style is just more 
> optimizable.
>
> I am always open to syntax suggestions, as I don't like the one 
> I have :)

In Lisp, the declarative syntax is typically implemented as 
syntax sugar on top of the procedural syntax, using the 
`quasiquote` or `backquote` macro. [2][3] In D, one could imagine 
doing something similar with string mixins: generate the 
procedural code at compile time from a declarative DSL, then mix 
it in.

[1] https://docs.racket-lang.org/guide/stx-obj.html
[2] https://docs.racket-lang.org/guide/qq.html
[3] 
https://www.ccis.northeastern.edu/home/futrelle/teaching/lisp/cltl/clm/node190.html#BACKQUOTE