join() in CTFE very low performance

[monkyyy]

On Saturday, 4 January 2025 at 13:56:47 UTC, realhet wrote:
> Hello,
>
> I have an array of array of strings, a 2D table encapsulated in 
> a struct:
>
> The first few rows look like this.
> ```d
> enum TBL_niceExpressionTemplates =
> (表([
> 	[q{/+Note: Name+/},q{/+Note: Example+/},q{/+Note: 
> Pattern+/},q{/+Note: op+/},q{/+Note: Style+/},q{/+Note: 
> Syntax+/},q{/+Note: Class+/},q{/+Note: Scripts @init: @text 
> @node @draw @ui+/}],
> 	[q{null_},q{},q{/+Code:+/},q{""},q{dim},q{Whitespace},q{NiceExpression},q{}],
> 	[q{magnitude},q{(magnitude(a))},q{/+Code: 
> (op(expr))+/},q{"magnitude"},q{dim},q{Symbol},q{NiceExpression},q{@text: put(operator); op(0); @node: put('|'); op(0); put('|'); }],
> 	[q{normalize},q{(normalize(a))},q{/+Code: 
> (op(expr))+/},q{"normalize"},q{dim},q{Symbol},q{NiceExpression},q{@text: put(operator); op(0); @node: put('‖'); op(0); put('‖'); }],
> ...
> ]));
> ```
> I have around 50 rows total, not much for a computer.
>
> Then I use the 'table' and try to generate an actual static 
> immutable array of runtime structs.
>
> I use an own makeNiceExpressionTemplate(string[] args) function 
> to convert those table rows into the runtime used structs.
>
> ```d
> static immutable niceExpressionTemplates = 
> TBL_niceExpressionTemplates.rows.map!makeNiceExpressionTemplate.array;
> ```
> This method is perfectly fine, it must be under a millisecond, 
> I can't see it in the ftime-trace.
>
> But when I try to put this table together using a string mixin, 
> it goes extremelyi slow:
> ```d
> mixin(iq{static immutable niceExpressionTemplates = 
> [$(TBL_niceExpressionTemplates.rows.map!((r)=>(iq{makeNiceExpressionTemplate($(r.text))}.text)).join(','))]; }.text)
> ```
>
> It took 2.6 seconds!!!
>
> It concatenates 50+ strings like this -> 
> makeNiceExpressionTemplate(["null_", "", "/+Code:+/", "\"\"", 
> "dim", "Whitespace", "NiceExpression", ""]),
> into a single long string.
> Put an array declaration 'container' around it, and then gives 
> it to mixin()
>
> Then the weird thing happens:
> Even the makeNiceExpressionTemplate() is called with the sampe 
> parameter: a string array, it generates all the code for it 
> again and again.
> Exactly that many times as the join() template is executed on 
> it.
>
> I narrowed down the code as much as possible:
> ```d
> static immutable very_slow_operation = 
> TBL_niceExpressionTemplates.rows.map!text.join(',');
> ```
> It is a combination of text(), join(), and formatting string 
> arrays to text.
>
> My only question is why?
> What is the exact thing I should avoid, why join() recompiles 
> their iterations from zero all the time?
>
> Thank You in advance!

Id check that it is a string and not some sort of lazy wrapper 
type doing worse of both world things; I usually use `enum 
string[]` for mixin-y things when possible, idk what style your 
doing and the q{} may have some extra logic.