Returning variable-sized stack data

[IchorDev]

On Wednesday, 17 July 2024 at 09:31:37 UTC, Nick Treleaven wrote:
> Isn't that a stack overflow footgun?

How so?

> Wouldn't that allocate on `foo`'s stack? It needs to be on the 
> caller's stack.

Depends on whether you place the alloca call in the caller or the 
callee.
This brings us to an important implementation question: what if 
there's 3 layers of function calls, and all of them each want to 
return a variadic stack allocation? The example in my OP could 
certainly be chained with many returns, but perhaps there's a 
better way? What if we just return a pointer to the stack memory, 
use alloca, and then copy the data to the newly allocated 
pointer? Granted, the newly allocated memory will overlap the old 
memory if it's sufficiently large, but this shouldn't be a 
problem under any calling conventions that I'm familiar with. 
This example works with dmd >= 2.084.1, but fails with ldc2/old 
dmd for some reason:
```d
import std.typecons;
struct A{ int a,b,c,d,e,f,g; }
struct B{ int a,b,c; }
void[] myFn(int n){
	import core.stdc.stdlib: alloca;
	auto nSqr = n * n;
	if(nSqr == 0){
		auto __returnPtr = alloca(A.sizeof);
		*cast(A*)__returnPtr = A(nSqr+1,2,3,4,5,6,7);
		return __returnPtr[0..A.sizeof];
	}else{
		auto __returnPtr = alloca(B.sizeof);
		*cast(B*)__returnPtr = B(nSqr,2,7);
		return __returnPtr[0..B.sizeof];
	}
}
void main(){
	int n = 0; //<—— can be any number
	void[] myMem;
	{
		import core.stdc.stdlib: alloca;
		void[] __returnMemory = myFn(n);
		myMem = alloca(__returnMemory.length)[0..__returnMemory.length];
		myMem[] = __returnMemory[];
	}
	import std.stdio;
	writeln(cast(int[])myMem);
}
```