static array is not a range

[Jonathan M Davis]

On Tuesday, January 9, 2024 4:13:23 AM MST Alexibu via Digitalmars-d-learn 
wrote:
> On Tuesday, 9 January 2024 at 10:44:34 UTC, Jonathan M Davis
>
> wrote:
> > How would it even be possible for a static array to be a range?
> > It has a fixed length. For a type to work as a range, it needs
> > to be possible to pop elements off of it, which you can't do
> > with a static array. Input ranges must have front, popFront,
> > and empty. Dynamic arrays have that from std.range.primitivies
> > via UFCS (Universal Function Call Syntax), and that works,
> > because it's possible to shrink a dynamic array, but it won't
> > work with a static array, because its size will be fixed.
> >
> > Now, what you can do is slice a static array to get a dynamic
> > array which refers to the static array. And since dynamic
> > arrays work as ranges, you can use that with range-based
> > functions. That being said, you do then have to be careful
> > about the dynamic array (or any ranges which wrap it) escaping
> > from the scope where the static array is, because if the static
> > array goes out of scope and is destroyed, then any dynamic
> > arrays referring to it will be referring to invalid memory, and
> > you'll get undefined behavior. So, while slicing static arrays
> > can be very useful, it needs to be done with caution.
> >
> > - Jonathan M Davis
>
> All good information, thanks.
> I suppose I use ranges as things that can be arguments to
> algorithms in std.algorithm and std.range.
>
> Although there is no state in the static array itself as you
> point out, couldn't we have a temporary input range created and
> then the compiler can elide it into whatever the foreach loop
> does.
> So all the range algorithms could auto convert a static array
> into a range backed by the static array ?
>
> Something like this : (although written by someone more competent)
>
> ```d
>
> struct TempRange(X)
> {
>     x[n] * array;
>     size_t i;
>     this(static_array a)
>     {
>        array = a;
>        i = 0;
>     }
>     X popFront() { return array[i]; }
>     bool empty() { return i == array.length;}
> }
>
>
> R each(R,F)(R r,F f)
> static if (isInputRange!R)
> {
>     normal implementation
> }else if (isStaticArray!R)
> {
>     return TempRange(r).each(f);
> }
> ```

If you want a range backed by a static array, simply slice the static array
to get a dynamic array. e.g.

int[5] a = [1, 2, 3, 4, 5];
int[] arr = a[];

However, it's not something that should be done automatically, because
having any kind of pointer or reference to a static array poses the risk of
leaking a pointer or reference to the stack - i.e. the exact same problem
that you get when taking the address of a local variable. The scope
attribute has a limited ability to track escaping references (and DIP 1000
increases those abilities), but ultimately, if you're doing stuff like
passing a dynamic array that's a slice of a static array to range-based
functions, there's a decent chance that the compiler will not be able to
properly detect whether any references to the static array actually escape
(which with DIP 1000 tends to mean errors about not being allowed to do
stuff, because the compiler can't prove that what you're doing won't escape
any references). If you're careful, you can slice a static array and pass
the resulting dynamic array to a range-based function, and it'll work just
fine, but you have to be very careful that no references / pointers to the
static array escape, or you're going to end up referring to memory that used
to be static array but is no longer, which would be a serious problem.

Any user-defined type that you created which was a pointer to a static array
would have the same problem as slicing the static array. If anything, you'd
basically just be implementing a more limited form of D's dynamic arrays
with such a type. Fundamentally, there really isn't a fully safe way to pass
around a pointer to a static array without risking escaping references - not
unless the compiler is smart enough to fully determine whether a reference
might escape, and it's quite difficult for the compiler to be that smart -
particularly when calling functions where the compiler can't necessarily see
the source code.

Ultimately, you really don't want anything to automatically slice a static
array or take its address, because you're risking undefined behavior from
references that escape.

Static arrays are nice in that they provide a way to have an array of
elements without allocating anything on the heap, but if you're going to
start passing them around, pretty quickly, you want a dynamic array that
refers to memory on the heap and not a static array. Slicing static arrays
does provide a middle ground, but it's not completely safe to do so and
really can't be, so having it be done implicitly for you is pretty much just
asking for bugs.

Unfortunately, if you pass a static array to a function that explicitly
takes a dynamic array of the type you get when slicing the static array, the
static array will be sliced automatically for you (which was a design
decision that was made for convenience many years ago without taking into
account how error-prone it is), but DIP 1000 tries to fix that by making it
so that slicing a static array is always scope, meaning that passing a
static array to a function that takes a dynamic array won't work
automatically unless the function paramter is also scope.

Regardless, range-based functions are templated, so they take static arrays
as static arrays and never slice them automatically. So, if you want to pass
a static array to a range-based function, you'll need to slice it explicitly
- and then be careful to make sure that no references to that static array
outlive the static array.

- Jonathan M Davis