Ranges and @safe

[Jonathan M Davis]

On Sunday, September 22, 2019 5:03:32 PM MDT SrMordred via Digitalmars-d 
wrote:
> I think that .front in ranges should'nt be safe by default.
>
> @safe{ iota(0,0).front(); } //BOOM, but compiles on @safe.
>
> I can put a @system on front method, but then
> @safe{ foreach(v ; range){ ... } } //dont compile, its not safe.
> But i think that this lowered code should at least be @trusted
> since the algorithm is using the range correctly and can´t do
> unsafe things (right?).
>
> I can wrap the range in opApply and solve this, but its a dirty
> trick i think.
> What's your ideas on this?
>
> all the code i'm talking here:
> https://run.dlang.io/is/yPy26j

You do not seem to understand what @safe is for. @safe is for checking for
memory safety. @safe code throw Exceptions. It can throw Errors. It can
segfault. It can be extremely buggy. But it can't access invalid memory
(e.g. by using an index which is out-of-bounds with an array or by accessing
memory that points to an object that no longe exists).

If code is marked as @safe, the the compiler mechanically verifies for the
programmer that no operations which are considered @system are used within
that code. Barring a compiler bug, code which the compiler has mechanically
verified as being @safe cannot access invalid memory and thus has no bugs
related to memory safety. That's all that it guarantees. It doesn't
guarantee that any other bugs don't exist within that code.

If a function is marked as @system, then the compiler considers that
function to be @system and thus potentially not memory safe. So, any code
that calls will consider it @system, and when the function itself is
compiled, its memory safety is not checked by the compiler. So, you'll never
get something like the compiler flagging a function that was marked as
@system as something that's actually memory safe. You'll just get the
compiler flagging @system operations within an @safe function.

You can mark a function as @trusted to indicate that that function is @safe
without the compiler doing any checks. When you do that, you're telling the
compiler that you know that the code is memory safe and that it doesn't have
to check it. This is intended for cases where a function has @system
operations but where the programmer is able to verify that the code is
actually @safe with what it's doing with those @system operations.

For instance, pointer arithmetic is @system, because the compiler cannot
verify that you're accessing valid memory when you use pointer arithmetic.
So, a function like

int foo(int[] arr) @safe
{
    if(arr.length < 2)
        return int.min;
    auto ptr = arr.ptr;
    ++ptr;
    return *ptr;
}

would fail to compile, because it's doing pointer arithemtic. However, it's
possible for the programmer to look at that code and see that the pointer
arithmetic it's doing won't actually access invalid memory. So, they can
mark it as @trusted to inform the compiler that it can be considered @safe.
As such,

int foo(int[] arr) @trusted
{
    if(arr.length < 2)
        return int.min;
    auto ptr = arr.ptr;
    ++ptr;
    return *ptr;
}

would compile, and any code that calls foo would consider it to be @safe. As
long as the programmer was correct that foo was actually @safe, then any
@safe code that calls foo is guaranteed to not violate memory safety.
However, if the programmer gets it wrong, then there's a bug in the program
which could result in it accessing invalid memory, and the @safe code isn't
actually memory safe. So, with @trusted, it's the programmer's
responsibility to get it right, and the memory safety of any @safe code that
calls that @trusted code relies on the programmer having gotten it right.

Because the @safety of templated functions often depends on the template
arguments (e.g. std.algorithm's find is @safe if the range that it's given
is @safe, but it wouldn't be @safe if it was given a range where one or more
of its range API functions were @system), @safe is inferred for templates.
So,

int foo()(int v)
{
    return v;
}

would have its @safety inferred by the compiler, and because foo contains no
@system operations, it's considered @safe. Similarly,

int foo()(int* ptr)
{
    ++ptr;
    return *ptr;
}

would be inferred to be @system, because it contains @system operations.
However, explicitly marking the function as @safe, @trusted, or @system
works exactly the same as it would on a non-templated function.

@safe is also inferred for functions whose return type is inferred. e.g.

auo foo(int v)
{
    return v;
}

would be inferred to be @safe, whereas

auto foo(int* ptr)
{
    ++ptr;
    return *ptr;
}

would be inferred to be @system. As with templated functions, explicitly
marking the function with @safe, @trusted, or @system overrides the
attribute inference and @safety is checked per the attribute that was
provided just like with a normal function.

As for ranges, the vast majority of range-based code is templated, so the
vast majority of it uses attribute inference, but regardless, the compiler
doesn't treat ranges as special in any way shape or form with regards to
@safety.

Your example of iota(0, 0).front is perfectly @safe. The compiler correctly
infers iota's front to be @safe, because it does nothing which violates
memory safety. Any program that has iota(0, 0).front is buggy, because it's
calling front on an empty range, and if that program is not compiled with
-release, then an assertion in iota's front will fail, resulting in an
AssertError being thrown, but none of that violates @safety, because it
doesn't do anything that can access invalid memory. It's buggy, and any
program that does it needs to be fixed, but it's @safe.

As for your larger example, you've marked Range's front as @system, so it
will be treated as @system even though it's not doing anything that violates
memory safety, whereas you've marked its empty and popFront as @trusted, so
they'll be considered @safe regardless of whether they violate memory safety
(which they don't). As such, the code

    auto range = Range(2);
    foreach(v ; range ) writeln(v); //not safe!

correctly gets flagged by the compiler as being @system. It can't actually
violate memory safety, but the compiler doesn't know that. It's just
treating Range's front as @system, because you told it to.

You then marked RangeWrap's entire implementation as @trusted. So, of course

    auto range = Range(2);
    auto range_wrap = RangeWrap(range);
    foreach(v ; range_wrap ) writeln(v); //fine, but its a trick.

compiles, and it's not a bug. You told the compiler to treat RangeWrap's
functions as @trusted, so the compiler doesn't check them, and it's up to
you to verify that they're actually @safe. RangeWrap's opApply calls the
@system function front on Range, so it's doing an operation that the
compiler considers @system, but you marked RangeWrap's opApply as @trusted,
so if it's not memory safe to call Range's front, then it's up to you to
catch that and fix it. There is no compiler bug here.

Of course, none of this code is actually doing anything that isn't memory
safe - presumably, you're just marking it the way you did to provide an
example without having to have actual @system operations in Range's front -
but all of the checks that the compiler is or isn't doing based on those
attributes are correct. Any time that you use @trusted, you have to manually
verify the code yourself for whether it violates memory safety. The compiler
trusts you when you use @trusted. So, don't expect it to catch any @safety
violations in @trusted code. You told it that that code was @safe when you
used @trusted.

And again, @safe is all about memory safety. Exceptions, Errors, and
segfaults are all memory safe. Many, many bugs are memory safe. Don't expect
@safe to catch all of the various bugs in your program or even to prevent
all crashes. It's just catching code that is potentially not memory safe,
and when you use @trusted, you're telling it to not check that code, so
any memory safety bugs in that code are on you.

- Jonathan M Davis