Yet another leak in the sinking ship of @safe

Thu Feb 18 12:24:18 PST 2016

On Thu, Feb 18, 2016 at 07:25:16PM +0000, Jonathan M Davis via Digitalmars-d wrote:
> On Thursday, 18 February 2016 at 18:58:56 UTC, H. S. Teoh wrote:
> >On Thu, Feb 18, 2016 at 06:50:34PM +0000, Jonathan M Davis via
> >Digitalmars-d wrote:
> >>On Thursday, 18 February 2016 at 18:41:25 UTC, Steven Schveighoffer
> >>wrote:
> >[...]
> >>>foo(void[] arr)
> >>>{
> >>>   void[] arr2 = [1234, 5678, 91011];
> >>>   arr[] = arr2[0 .. arr.length];
> >>>}
> >>
> >>Well, I'm not sure that that's actually not @safe.
> >
> >How can it possibly be @safe??? Consider:
> >
> >	void main() @safe {
> >		Object[] objs = [ new Object() ];
> >		foo(objs);
> >	}
> >
> >Now the pointer to the Object instance has been corrupted.
> 
> Does it matter what state the void[] is in until you actually attempt to
> cast it to something else? If you have
> 
> Object[] oArr = [ new Object ];
> void[] vArr = oArr;
> vArr = vArr[0 .. $ - 1];
> 
> it's not like that's going to cause any problems - not by itself.

I think you misread Steven's code.  This has nothing to do with reducing
the length of an array. Read the code again.  It has to do with
*overwriting* the contents of one void[] with another void[], which is a
legal and @safe operation under the current rules. Unfortunately, the
first void[] is an array of Object references, and the second void[] is
an array of ints, so this is overwriting pointers with arbitrary int
values.

Furthermore, as the caller of foo(), main() has no idea what has
happened to its Object[], because it continues to see the same area of
memory as Object[], even though it has been overwritten by an array of
ints.  So when it next tries to dereference the Object[], which is
perfectly legal and does not involve any casting, it ends up
dereferencing an arbitrary int as a pointer instead. Ergo, @safe has
been compromised.

[...]
> >I think you missed the point of his example. :-) The point is that
> >it's perfectly legal to (1) cast an array of int to void[], and (2)
> >it's also perfectly legal to cast an array of anything to void[], and
> >(3) under current rules, it's perfectly legal to copy one void[] to
> >another void[].
> >
> >Arguably, (3) should not be allowed in @safe code. Which again brings
> >us back to the point, that if any function takes void[] as an
> >argument, is there *anything* it can do with the void[] other than
> >reading it, that *won't* break @safe?
> >
> >If there's *nothing* it can legally do with a void[] (other than
> >reading it) without violating @safe, then shouldn't it be a
> >requirement that all functions marked @safe must take const(void)[]
> >rather than void[]?
> 
> It could pass it to something else without actually doing anything to
> it, in which case, I don't see why it couldn't be @safe. It's
> interpreting a void[] that's the problem. _That_ needs to be @system.
> Merely converting to void[] or passing it around is harmless.

No. Please look at Steven's code again. What it is doing, is (1)
implicitly casting Object[] to void[], which is perfectly legal
according to what you said. Then (2) it casts an int[] literal to
void[], again, another perfectly legal operation. Finally, (3) it copies
the contents of the second void[] to the first void[]. Since both arrays
are typed void[] at this point, the copy is perfectly legal (or so the
compiler thinks), and here is where the Object[] got overwritten by an
int[] and the compiler did not realize it.

Then back in main(), the code does not see void[] at all; all it sees is
the Object[] it has created all along.  There is no interpretation of
void[] going on here at all.  All that happened was (1) we passed
Object[] to void[], which is legal according to what you said, and (2)
we dereferenced the Object[], which is not suspect at all because, well,
it's an Object[], what could possibly go wrong?  At this point, however,
we are dereferencing an illegal pointer, because foo() has corrupted our
Object[]. Note that *nowhere* in main() is there any reinterpretation of
void[] as Object[].  The only thing that led to this mess, as far as
main() is concerned, is that it passed an Object[] to a function that
takes void[].

In other words, from the caller's POV, as soon as you pass an array to a
function that takes void[], all bets are off, anything could happen to
your precious data, and @safety is no longer guaranteed.  However, foo()
is marked @safe and the compiler does not complain.  Also, inside foo()
we did nothing unseemly, according to current rules.  All we did was to
convert two arrays to void[], which is perfectly legal and harmless, as
you claim, and copy data from one void[] to another void[], which, under
the current rules, is perfectly legal since we are just copying data
between two arrays *of the same type*. What could possibly go wrong with
copying data between two arrays of the same type?

The only problem is, they *aren't* of the same type after all. Just
because is(void[] == void[]) is true does NOT mean it is @safe to copy
one array to another, because the void here is a wildcard placeholder
representing arbitrary types.

So the bottom line is that the array copy cannot be @safe.

The larger question, is what, if anything, *can* you do with void[]
besides read-only operations, that doesn't break @safe?  Once something
is (implicitly or otherwise) cast to void[], all type information is
forgotten, and there is no way, that I can tell, to write to a void[]
without causing @safe breakage.  If so, wouldn't it make sense to
require that the type should be const(void)[] rather than void[]?

[...]
> The problem with std.socket is not that it accepts void[] and
> considers that @safe; it's that it passes that void[] to something
> else, treating that as @safe, when it's not. And since it's a C
> function that's being called, and it accesses the ptr member of the
> void[], it's already @system, forcing @trusted to be used to make the
> compiler happy. So, the problem is that @trusted was used when it
> shouldn't have been, not that the type system allowed void[] to be
> passed in without marking it as @system. And since the function was
> marked @trusted, it's not like having the compiler treat a function
> that accepted void[] as @system would have helped anyway.
[...]

Certainly, the @trusted was wrongly applied to that particular function.
That's easy to fix: remove @trusted and mark it as @system, as it should
be.  Or, to be a bit nicer to our current users, qualify it with a sig
constraint so that it only accepts arrays that have no further
indirections.

But I'm thinking about the larger question here.  If modifying the data
in a void[] is *always* @system, because of the type erasure that
basically eliminates all possibility of the lower-level function
recovering the type and adapting its operation to be @safe, then why
allow passing void[] in @safe code in the first place?  If *any*
modification to the data in the void[] carries the risk of breaking
@safe, then shouldn't we require that we only allow const(void)[] in
@safe code?

It's not as if we don't have @trusted as a fallback in those cases where
we *know*, outside the type system, that a particular function call is
actually safe even though it's marked @system.  But the point is, by
disabling this dangerous operation *by default* (albeit allowing it via
the @trusted escape hatch), we eliminate the possibility of screwups
like Steven's code demonstrated.

T

-- 
Never step over a puddle, always step around it. Chances are that whatever made it is still dripping.