@trust is an encapsulation method, not an escape

[H. S. Teoh via Digitalmars-d]

On Thu, Feb 05, 2015 at 11:50:05PM +0000, Dicebot via Digitalmars-d wrote:
> On Thursday, 5 February 2015 at 23:43:19 UTC, Dicebot wrote:
> >The fact that @trusted is contained in small block doesn't mean rest
> >of @safe function doesn't need to be reviewed. Only difference is
> >"review all manually" vs "review all manually with some help of
> >compiler".
> 
> I believe that was also the reasoning behind H.S.Teoh proposal for
> refining @trusted semantics - to keep help from compiler but mark
> functions as @trusted to make it clear that it still needs to be
> reviewed in total.

This whole discussion has been very tiring and frustrating, because we
keep talking past each other. The reason we keep talking past each other
is because we keep conflating several distinct, though related, issues.

1) I think we all basically agree that std.file is a mess and some
solution is needed to fix this mess. I don't think anyone is actually
advocating *for* the code that's currently in std.file right now. So can
we pretty please agree that this is a given, and stop using it to beat
each other over the head?

2) I think we also all basically agree that the *intent* of @trusted is
to be an encapsulation mechanism, or to present a safe API, or however
you want to describe it. I.e., if a function is marked @safe, then it
must be impossible to cause it to do something unsafe by passing it the
wrong arguments. Whatever it does under the hood should not have any
observable unsafe effect on the outside world. I'm pretty sure everyone
also agrees with this; I don't think anyone is advocating that we should
changee this intent. So can we please also take this as a given, and
stop repeating it at each other as if we don't all already know it?

This leaves the core of the contention, which is that @safe/@trusted, as
currently implemented, presents maintenance difficulties. Walter &
Andrei don't seem to be convinced that this is a problem, but the
evidence is right before us. We Phobos committers did not deliberately
set out to sabotage @trusted by introducing blatant abuses of it just
for fun (see (1) and (2) above). The current ugly (and arguably totally
wrong) hacks are a symptom of the underlying maintenance difficulty that
the current system presents. It is a desperate attempt to contain a
maintenance problem that's growing out of hand.

You probably don't believe me, but let's assume that we all agree on (1)
and (2), and let's suppose that everything is implemented the way Walter
& Andrei envision it. That is, no ugly hacks like what's currently in
std.file, and @trusted is only ever used on functions that present a
@safe API. What maintenance issues might there be? There are at least
the following (and probably more):

- First, let's get one thing straight: the body of a function is rarely
  only written once and never touched again later. In the real world,
  function bodies tend to be written piecemeal -- you write an initial
  implementation, then refine it, then commit it, then somebody comes
  along later and fixes a bug, somebody else implements an enhancement,
  etc.. I.e., code is in a constant flux of changes. Especially in a
  public project like Phobos, these changes are pouring in everyday from
  all manner of contributors of varying levels of competence. No single
  person can possibly keep up with every code change that gets merged.

Now, with this fact as background, let's consider:

- The body of a @trusted function is completely unrestricted, and can
  contain any manner of @system operation. The compiler doesn't give you
  any help in ensuring there aren't careless mistakes that slipped in.
  Any unsafe operation you can imagine can be freely thrown into such a
  function, and the compiler will happily accept it and allow @safe code
  to freely call it.

- The fact that the compiler does zero verification of the body of a
  @trusted function, should already be a sign of deficiency, since the
  responsibility of checking correctness of the *entire* function body
  now falls upon fallible human beings. How sure are we that even the
  initial review was foolproof? At the very least, one would hope that
  the compiler could give us some help where it can, and leave the parts
  it cannot to a few isolated places where human inspection can be more
  focused on.

- Now, couple this with the volume of changes pouring into Phobos
  constantly, which means that inadvertent mistakes will creep into the
  code, either in the function body directly, or indirectly into any of
  the functions that the @trusted function calls. Some of these mistakes
  may break the @trusted promise of the function, but currently there is
  NO WARNING for this. How confident are we that changes to the function
  after the initial review did not break the @trusted promise?

- The currently recommended approach is to thoroughly review all changes
  to @trusted functions before merging. Given that just today, somebody
  merged a PR that has no unittests for verifying the correctness of a
  bugfix and preventing regressions, how likely is it that changes to
  @trusted function will be thoroughly reviewed for all possible
  implications before merging?

- Not to mention, this is a direct change to a function, that is
  immediately visible from the function itself.  We haven't considered
  yet the implications of changes to helper functions called by @trusted
  functions, which may have safety implications on previously-verified
  @trusted functions. If a direct change passes review without a proper
  accompanying unittest, how much less will changes to functions
  *indirectly* called by @trusted functions cause reviewers to inspect
  the @trusted function that calls them?

  Sure, we *could* institute a draconian policy that we must keep track
  of every change to every function that might be called by a @trusted
  function, and review all those @trusted callers everytime any one of
  these functions change. That would ensure the promise of @trusted is
  not broken. That would also ensure that nobody will maintain Phobos,
  because the burden of such a review requirement is far too onerous to
  be practical.

So basically, once a function is marked @trusted, we basically have to
accept its safety based on faith and hope. Since reviewing the entire
call tree of the function is impractical, we just need to have faith
that whoever reviewed the initial version of the function 5 months ago
did his job correctly, and we hope that nothing has changed since that
may have broken the safety of the function. External @safe callers of
the function simply take its safety on faith, because it presents a
@safe API, and there's no reason whatsoever to doubt whether or not it
is truly safe. In any case, since the compiler happily accepts @safe
code calling @trusted code, and we can't live our lives in fear that
perhaps a recent PR has introduced a change that broke the safety
promise of said @trusted code, we just have to take it on faith that the
@trusted code is, in fact, @safe.

Meanwhile, the changes continue to pour in, and the compiler doesn't
even check the most basic @safe-ty concerns in the body of the @trusted
function. When a change touches something that causes a previously @safe
operation in said function body to become @system, the compiler happily
accepts it without any warning or indication whatsoever that perhaps
there is some kind of mistake, a previously @safe primitive has now
become un- at safe, and we'd better re-review the @trusted function to
ensure that no new safety holes have been introduced.  But surely, this
isn't a problem at all, since we reviewed this @trusted function 5
months ago, and nothing could possibly have gone wrong in the interim,
-- we believe.

(After all, our reviewers are doing their jobs properly, right? That is,
the onerous obligation to review not only changes to @trusted functions,
but every change to every function that occurs in the downstream call
tree of every @trusted function. In a *volunteer* project, where people
only work on what interests them. Yes, this just fills me with
confidence that there's absolutely nothing wrong with @trusted. What do
you mean, the emperor has no clothes?)


T

-- 
Государство делает вид, что платит нам зарплату, а мы делаем вид, что работаем.