Fantastic exchange from DConf

[Moritz Maxeiner via Digitalmars-d]

On Friday, 19 May 2017 at 15:12:20 UTC, Steven Schveighoffer 
wrote:
> On 5/19/17 9:46 AM, Moritz Maxeiner wrote:
>> On Friday, 19 May 2017 at 11:53:57 UTC, Steven Schveighoffer 
>> wrote:
>
>>> This provides a foundation to build completely @safe 
>>> libraries.
>>
>> Agreed if you mean libraries being marked completely as @safe 
>> (which I
>> assume).
>> Disagreed if you mean libraries that are proven to never 
>> corrupt memory
>> (not possible with unsafe operating system).
>
> I mean libraries which only contain @safe and @system calls.
>
> i.e.:
>
> $ grep -R '@trusted' libsafe | wc -l
> 0

In that case, agreed. There will be no need (with regards to 
memory safety) to audit such libraries.

>
>>> 2. @trusted blocks in any project need to be considered red 
>>> flags. You
>>> should not need to audit @safe code.
>>
>> Yes you do, because it can call into @trusted like this:
>>
>> ---
>> void foo(int[] bar) @safe
>> {
>>   () @trusted {
>>     // Exploitable code here
>>   }();
>> }
>> ---
>>
>> You *must* audit third party @safe code for such hidden 
>> @trusted code
>> (e.g. grep recursively through such third party code for 
>> @trusted and
>> verify).
>
> This is what I mean by auditing @trusted when it interacts with 
> @safe code.

Ok, I was not sure what exactly you meant (because interaction is 
a broad concept), so I went explicit as I did not want to assume.

>
> Using your example, if we confirm that no matter how you call 
> foo, the @trusted block cannot break memory safety, then foo 
> becomes a verified @safe function. Then any @safe function that 
> calls foo can be considered @safe without auditing.

Yes (assuming any such @safe function does not call another 
@trusted function that wasn't verified).

>
> This is actually a necessity, because templates can infer 
> safety, so you may not even know the call needs to be audited. 
> The most dangerous thing I think is to have @trusted blocks 
> which use templated types.
>
> A real example recently was a PR that added @safe to a 
> function, and made the following call:
>
> () @trusted { return CreateDirectoryW(pathname.tempCStringW(), 
> null); }
>
> Where pathname was a range. The trusted block is to allow the 
> call to CreateDirectoryW, but inadvertently, you are trusting 
> all the range functions inside pathname, whatever that type is!
>
> The correct solution is:
>
> auto cstr = pathname.tempCStringW();
> () @trusted { return CreateDirectoryW(cstr, null); }

Interesting case. I have not encountered something like it 
before, since all code that I mark @trusted uses OS provided 
structs, primitive types, or arrays (nothing fancy like ranges).

>
> So yes, if the third party has @trusted code, you need to audit 
> it. But once you have audited the *block* of trusted code, and 
> how it interacts within its function, you can consider the 
> calling @safe functions actually safe without auditing.

Indeed (again, though, only if that calling @safe function 
doesn't call some other @trusted you forgot to audit).

>
>>> If we get into "@safe really means @trusted" territory, we 
>>> have lost.
>>
>> For code that you write yourself, @safe means @safe, of 
>> course. For code
>> other people write and you want to call, it being marked @safe 
>> does
>> really mean @trusted as long as you yourself have not looked 
>> inside it
>> and verified there either is no hidden @trusted, or verified 
>> *yourself*
>> that the hidden @trusted is memory safe.
>> I consider any other behaviour to be negligent to the degree 
>> of "you
>> don't actually care about memory safety at all".
>
> I think there will be a good market for separating libraries 
> between @trusted-containing libraries, and only 
> @safe-containing libraries.

I totally agree. Unfortunately, most of what I use D for is 
replacing C in parts where I have to directly interface with C, 
specifically glibc wrapped syscalls.
I considered calling the syscalls directly and completly bypass C 
at all, but since syscalls in Linux are often in reality 
available in userspace without context switch (vdso), the code is 
not trivial and I decided to just use interface with the C 
wrappers.

> This will make the auditing more focused, and more shareable.

Sure.

> I don't expect people to use Phobos and audit all the @trusted 
> blocks personally.

As long as they don't actually call them, that's reasonable. But 
if your application ends up calling @trusted code and you did not 
audit that @trusted yourself, you have violated the @trusted 
requirement:
You cannot promise to the compiler that the code is memory safe 
since you have no knowledge of what it actually does.

> If "D is  memory safe" means "D is memory safe ONLY if you 
> verify all of the standard library personally", we still have 
> lost.

It is more like "D is memory safe" meaning "D is memory safe ONLY 
if you verify all of the @trusted code your application end up 
compiling in / linking against".
There is no way around that I can see without getting rid of 
@trusted, which is impossible for a systems PL.