__restrict

[Manu]

I'll start by disclaiming that I'm from a gaming and high performance
embedded systems background. While I am entirely enthusiastic about what D
brings to the table, D does not exclude most of the code possible in C, and
for good reason. I don't intend to stop using loose pointers in the near
future (when necessary), and in addition to that, language features like
__restrict are also necessary to me.


On 19 October 2011 17:08, Robert Jacques <sandford at jhu.edu> wrote:

> Well, __restrict was mainly added to C (IIRC), to allow loop vectorization.
> In D, we have explicit array operations, which carry a lot of the same
> caveats as __restrict, except are checkable.
>

I don't know if that's true or not, but has never occurred to me that
__restrict might be useful in that way.
__restrict asserts to the compiler that the memory it points to is not
aliased by any other pointer, this means the compiler can schedule
loads/stores outside of loops or other constructs, which is not only common
sense, but is required in performance critical code.
In C, any data access through any pointer must be committed on the spot,
since it's possible that any call to a function, or any access through
another pointer may affect, or be affected by the operation.
As a result, code that repeatedly uses a pointer, maybe in a loop, or simply
interleaved by other function calls/pointer accesses will redundantly load
and store in between every other operation.

I can't imagine that the compiler assumptions about pointer aliasing could
possibly be any different in D? The compiler certainly must consider the
possibility that any memory, managed/garbage collected or not, could be
aliased by some other object, and therefore commit/reload any changes to
that object in between calls to distant functions, or indirect references
through other pointers?
Does D have a solution to this problem that was not possible in C?


That's the point, the compiler can't verify or enforce the assumptions of
> __restrict. Think of array indexing in C. The compiler has no way of
> verifying or enforcing that ptr[10_000] is a valid memory location. And this
> has lead to undefined behavior and a large number of security exploits. In
> D, the arrays always carry around their lengths, so indexes can be checked.
> No more undefined behavior and less exploits. Similarly, C's const provides
> no actual guarantees. But compilers used it to optimize code anyways, which
> has lead many a programmer to long hours debugging.


I'm not sure why the compiler needs to verify the assumptions of __restrict,
the assumption is that it is not aliased by anything else, which is contrary
to its default assumption (that it MAY be aliased), and therefore must be
commanded explicitly.
I can't see the relation to the array example, and I don't think the const
comparison is fair.
The only insecurity/exploit that can come to being through the use of
__restrict is if you are lying to the compiler, and the pointer IS aliased
somewhere... Surely you wouldn't specify a pointer as restrict if you don't
know for sure.
In comparison to const, which is a qualifier that gives information to
external code, __restrict's effects are completely local and
don't propagate outwardly like const.... if that makes sense :/ .. so I
don't think that's a fair comparison.


Is it that it would be preferred if
>> __restrict-ability could be implied by carefully crafted language rules?
>>
>
> Yes, and it is for array operations.


And I appreciate that general language goal, but I don't think it's
applicable, or *possible* in the case of __restrict.
There is very likely to be some conditional logic based on foreign data
where the compiler can no longer form a conclusion and defensively presume
it may be aliased.


It's very easy to contribute. The source code for DMD, LDC, GDC is all
> available and patches are regularly submitted and accepted. Check out D's
> github repositories and the mailing lists for more.
>

Cool. Sadly I have virtually no spare time at the moment, but I'll
definitely be giving it a look.
At very least I hope to find the time to contribute to the standard library
for some embedded platforms/games consoles.


On 19 October 2011 12:36, bearophile <bearophileHUGS at lycos.com> wrote:
>>
>>  Manu:
>>>
>>> > I sent an email about this once before... but there was no real
>>> > response/discussion on the topic.
>>>
>>> It was discussed a bit in past, and restrict was not appreciated a lot,
>>> maybe also because in D where possible we prefer things that the compiler
>>> is
>>> able to verify/enforce.
>>> And I think D/DMD is not yet in a development stage where it cares for
>>> max
>>> performance details. I think there are plenty of more important things to
>>> work on before that. The recently almost-fixed "inout" was more urgent
>>> than
>>> "__restrict".
>>>
>>> Bye,
>>> bearophile
>>>
>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.puremagic.com/pipermail/digitalmars-d/attachments/20111019/b56918eb/attachment-0001.html>