Memory/Allocation attributes for variables

[Elmar]

On Sunday, 30 May 2021 at 05:13:45 UTC, Ola Fosheim Grostad wrote:
> On Sunday, 30 May 2021 at 02:18:38 UTC, Elmar wrote:
>> Looking at what attributes D provides I definitly see that 
>> memory safety related concerns are addressed with existing 
>> attributes. But I personally find them rather unintuitive to 
>> use or difficult to learn for beginners and not flexible 
>> enough (like two different versions of `return ref scope`). As 
>> currently defined, those attributes don't annotate 
>> destinations of data flow but sources (like function 
>> arguments) of data flow.
>
> I agree that D has jumped down the rabbithole in terms of 
> usability and function signatures are becoming weirder. The 
> reusage of the term "return" is particularly bad.
>
> To a large extent this is the aftermath that comes from 
> changing course when it went from D1 to D2. Where simplicity 
> was sacrificed and it was opened for more and more complexity. 
> Once a language becomes complex, it seems difficult to prevent 
> people from adding just-one-more-feature that adds to the 
> complexity. Also, since experienced users influence the process 
> most... There are nobody to stop it.
>
> The main issue is however not specifying where it allocates, 
> but keeping track of it when pointers are put into complex 
> datastructures.

Thank you for your reply. Also sorry for the wordyness, I'm just 
awkwardly detailed sometimes.

In your case it's not what I was thinking. I would count myself 
to the sophisticated programmers (but not the productive ones, 
unfortunately). I can cope with all those reused keywords even 
though I think at this place their design is unintuitive to use. 
Intuitive would be annotation of the return type because the 
aliasing is a property of the return type, not the argument. At 
least I feel like I understood the sparsely explained intension 
behind the current scope-related attributes but my main point is, 
I find they can be improved with more expressiveness. It would 
give programmers a hint of what kind of allocated argument is 
acceptable for a parameter. And no, this is not trivial. It's the 
reason for my decision to start this thread:

*Functions in phobos accept range iterators of fixed-sized arrays 
as range argument but even if it fails miserably, it compiles 
happily and accesses illegal memory without any warning, creating 
fully non-deterministic results with different compilers. I 
noticed this when I tried to use "map" with fixed-size arrays. It 
simply misses any tool to check and signal that fixed-size arrays 
are illegal as range argument for "map". And sometimes mapping 
onto fixed-size arrays even works.*

Without better memory safety tools, I'd discourage more memory 
efficient programming techniques in D although I'd really like to 
see D for embedded and resource constrained systems to replace C.

---

I wonder how programming languages don't see the obvious, to 
consider memory safety as a part of type safety 
(address/allocation properties to be type properties) and that 
memory unsafe code only means an incomplete type system. I also 
don't know whether conventional "type safety" in programming 
languages suffices to eliminate the possibility of deadly 
programming bugs (aliased reference variables e.g.). But of 
course, security and safety is complex and there is no way around 
complexity to make safe code flexible.


The important part is the first one (without generalization to 
allocation and control structures which I only mentioned as an 
interesting thought) because I think it's an easy but effective 
addition. D already has features in that direction which is good, 
the awareness exists, but it's still weak at some points. My post 
should be seen as a collection of ideas and a request for comment 
(because maybe my ideas are totally bad or don't fit D) rather 
than a request to implement all this. The main point is to 
consider references/pointers as values with critical type safety 
which means a way to specify stricter constraints. Memory safety 
is violated by storing a pointer value in a reference which is 
out of the intended/reasonable value domain of the pointer (not 
matching its lifetime).

If someone already thought the same like me, there could be a 
safepointer-like user library which supports additional 
attributes (which represent restricted pointer domains) by 
implementing a custom pointer/reference type. (It's not a smart 
pointer because smart pointers try to fix the problem at the 
other end and require dynamic allocation which is not that nice.) 
Due to D's nature, it would support safe pointers and safe 
references (reference variables) and provides static and dynamic 
type checks with overloaded operators and memory attributes. 
Attributes couldn't be inferred automatically I guess but  
annotation of variables could entirely allow static memory safety 
checks (which doesn't need to explicitly test whether a pointer 
value is contained in a set of allowed values) and maybe prevents 
bugs or unwanted side effects.

---

One important aspect which I forgot: aliasing of variables. I 
know, D allows aliased references as arguments by default. Many 
memory safety problems derive from aliased variables which were 
not assumed to be aliased. Aliased variables complicate formal 
verification of code and confuse people. I would add 
`@alias(symbol)` to my collection which indicates that a 
reference explicitly aliases (overlap) another reference in 
memory or a `@noalias(symbol)`.

---

If someone thinks, I heavily missed something, please let me know.