Report: Analysis of survey: Memory safety in D, what is your view?

Fri Nov 8 16:51:07 UTC 2024

The highlights of this survey, was presented at the DLF meeting 
on the 9th of November 2024.

No actions were immediately taken from the information given here.
However it is of interest for the future.

## Interest in Memory Safety

Of the respondants, nearly half responded that they use memory 
safety for everything they do (49.2% 32 people), the second 
largest group use memory safety for somethings but not everything 
(30.8% 20 people), for some it isn't worth their time and can be 
combined with the last group of ``@system`` only (20% 13 people) 
with more than half being convincable.

For those that were ``@system`` only, they were not asked the 
majority of questions as they would not have an answer to the 
questions.

## Turning on Memory Safety

It was asked if turning on additional memory safety using a new 
attribute would be a desired addition.

Of the 54 responses, 63% (34 people) said that if we are going to 
do it, it should be in ``@safe``, no new attribute.

The second largest category at 11.1% (6 people) did not prefer 
it, but would accept it.

The third largest category at 7.4% (4 people) prefer it for 
invasive changes like borrow checker.

About 5 of the single submissions were related to no new 
attributes.
The rest have single submissions and won't be mentioned here.

## Feature Usage

The following features have been tried out of 58 responses:

- ``@safe`` 57 people
- ``@live`` 9 people
- ``DIP25`` 27 people
- ``DIP1000`` 40 people
- No shared access 20 people
- ``-preview=in`` 21 people
- System variables 9 people

Of these 49 people, the following features people are continuing 
to use:

- ``@safe`` 44 people
- ``@live`` 3 people
- ``DIP25`` 20 people
- ``DIP1000`` 24 people
- No shared access 17 people
- ``-preview=in`` 20 people
- System variables 11 people

Of these 44 people will recommend the features:

- ``@safe`` 38 people
- ``@live`` 4 people
- ``DIP25`` 13 people
- ``DIP1000`` 18 people
- No shared access 10 people
- ``-preview=in`` 17 people
- System variables 10 people

Some of these numbers went up from the prior question, which by 
itself is an interesting result.

Of note is that DIP1000 had a massive difference between people 
trying it and recommending it.

Another observation is that ``@live`` is attempting to solve a 
problem that those who care about memory safety do not have.
This does not imply that lack of desire or need for a borrow 
checker of a different design.

### What Prevents Adoption?

The following comments have been made regarding why people have 
not adopted or tried out language features:

- Do not like them
- Not turned on by default
- Bad error messages
- Reuse of attributes
- Lack of examples and documentation
- Too complicated
- Does not add anything (``@live``), does not allow ``@trusted`` 
code to become ``@safe``.
- Not worth typing of attributes
- Usage of Phobos doesn't compile
- Limited expressiveness

Each of these may have multiple people stating it in some form.

The following comments have been made regarding why people have 
not continued to use memory safety language features:

- Too much effort to add, needs to be the default
- Can be difficult to interface to non-D code
- Uncertainty of the future
- Introduces fighting the compiler, without reducing bugs
- DIP1000 has cases where it does not work
- Too restrictive and poorly implemented
- ImportC cannot be marked as ``@trusted``
- Limited expressiveness
- DIP1000 indirection doesn't compose well
- Library code does not work with them (including Phobos)

### Confidence in Features

Given current features 57 people responded between 1 and 5 if 
they would recommend D.

1. 5.4%
2. 10.5%
3. 29.8%
4. 40.4%
5. 14%

Given additional features 55 people responded:

1. 5.5%
2. 1.8%
3. 20%
4. 30.9%
5. 41.8%

## Improvements to Escape Analysis

The following improvements to escape analysis (DIP1000, 
``@live``) are desired out of 46 people:

- Inferation 45.7% 21 people
- What is needed to unblock reference counting 39.1% 18 people
- Differentitate strength of relationship for escaping 26.1% 12 
people
- Replace attributes with escape set 17.4% 8 people
- Replace combination of attributes 17.4% 8 people

The rest were write ins all at 1 person.
For which multiple said removal.

## Additional protection

Of the 45 responses people wanted the following features:

- Type state analysis 71.1% 32 people
- Ownership transfer 60% 27 people
- Guaranteed protection of variables with a lock 31.1% 14 people

The rest were write ins all at 1 person.

## What Can We Do Better?

Unrelated to memory safety the question was asked, what can we do 
better?

Some of the responses (can have multiple responses per item):

- Allow turning on preview switches within a D file rather than 
as a CLI argument to the compiler.
- There may have been better things that we could have worked on 
in the last 10 years.
- Have D leadership use tooling and IDE's that they don't 
normally use so as to see what the current state of the ecosystem 
is.
- Break code, if it means fixing things.
- Infer, not attributes for lifetimes.
- Memory safety by default.
- Use the GC as primary safety mechanism.
- Add sum types.
- Make containers and RC work.
- Less annotations that need to be written.
- ``-betterC`` needs better documentation.
- There are too many custom druntimes (7) was countered.
- Uses cases such as commercial games have too much time taken up 
by ``@safe``.
- The flexibility of D is an asset. Can pick memory safety level.
- Type safety, memory safety and formal verification would be a 
desired trait of D for security sensitive applications.
- Add the inverses of attributes (``@impure``)
- Better defaults i.e. ``@safe``
- Don't chase Rust.

## Population of Respondants
The number of people who responded to this question was 61, at 
93% of total people who answered the survey (65).

- Hobbyist: 75.4%
- Employee: 27.9%
- Student: 4.9%
- Owner or management of a business: 19.7%

According to the global statistics on the users of [Stack 
Overflow for 
2024](https://survey.stackoverflow.co/2024/work#employment-employment), the following results were obtained with 65,437 responses:

Unlike the D survey, it is a multiple choice question that 
telleys to a total of 118.6%.
It was answered by a very broad representation of people in the 
programming population globally.

- Hobbyist: 8.8%
- Employee: 75.3%
- Student: 13.2%
- Self employed: 16.4%

At 65 respondants, the D survey is quite small, in comparison to 
the Stack Overflow survey of 65,437 responses.
This makes the resulting statistics not useful for drawing a 
direct conclusion from.

The one note of interest is that employee and hobbyist appear to 
be swapped with D's employee number being larger than Stack 
Overflows hobbyist number by a wide margin.
This could be indicative of a very different group of people 
attracted to D, compared to the wider field.

## Recommendations

Any new memory safety features should focus upon what can be 
turned on in ``@safe`` code and be tuned in such a way that what 
affects everyone has a high degree of certainty of being wrong.
Tuning where possible to turn a feature on as a consequence could 
be preferred when the breakage is invasive, but not required for 
normal operation.
This is exemplified by the significant lack of usage of 
``@live``, this is an attribute that adds behavior on top of 
``@safe``.

While a good portion of respondants are ok with the memory safety 
features of D in terms of quantity, it is clear that there is a 
strong desire to add additional memory safety protection.

New features that people strongly want are type state analysis 
and ownership transfer.
With the next being guaranteed protection of a variable with a 
lock at less than half the other two.

A common theme throughout the write ins is poor error messages.
More descriptive error messages with the explanation explaining 
why the compiler came to a decision would be of significant 
benefit for complex language features that people may not be 
familiar with such as escape analysis.
Some include language features unrelated to memory safety 
including a desire for sum types.

The population of D users appears to have a very different 
constituent members and therefore the trends, desires and needs 
will not reflect other language communities.
Any language or ecosystem changes should reflect the D population 
base, and therefore need not prioritise what other language 
communities are doing, but must prioritise the existing D 
community population when apply the D Language Foundations IVY 
statement.

To get a more accurate population of D, future surveys should be 
made available on the D home page.