Helping with __mutable (which will be renamed to __metadata)

[Doc Andrew]

On Monday, 15 April 2019 at 11:09:23 UTC, Timon Gehr wrote:
>> 
>> A few questions/thoughts (or at least food for thought for 
>> others more informed than me):
>> 
>> 1. Maybe it's just me, but why the leading underscores? I'm 
>> wondering if this is because it's a scary/ugly-on-purpose 
>> feature like __gshared that is intended only sparingly?
>> ...
>
> Yes. Also, it means we don't reserve a previously unreserved 
> identifier.
>
>> 1b. A lot of people have been clamoring for immutable by 
>> default, might keeping the DIP's original annotation "mutable" 
>> (with no underscores) make this a step in that direction? 
>> (with all the baggage and argument that carries with it, which 
>> is out of scope for this discussion).
>> ...
>
> It's not the same.

Oh no, I'm just suggesting that, under the original proposed DIP, 
the __mutable keyword might as well be made "mutable", which may 
be useful later on for other purposes. The metadata concept could 
be a lot more powerful, though, and I think it's basically 
orthogonal to the mutability concern:

>> 
>> 2. If the __metadata qualifier can only be applied to private 
>> fields, and carries special semantics w.r.t. escaping the type 
>> system, is there any benefit to treating it more like a 
>> visibility attribute?
>> 
>> struct something
>> {
>>      metadata        // maybe type qualifiers disallowed in 
>> here
>>      {
>>          int a;
>>      }
>>      private
>>      {
>>          int b;
>>      }
>>      public
>>      {
>>          int c;
>>      }
>> }
>> ...
>
> Is your suggestion just to make __metadata imply private, or is 
> there anything else you are trying to say?

Not just private, but it would be a way to annotate data that 
lives "outside" of the POD of the class. For instance, 
serializing an object with variables in the metadata{} block 
wouldn't (or couldn't?) include them.

Another idea I had this morning is that the metadata{} block 
might be a nice place to put class/function attributes to avoid 
cluttering the signatures:

class Foo
{
     metadata
     {
         synchronized;     // can move class attributes in here
         scope;

         @(someUDA);       // UDA's live here too

         int secretSauce;  // Can always be accessed or modified 
by Foo, regardless of
                           // mutability
     }
}

The concept fits nicely with contracts and functions, too. Just 
as metadata{} blocks in a immutable object allow modification of 
those member variables, maybe member variables in a function 
metadata{} block can be modified and persisted across calls for 
pure functions, too. It would act just like a static variable, 
available at both compile-time and run-time, maybe useful for 
things like debugging, profiling or benchmarking. Maybe code in 
the in & out contract blocks can access it, but we'd make it 
illegal for the function body to do it. I'd need to think through 
the side-effects a little more.

int Bar(int a, int b)
metadata
{
     pure:                       // Not 100% sure what syntax 
would make sense
     @nogc:                      // maybe just force @ for 
everything here
     @safe:

     long startTicks;            // Stuff for benchmarking
     long endTicks;              // Not pertinent to the function 
itself
     static int numTimesCalled;  // This is a pure function...
}
in
{
     numTimesCalled++;           // ...but it's OK, we're just 
modifying it's metadata
     ...
}
do
{
     if(numTimesCalled > 4)      // Illegal to do this here, 
breaks purity
     ...
}

>
>> And... maybe metadata shouldn't be stored with the object at 
>> all? This way, it doesn't affect the layout of the object in 
>> memory, and doesn't require the need to short-circuit the type 
>> system if a later use of the class is marked Immutable. The 
>> object's metadata lives, not sorta-outside the type system, 
>> but outside the object itself? There are some challenges I can 
>> think of with this approach that I can expand on, but also 
>> some benefits.
>> 
>> -Doc
>
> Storing the data within the object is the point. As the 
> original post states, the motivation for the DIP is to allow 
> memory allocation schemes such as reference counting, as well 
> as lazy initialization to be implemented for 
> `immutable`-qualified data structures. The mutability is 
> supposed to be an implementation detail -- from the outside, 
> the data will still appear to be `immutable`. (The current 
> version of the DIP completely misses this point though, leaking 
> __metadata into the type system.) Compiler optimizations can 
> change the semantics of code that operates on __metadata. 
> (e.g., if it is able to elide a read to a lazily-initialized 
> field, that field will not be initialized, if it elides a 
> reference copy, the reference count does not need to be 
> updated, etc.) Therefore, modifications of __metadata need to 
> be consistent with rewrites that are valid for strongly pure 
> functions.

Sure, I'm just suggesting that keeping that reference count or 
other metadata separately is a possible workaround - as long as 
there's an easy way to get the metadata for a given object 
(whether in memory alongside the rest of the object's data or 
not).

The idea of storing metadata member variables separately is an 
implementation detail. To the class, it appears that the metadata 
members are no different than regular variables, but casting to 
void* or serializing it leaves out the metadata, so it's a safe 
place to keep things like reference counts. In this scheme, 
metadata can be added to classes or structs without changing 
their memory layout, and metadata vars have some other special 
powers as far as being treated differently by the type system, 
strictly because they are stored "outside" the object memory, not 
because of a special-case in the type system.

-Doc