Newbie initial comments on D language - scope

[Edward Diener]

Walter Bright wrote:
> Edward Diener wrote:
>> Your analogy to C++'s 'const' is a bad one.
>>
>> The C++ 'const' refers to a quality of the object while the D 'scope' 
>> refers to a quality of the type. There is no equivalent of 'const' in 
>> C++ which refers to the type.
> 
> 'const' in C++ is very much a characteristic of the type of the object. 
> It pervades the semantics of the type. It's easy to envision a scheme of 
> const where the const-ness is controlled by a bit in the runtime 
> instantiation of the object, and any mutating operations would first 
> check that bit.

In your example justifying treating 'scope' as C++ treats 'const' the 
'const' is attached to the object upon instantiation of it. My point is 
that 'scope' is attached to the type by the class designer. To me these 
are conceptually two different things. That is why I said that your 
example is a poor analogy. I should have made that clearer by my argument.

We can argue the concept similarity/dissimilarity between 'const' and 
'scope' all day without getting much of anywhere. I was simply arguing 
against your assertion, using 'const' as an example, of not allowing a 
'scope' object to be assigned to an object that is not 'scope'. Clearly 
in the polymorphic world of D, where a base class may not be 'scope' 
while a derived class may be 'scope', such a treatment can not be right, 
since the basis of polymorphism is to assign to a base class object a 
derived class reference.

> 
> C++ avoids the overhead of that by having a static typing system, and 
> such 'bits' become part of the compile-time information (i.e. the type) 
> rather than the run-time information.
> 
>> Once we say that a type is 'scope' in D we should no longer have to 
>> say that an object of that type is 'scope'. An object of that type 
>> should be 'scope' automatically and the user of that object should not 
>> care or even need to know. In C++ the user of an object specifically 
>> says it is 'const' to set the quality of the object to something ( one 
>> can not change the object ). Your analogy is mixing apples and 
>> oranges. These are different things.
> 
> I don't believe they are different at all. Consider also languages that 
> have no static types - the types are determined at runtime (Javascript 
> is an example). What a language chooses to specify about an object at 
> compile-time vs run-time is a spectrum with various tradeoffs, not an 
> apples-oranges with a sharp dividing line.
> 
> Certainly, there is plenty of debate about static typing vs dynamic 
> typing. D is a statically typed language primarily for performance 
> reasons - a dynamically typed language can run 100x slower.

I am fully cognizant of a dynamically typed language since I program in 
Python also. I agree there is no fixed dividing line. But the difference 
between static typing and dynamic typing is well defined in a statically 
typed language like D. My argument was that for 'scope' to be really 
effective it needs to consider the dynamic type at run-time and not just 
the static type as it exist at compile time.

> 
> 
>> What I am saying is that an object whose type is 'scope' is treated 
>> magically by the compiler in that the compiler is now doing reference 
>> counting on it and calling its destructor when the last reference goes 
>> out of scope. Furthermore as that object gets reference assigned the 
>> reference count is manipulated and whatever object is specified in 
>> that reference assignment, as long as it is allowable by the compiler 
>> by the rules of D, takes part in the 'scope' magic. In a polymorphic 
>> language this means that you should associate 'scope' with the dynamic 
>> type of the object, not its static type, and how you decide to do that 
>> is up to you. Think of it as wrapping a boost::shared_ptr around the 
>> object and for every object to which you legally assign/copy it a 
>> boost::shared_ptr gets wrapped around that object.
> 
> Pulling on that string leads us to every object having scope semantics, 
> because that machinery will have to exist and be checked at runtime for 
> every object.

When the scope changes in D you need to make sure that any 'scope' 
object is treated appropriately.

But I do not see why you think that every object must therefore have 
scope semantics. Inserting compile time code when a scope changes to 
treat 'scope' objects in a special way does not mean to me that every 
object must have scope semantics. Perhaps you mean that the execution 
slows down a GC system too much to do that for every object.

If that is the case then I agree that RAII can not be done in a GC 
language in the terms in which I have defined it, although it probably 
can be done in lesser terms, as 'scope' currently exists in D.

> 
>> I agree this adds some overhead, but so what.
> 
> And there lies the crux of our disagreement. My experience with memory 
> allocation is that ref counting is appropriate for scarce resources, and 
> gc is appropriate for abundant resources (i.e. memory). We both agree 
> that gc is a poor choice for scarce resources, and I'm going to argue 
> that rc is a poor choice for abundant resources.

You have already won that argument as I fully agree to what you say 
above. But I have no idea why you say that it is the crux of our 
disagreement. Care to elaborate ?

> 
> 
>> Using boost::shared_ptr also imposes overhead and whole generation of 
>> programmers have somehow survived the extra x bytes per object in an 
>> age where physical memmory is in the gigabytes and virtual memory in 
>> 64 bit systems in the quadrabytes.
> 
> There is a large push to add gc to C++. (rc has disadvantages besides 
> using more memory - the overhead to allocate two objects instead of one, 
> and the overhead of doing the inc/dec/test. A further disadvantage is 
> you cannot do array slicing with rc without adding substantial more 
> overhead - memory and runtime.)
> 
> 
>> My added suggestion is that when applying the 'scope' keyword to the 
>> object, and not the type, this essentially means that the compiler now 
>> treats that object as 'scope' even though the type is not 'scope'. I 
>> will call this object 'scope' injection. My suggestion for this is 
>> based solely on the practical consequences:
>>
>> 1) Allow the instantiator of a type to have 'scope' control over the 
>> object even when the designer of the type does not specify it as a 
>> 'scope' type. The user may know something about using the type at 
>> run-time that the class designer can not know, makes optional, or even 
>> disregards.
> 
> I agree with you that the scopeness of an object is best determined by 
> the user, not the class designer. But this doesn't preclude making scope 
> part of the type any more than the user adding 'const' precludes it.

No, I do not think that the scopeness of an object is best determined by 
the user and not the class designer. In fact I feel very strongly the 
opposite. The class designer knows whether his class has RAII or not and 
in the cast majority of cases the end user should not know or care.

My argument for scope injection is based purely on the practical 
considerations that there are types which can not possibly know if it is 
to be used with RAII or not. Template classes/structs which are 
containers are the most obvious as well as built-in arrays. That is why 
besides the ability for the class designer to specify 'scope' the end 
user should be able to do it at object creation time also.

> 
> 
>> 2) Following from the above, the most obvious practical cases occur 
>> when the type is created from a template class/struct, when the type 
>> is some built-in language container which can hold polymorphic objects 
>> of some base class type, and when the type embeds an object of 'scope' 
>> class type which may only be used in a corner case so that the 
>> designer of the type leaves scoping up to the user.
>>
>> In other words the flexibility of control would be wonderful and, I 
>> believe, often necessary. Having both the class designer be able to 
>> 'scope' the type and the end user be able to 'scope' an object of any 
>> type ( which has a destructor ) is the ultimate ideal. If you wanted 
>> to go even further you could allow the end-user to 'unscope' an object 
>> of a 'scope' type when instantiating the object, even though the 
>> benefits of doing this seem to be practically negligible.
>>
>> I view your choices as, from most desirable to least desirable:
>>
>> 1) Keep tracks of the objects themselves at run-time to see if they 
>> are 'scope' or not. This allows object 'scope' injection and refernce 
>> assignment to objects whose static type is not 'scope' to make them 
>> 'scope'.
>>
>> 2) Keep track of the dynamic type of the objects themselves in order 
>> to see whether the dynamic type of the object is 'scope'. This does 
>> not allow object 'scope' injection, but does allow reference 
>> assignment to objects whose static type is not 'scope' since you are 
>> only considering the dynamic type of the object and not the static 
>> type to determine if the object should be 'scope'.
>>
>> 3) Keep track of only the static type of the object. This does not 
>> allow object 'scope' injection nor even reference assignment to 
>> objects whose static type is not 'scope'.
> 
> I believe that adding scope to the type allows for scope 'injection' as 
> you defined it. But you're right in that (3) does not allow an object to 
> be dynamically retyped as scope, though it could be 'wrapped' at runtime 
> with a proxy struct that is itself statically scoped.

Whether one does 'scope' injection using the 'scope' keyword on the 
object when it is declared or by using the equivalent of a 
boost:shared_ptr construct in D is of little practical matter to me.
This is purely syntax so that if D could silently translate 'scope' to 
such a boost::shared_ptr construct that would be better IMO because it 
would unite such a treatment under the same concept with the 'scope' 
keyword as it applies to a class.

The crux of my argument against 3) above is simply that the end user 
will not and should not be expected to know that an object is of a 
'scope' type.

// In a module

class C { ... }
scope class D : C { ... }

// In the end user's code

C d = new D(...);

Under 3) the d object is not 'scope', because its static type is not 
'scope' even though its dynamic type is 'scope'. This can not be right 
IMO. Requiring the user to have knowledge that D is a 'scope' negates a 
great deal of the transparency of having RAII in a GC language.

I can understand your feeling that the above should be:

class C { ... }
scope class D : C { ... }

// In the end user's code

scope C d = new D(...); // End user is required here to specify scope

This may make things much easier for the compiler, but it requires the 
end user knowledge of 'scope', which has been specified at the class 
level, to be applied at the syntax level. Intuitively I feel the 
compiler can figure this out, and that 'scope' should largely be totally 
transparent to the end user above at the syntax level.

I do agree that the end user should "know" that a class is 'scope' (RAII 
) by reading the documentation of that class. This is useful for scope 
injection for container objects and for the end user designing his own 
class as 'scope' when an object of a 'scope' class is a data member.