Transitive const sucks

[Walter Bright]

Janice Caron wrote:
> I've written a lot of multithreaded programming. A lot.

Ok. I've written some, enough to know it is very, very hard to get 
right. I also attended a conference of the top C++ people on what to do 
about multithreaded programming support in the C++ language, and came 
away from it with the realization that only a handful of those people 
even understood the problem (no, I don't count myself as one of them). 
These are the experts.


> (By the way - could you also allow the spelling "synchronised" for the
> benefit of speakers of British English?) :-)

Maybe. Colour, never. <g>


> synchronised is a blunt tool though. In my C++ code, I arrange it so
> that multiple threads can simultaneously obtain read-access, or
> exactly one thread can obtain write-access. Correct me if I'm wrong,
> but the built-in synchronized feature isn't that smart?

It's nothing more than a sugary mutex. It certainly isn't the 
overarching solution to multithreaded programming I had earlier thought 
it was.


> So anyway, as I mentioned further up this thread, I had this loopup
> class member function that got stuff from a file and cached it. I
> assure you it was thread-safe. It even allowed multiple readers (if
> the key/value pair was already cached), but if a lookup was not
> cached, one thread and one thread only got to open the file, read a
> chunk of data, and stick it in the file. This was a class that knew
> what it was doing.

If you can assure me it is thread safe, and it is, then you are a very 
rare programmer. Even if it is thread safe, it can still have deadlocks 
when used in conjunction with other thread safe packages. What I'm 
getting at is multithreaded programming is very, very hard. It needs to 
be simpler, a lot simpler.


> You're probably going to tell me that I can still do all that in D,
> providing I don't declare it const. The problem is, if I don't declare
> it const, /and/ all consts in D are transitive, then I can't store a
> reference to that object in any const structure. Basically I'd end up
> declaring just about nothing const, and I'd see no benefit.

You're right, I'm going to say that if it can change state, then it 
should not be declared const. I believe there is a greater benefit to 
things being reliably, checkably, constant, than there is for declaring 
things const that aren't constant.

I think what is pretty clear at this point is that const in D will be 
used differently than in C++.

>> Being in a module doesn't prevent another thread from using the module's
>> interface and thereby changing the state.
> Is that why you object?

That's one reason.


> I would have thought it the programmer's problem to ensure thread
> safety, not the compiler's. If I have a variable that's going to be
> accessed by multiple threads, then I'm going to make sure it's
> properly synchronised, and I if screw up, then it's my bug.

There are a couple problems with this:

1) Programmers aren't that good. Sure, a few are, but most of us aren't, 
and we write code we imagine is thread safe but really isn't. See Scott 
Meyer's article on double checked locking, for example.

2) Even if a programmer is that good, programmers still have bad days. 
It's why pilots have checklists for everything, and always use them, no 
matter how well they've memorized them.

3) Programmers being good doesn't help the code auditor, who has to 
always assume the worst.

4) Automation to detect flaws is better than relying on human failings.


>> Yes, but that requires programmer discipline, which is unreliable and
>> scales poorly.
> 
> I think it could scale very well with the right language support. But
> that's another subject.

I think this is the subject <g>.


>> Very few programmers are able to successfully write such
>> code. With FP programming, most programmers will be able to.
> 
> That's obviously true. I'm not sure why it's relevant though. You said
> you had a "pure" keyword in mind for that. But if I don't use the pure
> keyword, then thread safety should be down to me, right?

The const will still be needed.


>> It has come up before. The answer is to not declare logically const
>> things as being const, because they aren't const. Logical constness
>> belongs as a comment because it is not compiler checkable.
> But you might want to declare logically const things as being const
> for efficiency reasons.

There aren't any optimizations that logical const enables, for the 
simple reason that a state change will invalidate the optimization, and 
you already agreed that logically const objects change state.


> And to argue that "they aren't const" is a
> matter of definition. "const" in C++ is /defined/ to mean logical
> const, so const they are! You use the word to mean "physically const".
> We don't all agree on the definition.

C++ invented the term "logical const" because people clearly were 
surprised by const references not being constant. But that's not really 
relevant here, what is relevant is what is the right definition of const 
for D. I think D should have an intuitive, useful, and checkable notion 
of const.

C++'s is not intuitive, marginally useful, and not checkable.


> Suppose there exists a class String with member function uppercase(),
> declared const (or invariant, as you would have it in D). So far, so
> good.
> 
> But then, along comes a better (or at least, different) string class
> with the same interface, and it happens to be faster, and I want to
> use the new FastString class in place of the old String class. All I
> have to do is search and replace, right? Or just make an alias. But
> there's a problem. Turns out, the reason it's faster is because it
> caches some results internally so it doesn't have to keep recomputing
> them.
> 
> From the point of view of "implementation should be independent of
> interface" this internal detail is something I shouldn't need to know
> or care about.
> 
> But now suddenly, I do need to know about it. And care. Because now
> that uppercase() function I mentioned earlier is no longer physically
> const. That means, if I use it as a drop-in replacement, my code now
> won't compile.

It's a different interface if one is const and the other mutable. It 
really IS different (repeating myself!). A user of it would care, 
because now he'd know that it may no longer be thread-safe.


>> I'll reiterate that const-that-isn't-checkably-constant has no value
>> beyond being a comment, so it might as well just be a comment.
> 
> Oh contraire. It /is/ meaningful. It is a cast iron guarantee that no
> non mutable (which I argue should imply non-private) variables will be
> modified.

No, it isn't, because:

1) The supplier of the type may change its internals to mutable. You 
recompile, now it crashes in your multithreaded environment.
2) D has opaque types where the contents are not knowable.
3) It becomes impossible to write thread safe generic code.

> Without the const declaration, you do not have that
> guarantee. It's a guarantee *which the compiler can use* to help the
> programmer catch errors! For example, this C++
> 
> class C
> {
>     mutable int x;
>     int y;
> 
>     void f() const;
>     {
>         x = 1;
>         y = 2; /* The compiler catches this error */
>      }
> }
> 
> See - this is useful to me. If I had not declared f const (which is
> basically what you're suggesting), then the compiler would have been
> no help to me whatsoever. It would not have flagged y=2 as an error.
> 
> I /want/ the compiler to assist me by flagging errors like this. I
> want it to be a compile-time error for f to modify y. But as soon as
> remove the words "const" and "mutable" from that example, I lose that
> compiler-assistance.

You can write your class as:

	int x;
	const int y;


>> The D compiler does not necessarily know all there is to know about a type:
> 
> OK. Good point. I hadn't thought that one through.
> 
> The difficulty is, if you don't allow mutable member variables, then
> we're back to what started this thread - transitivity sucks.

Transitivity doesn't have any use for logical const'ness, I'll agree 
with that.

> See, I was basically arguing with you, that transitivity is a good
> thing -- /provided/ we can have mutable members. Now, if we can't have
> that, then suddenly transitivity no longer seems so welcoming. If
> const-transitivity becomes a strait-jacket, then folk are going to end
> up just not declaring anything const at all, because they can't get
> their code to compile otherwise.

The design patterns const is often used for in C++ won't work in D, that 
I agree with. But there will be new design patterns available, I argue 
very valuable ones, that are impossible with C++ const.