D array expansion and non-deterministic re-allocation

[Bartosz Milewski]

Steve, I don't know about you, but this exchange clarified some things for me. The major one is that it's dangerous to define the semantics of a language construct in terms of implementation. You were defending some points using implementation arguments rather than sticking to defined semantics. We have found out that one should never rely on the array being re-allocated on expansion, even if it seems like there's no other way. The only correct statement is that the freshly expanded part of the array is guaranteed not to be write-shared with any other array. 

However, this discussion veered away from a more important point. I don't believe that programmers will consciously make assumptions about re-allocation breaking sharing. 

The danger is that it's easy to miss accidental sharing and it's very hard to test for it. 

That was my original point and I haven't been convinced otherwise.


> Bartosz Milewski Wrote:
> 
> > The problem here is that the guy is sort of right. Indeed quote() expands the slice before overwriting it and that requires re-allocation to avoid stomping over the original buffer. There is however a boundary case, when the whole buffer matches the pattern. Then the expansion is done in place and the buffer is modified. This would never happen if expansion were guaranteed to re-allocate. 
> 
> Actually, the requirement for not reallocating is that the slice ends at the end of an allocated block, so for example if the pattern matched at the end of the buffer, you may have problems.
> 
> The larger problem in the code is that the developer expects reallocation when it is not guaranteed.  That is, he is *depending* on reallocation.  In that case, you should explicitly reallocate.  This doesn't necessarily mean using .dup, the following works just fine:
> 
> return '"' ~ word ~ '"';
> 
> And it actually performs much better than the example.  In all honesty, I was sort of trying to play your game for this example.  In real life, I would have mentioned that the above line replaces the quote function, and most likely the coder would have accepted it not because it deterministically allocates but because it's cleaner and shorter :)
> 
> > 
> > What this example was supposed to illustrate is that it's easy to forget to explicitly duplicate an array, especially if it's not clear who's responsible--the caller or the callee. 
> 
> I don't think the example illustrates that.  It illustrates that there are convoluted ways to try and explicitly use appending to duplicate arrays, and why one shouldn't do that.
> 
> -Steve