dcollections 1.0 and 2.0a beta released

[Andrei Alexandrescu]

On 05/24/2010 10:01 AM, Steven Schveighoffer wrote:
> On Fri, 21 May 2010 13:42:14 -0400, Andrei Alexandrescu
> <SeeWebsiteForEmail at erdani.org> wrote:
>
>> On 05/19/2010 08:42 PM, Steven Schveighoffer wrote:
>>> Andrei Alexandrescu Wrote:
>>>
>>>
>>> I.e. there aren't many kinds of HashMaps that derive from each
>>> other. But the interfaces are not detrimental to your ideas. The
>>> only thing interfaces require is that the entities implementing
>>> them are classes and not structs. As long as you agree that
>>> classes are the right call, then interfaces can co-exist with
>>> your other suggestions without interference.
>>
>> This brings back a discussion I had with Walter a while ago, with
>> echoes in the newsgroup. Basically the conclusion was as follows:
>> if a container never escapes the addresses of its elements, it can
>> manage its own storage. That forces, however, the container to be a
>> struct because copying references to a class container would break
>> that encapsulation. I called those "perfectly encapsulated
>> containers" and I think they are good candidates for manual memory
>> management because they tend to deal in relatively large chunks.
>>
>> I noticed that your collections return things by value, so they are
>>  good candidates for perfect encapsulation.
>
> Then you need reference counting, I don't think that is a good
> design.

Why?

I think refcounting for containers is perfect.

Refcounting small objects is bad because the counter overhead is large.
Also, small objects tend to be created and passed around a lot, so the
time overhead is significant.

In contrast, refcounting and containers are a perfect marriage. The
container is a relatively large conglomerate of objects so refcounting
that is bound to yield good benefits in terms of memory reclamation.

>>> Yes, if you want to define "this function needs something that
>>> is both addable and purgeable, I don't have an interface for
>>> that. But a concept can certainly define that generically (which
>>> is what you want anyways), or you could just say "I need a List"
>>> and get those functions also. It also does not force entities
>>> other than dcollections objects to be classes, they could be
>>> structs and implement the correct concepts.
>>>
>>> I myself don't really use the interface aspect of the classes, it
>>> is mostly a carryover from the Java/Tango inspirations.
>>
>> I don't know Tango, but Java's containers are a terrible example to
>>  follow. Java's container library is a ill-advised design on top of
>> an underpowered language, patched later with some half-understood
>> seeming of genericity. I think Java containers are a huge
>> disservice to the programming community because they foster bad
>> design.
>
> Java has some warts as you have rightfully pointed out in the past
> (i.e. O(n) lookup), but I have attempted to remove all those warts.
> Dcollections I would hope does not suffer from the problems that
> Java's collections suffer from.

That's great. But let me quote what you said: "I myself don't really use
the interface aspect of the classes, it is mostly a carryover from the
Java/Tango inspirations." I took that to mean you don't have a strong
justification for structuring dcollections as a hierarchy, and
furthermore that makes me hope it's possible you'd be willing to yank
that dinosaur brain.

>>> But I can see one good reason to keep them -- binary
>>> interoperability. For example, it might be the case some day when
>>> D has good support with dynamic libraries that a library exposes
>>> some piece of itself as a Map or List interface.
>>
>> I need to disagree with that. I've done and I do a ton of binary
>> interoperability stuff. You never expose a generic container
>> interface! Interoperable objects always embody high-level logic
>> that is specific to the application. They might use containers
>> inside, but they invariably expose high-level, application-specific
>> functionality.
>
> It's done all the time in Java and .NET. For example, A GUI listbox
> widget exposes its elements as an array of elements, which implement
> the List interface. You don't ever see the implementation or need it.
>  Granted Java and .NET have less problems than C++ and D with binary
>  compatibility, since the function tables are dynamic, but the
> potential is there for D to make binary compatibility possible with
> interfaces.

I see.

> In the past I have built a C++ library that abstracted features of
> the OS. My goal was to make it possible to dynamically load a module
> that abstracted things like setting the IP address of a network
> interface. My modules used std::string instead of char * to lookup
> services to get objects that implement the interface. Big mistake. On
> a later version of the standard C++ runtime, the private
> implementation of std::string changed, so the dynamically loaded
> libraries crashed horribly. No change in string's interface, just the
> private stuff changed, but because it's a template, the code that
> uses it necessarily has to be aware of it. We ended up ditching the
> standard C++ library's version of string, and used STLPort so we
> could control the library.
>
> I envision this same sort of problem would be likely with D
> collection objects that were not used via interfaces.

I see no problem retrofitting a no-interface container into a formal
interface if so needed.

>>> So my answer is -- go ahead and define these concepts and
>>> required names, and you can ignore the interfaces if they don't
>>> interest you. They do not subtract from the possibilities, and
>>> others may find good use for them.
>>>
>>> Does that make sense?
>>
>> I understand I could ignore the interfaces and call it a day, but
>> it seems that at this point we are both convinced they are not
>> quite good at anything: you only put them in because you suffered
>> the Stockholm syndrome with Java, and I hate them with a passion.
>
> The reason I put them in is because they existed before, but thinking
>  about what they would be useful for, D doesn't really support
> dynamic libraries all that well (recent advances have brought that
> closer). But once it does, I think it will be much more important to
> keep compiled libraries compatible between versions of the standard
> library. This is typically not possible with templates, I know C++ is
> horrible at it from personal experience. So I'm not convinced it is
> not quite good at anything. I think it's a solution to a problem that
> doesn't exist yet on D because D's linking features are stuck in
> 1995.

Clearly interfaces have their advantages. I just happen to think they
are much thinner than their disadvantages for this particular case.

>> Why would we keep in the standard library bad design with the
>> advice that "if you don't like it ignore it"?
>
> People have continuously used that argument against const and
> immutable. Really that argument is fine, as long as you don't
> consider it bad design, which I don't :)

Looks like we're heading straight to stalemate once again.

In the interest of time, it would be better to get to stalemate (or, 
hopefully, agreement) so we know whether dcollections will be integrated 
within Phobos or whether I should spend this and next weeks' evenings 
coding. To that end, please let me know whether it's worth that I spend 
time on putting together a list of proposed changes.


Andrei