Std Phobos 2 and logging library?

[Robert Jacques]

On Sat, 11 Apr 2009 01:21:04 -0400, dsimcha <dsimcha at yahoo.com> wrote:
> == Quote from Leandro Lucarella (llucax at gmail.com)'s article
>> Andrei Alexandrescu, el 10 de abril a las 16:49 me escribiste:
>> > >And Braddr just made a documentation fix, and Walter only commits
>> > >portability stuff and an occasional bug fix now and then, so...
>> > >Yes, it really looks like a five-person show =)
>> > >I think most work in Phobos now it's done by Andrei, there are other
>> > >*collaborators* (the four other you named plus people sending  
>> patches), but
>> > >it looks like Andrei's show to me. This is not necessarily bad, it's
>> > >definitely  better than before, when it was Walter's show, now at  
>> least he
>> > >can dedicate his efforts in the compiler and language and Phobos is  
>> having
>> > >a lot more attention.
>> >
>> > We'll be very happy to integrate credited contributions from anyone,  
>> and
>> > to give dsource.org write access to serious participants. What I think
>> > right now stands in the way of large participation to Phobos is that  
>> we
>> > all still learn the ropes of D2; the possibilities are dizzying and we
>> > haven't quite zeroed in on a particular style. Nonetheless, as it's  
>> been
>> > noticed I'm always summoning help from this group. So again, if you  
>> feel
>> > you want to contribute with ideas and/or code, don't hesitate.
>> I hope I can come up with something useful with my thesis (improving D's
>> GC) and I can contribute that. Right now all my energies are focused on
>> that, and I'm very close to the point to finally start playing with
>> alternate implementations.
>> BTW, is there any real interest in adding some more power to the GC
>> implementator to allow some kind of moving or generational collector?
>
> Absolutely.  When writing parallel code to do large scale data mining in  
> D, the
> lack of precision and multithreaded allocation are real killers.  My  
> interests
> are, in order of importance:
>
> 1.  Being able to allocate at least small chunks of memory without  
> locking.

After reading Leandro's blog about the current GC, converting the  
free-lists to a lock-free data-structure would be a simple (i.e. library  
only) way to provide this. Another is to provide per thread heaps, which I  
realized this morning can also be done without changing the complier.

> 2.  Precise scanning of at least the heap.
> 3.  Collection w/o stopping the world.

*Sigh*. A concurrent GCs (which is what is generally meant by Collection  
w/o stopping the world) is actually the wrong choice for you. In  
data-mining you're generally concerned with throughput. A concurrent  
collector is used solely for gaining latency back, and does so by  
sacrificing throughput. i.e. the total time your program spends collecting  
is increased. A parallel collector is probably what you're looking for,  
since it decreases the total collection time (i.e. increases your  
throughput) (It also reduces the latency on multi-core systems, which is  
why you often see synergistic parallel-concurrent collectors) And if you  
really want to have your cake (low latency) and eat it too (high  
throughput) there are thread-local heaps.

> 4.  Moving GC so that allocations can be pointer bumps.
>