More radical ideas about gc and reference counting

[Manu via Digitalmars-d]

On 7 May 2014 01:46, Andrei Alexandrescu via Digitalmars-d
<digitalmars-d at puremagic.com> wrote:
> On 5/5/14, 11:39 PM, Manu via Digitalmars-d wrote:
>>
>> On 6 May 2014 14:09, Andrei Alexandrescu via Digitalmars-d
>> <digitalmars-d at puremagic.com> wrote:
>>>
>>> On 5/5/14, 8:19 PM, Manu via Digitalmars-d wrote:
>>>>
>>>>
>>>> On 5 May 2014 14:09, Andrei Alexandrescu via Digitalmars-d
>>>
>>> So it would be nice if you reviewed that code.
>>
>>
>> I don't really know anything about it... and that's not the point.
>> I'm just suggesting by my prior email that some steps like creating an
>> experimental space with a lower barrier to entry might encourage
>> growth in the number of overall contributors, which I think was the
>> basic flavour of the emails leading up to it.
>
>
> Oh but that's very much the point. "Creating" is the right word (and the
> gerund form is delicious). In order for the "creating" to happen someone
> would need to do the "creation" - and of course the subsequent related work.
> You frame the matter as something "they" should do, when the reality is
> you're very much a part of "they".

Throughout this email, you seem to have completely misunderstood my
intent. I haven't asked you to do anything. I originally replied in
support of prior comments that raised yet again the idea of having an
experimental space with significantly relaxed contribution policy.
I was supporting the notion that an agreement to the existence of an
experimental space might be a valuable thing. It requires nothing more
than discussion and agreement that it is actually something the
community thinks is a good idea.

> The idea of an experimental/unstable package has been discussed a number of
> times. The conclusion I seem to remember is that currently with dub
> available there's a really simple way to experiment with things in a
> comfortable and decentralized way.

Is that something that's general knowledge, or a strategy that people
are already using? This is the first time the idea has ever occurred
to me.

> That said, it's not impossible that distributing things in an
> "std.experimental" package along with the official distribution gives them
> an extra umph. We could arrange for that to happen. Is this what you think
> would make the difference?

Yes. This is basically the entire point of my post.

>>>> What about AST macros? It seems to me that this is never going to be
>>>> explored and there are competing proposals, but I wonder if there's
>>>> room for experimental implementations that anyone in the community can
>>>> toy with?
>>>
>>>
>>>
>>> There would be of course room as long as there's be one or more champions
>>> for it. Would that be something you'd be interested in?
>>
>>
>> I have no horse in that race, but I see it come up all the time, and
>> it is something I am passively interested in.
>
>
> Same here. I'm too busy working in and on other aspects of D (which I
> believe are more important and more urgent) to have another project added to
> my plate. I'd be in your debt if you could clarify to me that AST macros are
> not something you believe I must work on right now.

I haven't suggested you need to work on anything. Other people with
ideas on the topic have appeared to be interested in working on it,
and it seems they need a place where they can produce the experimental
system and make it available to everyone.
I wonder if that would motivate them to do it.

You can see how there's a distinction between contributors getting a
system to an experimental point and then letting people have at it
while it bakes, and getting it to the point it's perfect and ready for
formal review and acceptance, prior to people really having convenient
access to it to try it out?

I'm not sure how you perceive that I'm saying you need to do something
in this scenario, other than consider some changes in contribution
policy as part of a committee.

>> There's at least one DIP which received little attention afaict, it's
>> an example of something that I think would probably manifest into code
>> in an experimental space, but clearly couldn't be accepted as a
>> language feature without lots of field time.
>> In lieu of an experimental space, there will be no action.
>
>
> What makes dub inappropriate as an experimental space?

I don't know. But at face value, I'd suggest that it's a concept
that's quite foreign to Windows users. It never occurred to me until
right now.
If that's a popular approach, then the the dub package listing
probably needs greater visibility. It well maintained?

dub should almost certainly be bundled with DMD if this is to become a
standard approach.

This only addresses libs though, not experimental compiler features.
That needs an agreed policy for enabling/disabling such features, and
I don't see how that can escape living in the DMD repo?

>> It's an interesting example actually. I think lots feel the DIP isn't
>> really an effective solution, but nobody has the motivation or ideas
>> to refine it. The DIP author clearly has no motivation to test it
>> experimentally, but perhaps that's what it needs to progress? The
>> DIP's shortcomings might be discovered by experimental users in the
>> field? It's hard to know, but it's an example of the sort of things
>> that may have a stifling effect on progress and contribution.
>
>
> I think DIPs are a marked improvement over informal discussions on the
> mailing lists. For them to work well, two ingredients are needed: champions
> and reviewers. In some cases that did happen, in others it didn't.

Wholesale inclusion is different than getting something in as an
experimental feature though. If there's a proper policy for
experimental features, and there is a much lower standard against
which they are included, then there'll possibly be more people
experimenting. 10x so for compiler features, since most people don't
build their own compiler; unless experimental compiler features are
included, they'll never be tested by end-users.

I'm not suggesting any change to the DIP system, it's fine, and a good
starting point for new development. But I think there's a lot of DIP's
there which may be interesting features, and I wonder if there would
be a difference in perceived friction if they knew there was an
experimental stage available where the community can initially try new
things out, offer useful feedback, and motivating the push to the
goalpost.

Ideally, experimental features should be available to all D users for
maximum possible user testing. Only people on this forum would ever
find someone's personal fork, merge with their local source and
compile themselves a compiler. Most people even here would never do
that.

>>>> UDA's are super-useful, but they're still lacking the thing to really
>>>> set them off, which is the ability to introduce additional boilerplate
>>>> code at the site of the attribute.
>>>
>>>
>>>
>>> Interesting. Have you worked on a related proposal?
>>
>>
>> Not really, I've initiated numerous discussions which always seems to
>> end at AST macros.
>> The only other semi-reasonable idea I've had is the concept that
>> tagging mixin templates as UDA's might be a practical angle, but it
>> doesn't really make clean sense, creates a syntactic special case and
>> also doesn't seem powerful enough, so I'm not with any solid proposal
>> that I can imagine within the current language framework.
>>
>> There are presently bigger issues that keep me awake at night.
>
>
> Same here. So then I think you'd find it reasonable that I asked you to
> clarify these are not things you believe I need to work on.

I'm not asking you to work on anything.

> What I'm trying to get across here is that your and my role and
> opportunities are very much comparable. Anyone's clout within the D
> community is proportional to the work they're doing on D, and even Walter's
> role would be reduced to an honorific one if he ceased spending time working
> on D.
>
> My pedal is sealed to the floor already. I'm tapped out. Walter's too.
> Meaning that anything work you'd find fit to give us to do would need to
> come at other work's expense. In my case, that would mean overseeing four D
> projects at work and allocators on my free time. Walter's last piece of work
> is interfacing with C++ namespaces, which is very necessary for adoption. He
> and I have gotten pretty good at investing our time efficiently, but at some
> point there's only so many hours in a day and we need talented people -
> people like you - to help making things happen.

I'm not sure what you think I'm asking you to do.
I'm just suggesting to consider a change in policy wrt experimental
contributions.
My suggestions are:

Allow people to commit to an 'exp' phobos package; reduce the approval
burden for people that want to make experimental modules available.
This is a no-action action. Just a decision that it's a good idea or
not.
I imagine criteria for inclusion as an experimental package might be
something like; as long as a decent amount of people agree that it is
a feature that should eventually be in phobos, and that the initial
API is sensible and reasonably conformant, let it exist in exp while
it bakes, prior to the lengthy formal review process.

Allow people to commit experimental features to the compiler, like
Obj-C, so people can try it. There will need to be some agreement on
how experimental compiler features are to be enabled/disabled in a
standard way, and obviously approval would be subject to reasonable
community approval. Again, I don't think that implies any work on your
part.

The difference is, there's no massive threads and debates about what
something should or shouldn't be, and people can get to work, try
things, make it available, get feedback, etc, prior to a formal review
process.
I'd also suggest that experimental code in phobos or the compiler not
be a required burden for GDC/LDC if it runs into trouble with their
update process. Policy should allow them to stub it out, and refer it
back to the exp author.

>>>> I reckon there's a good chance that creating a proper platform for
>>>> experimental features would also have an advantage for community
>>>> building and increase contribution in general. If new contributors can
>>>> get in, have some fun, and start trying their ideas while also being
>>>> able to share them with the community for feedback without fear
>>>> they'll just be shot down and denied after all their work... are they
>>>> not more likely to actually make a contribution in the first place?
>>>
>>>
>>>
>>> I'd say so, but we'd need initiative and quite a bit of work for such a
>>> platform. Would you be interested?
>>
>>
>> Well, in phobos, just approve 'exp' which has been raised countless
>> times. I've got contributions that should be in exp, but instead,
>> they're in limbo, and I've lost momentum and motivation since their
>> completion is blocked by other issues, and I'm receiving no feedback
>> from field testing.
>
>
> I'm fine with adding an "exp" package to phobos if you believe that that's
> what's stopping things from happening. But do you really?
>
>
>> What happened to std.serislisation? There was motion there a year or
>> so back... I was looking forward to it, and did some minor reviewing
>> at the time. I wonder if that's an interesting case study? (I haven't
>> looked)
>
>
> I could ask you the very same thing. What happened to std.serialization? Do
> you want me to work on that as well?

I haven't asked you to work on anything!
I just raised it as a potential example where something might have
gone off the rails because there wasn't an effective experimental
place for it to go while baking.

>> In the compiler... I may be interested, but I don't have any such
>> compiler feature in mind to motivate the effort.
>> I have no idea what an experimental feature platform should look like
>> in the compiler, and if it were to exist, I have no such feature in
>> mind to make use of it, but I have raised examples of others that
>> have.
>
>
> Well then someone else would need to do all that work, so you'd need to
> convince them that it's a good idea.

I don't know what work you refer to. Maybe there is work to do to
support this? I don't know. It seems more like a policy decision to
me.

>>>> I personally feel the perceived unlikeliness of any experimental
>>>> contribution being accepted is a massive deterrence to making compiler
>>>> contributions in the first place by anyone other than the most serious
>>>> OSS advocates.
>>>
>>>
>>>
>>> Contributions make it into the compiler and standard library if and they
>>> are
>>> properly motivated, well done, and reviewed by the core team which is
>>> literally self-appointed. The key to being on the core team is just
>>> reviewing contributions. Have you considered looking at submissions that
>>> are
>>> "hanging around for years"?
>>
>>
>> Perhaps you misunderstood the point of my post. I've watched people
>> make solid contributions that haven't gotten through. That is
>> discouraging to others considering starting their own work, and for
>> the person who has already put in the effort to continue to do so in
>> the future.
>
>
> Contributions haven't gotten through because there are not enough people to
> review them. People like you. We have an inflation of contributions, or if
> you wish a deflation of reviews.

And again, maybe they would have more inertia if they made it 'in' at
an earlier point, as an experimental, and they built a community of
users behind the feature proving it's value, and providing feedback
and motivation?
Experimentals shouldn't require extensive reviews. Basic sanity checks
and code quality compliance are much easier for any uninvested 3rd
party to review/verify.

>> The Obj-C thing as an example. Granted, it's a huge feature and has
>> extensive implications. The Authors have said themselves that they
>> agree it's not 'ready' for inclusion... so, what? It sits and rots?
>> I think it needs an experimental place to live and have people make
>> use of it for what it is. If it's blocked by other unpopular issues
>> that aren't receiving attention, perhaps it's presence will manifest
>> the appropriate motivation to see those other unpopular issues
>> resolved at some point?
>
>
> (I assume you refer to http://wiki.dlang.org/DIP43; couldn't find associated
> pull requests.) I don't know. I am not using Objective C, and I think it
> would be more efficient to have people who already have Objective C
> expertise to author and review the contribution. Would I be glad to see good
> interfacing of D with Objective C? Of course!

But, the question is, should it be accepted as an 'experimental'
(unfinished) feature, that people can access on some flag or whatever?
This particular one has been resurrected multiple times, but always
gets stuck. Would there be more pressure to finish the job if people
were actively using it, despite the fact it's yet incomplete?

>> My point is that successful OSS seems to be about enabling the
>> lowest-friction contribution, and my feeling right now, is that the
>> barrier to entry is high. Whether that's true or not I can't really
>> comment, but it's a perception, and the mental barrier inhibiting the
>> first step is perhaps the most significant barrier of all.
>
>
> Forgive me but from where I stand you seem completely wrong. There are
> currently 190 open pull requests in our github projects. The primary reason
> that keeps them from being merged is good peer review from people like you
> and me. So currently the bottleneck is on the receiving side, not the
> initiating side of contributions.

This is probably true.
I can imagine that I'd feel much more confident and qualified to
review features for inclusion as experimentals, since it should only
require quality compliance and sanity checking, and not deep
understanding of the feature itself, potential interaction with other
features or code that I'm unfamiliar with, etc.

> You can't claim incompetence on 190 distinct matters. You are fluent in D
> and a good engineer who could tell good design from bad.

I'm not even sure what the process it... if I go through and "LGTM" a
bunch of pulls, does someone accept my judgement and click the merge
button?
You can see why I might not feel qualified to do such a thing?

I'll criticise at least 10 pull requests in the morning. I'll be
curious to see what happens.

>> I can't review the Obj-C patch. 1, it's huge, 2, I don't know anything
>> about it, other than I'd really like to use D on iOS and that's a
>> major hurdle.
>
>
> Surely there must be something that should catch your fancy among the
> remaining 189 patches.

We'll find out... when I wake up.

>> Also, the authors themselves said they recognise it's
>> not 'ready'. But is it 'experimental'?
>
>
> Most likely. Have you tried to build their fork?

I download DMD from dlang.org. It would be nice if end-users could try
it out under these conventional usage scenarios.
Are they welcome to have the feature enabled on a switch or pragma or
something, and merge into trunk such that it is available to end-users
in the next release?

>>>> I have no prior experience with OSS, and it's certainly
>>>> a factor that's kept me at arms length.
>>>
>>>
>>>
>>> It's as easy as just reviewing stuff. Acta, non verba.
>>
>>
>> I've never felt I have any particular authority to comment on pulls
>> that I have no experience or vested interest in. (I do occasionally
>> comment on pull requests that I have some interest or knowledge in)
>> I've also had some (hopefully useful) commentary in features that did
>> make it; Win64, UDA's, some traits extensions, and lots of bug reports
>> and fix confirmations.
>
>
> That's a great start, thanks. Keep them coming.
>
>> I'm plenty vocal and active on things I do feel I know about, but
>> they're often pretty radical, unpopular, and rarely come even close to
>> turning into code. I'm pretty certain that nothing left on my short
>> list that I personally *really* care about will ever get pulled, even
>> if I did do the work.
>> There's a perfectly good pull there for not-virtual-by-default. No
>> amount of beating will get that horse through, despite almost
>> unanimous community support. That was... extremely discouraging, to
>> say the least.
>
>
> What can we do (short of pulling that) to be more encouraging of you to do
> good work?

Weigh in on policy relating to an experimental feature space in the
compiler, and in phobos?
How should experimental compiler features be enabled? pragma? switches?
How should experimental libraries be presented? phobos 'exp'? dub?
These are just conversations and decisions.

It's this simple; I (and apparently others, it wasn't my idea) think
an experimental space would be good. Do you agree? It's an idea that
has always been rejected in the past.
My logic is, such a thing may theoretically reduce friction and lower
the bar for new development, and maybe stimulate new contributions and
ideally contributors.