allMembers broke for __

[Jonathan M Davis]

On Sunday, June 17, 2018 01:02:17 DigitalDesigns via Digitalmars-d wrote:
> On Saturday, 16 June 2018 at 23:33:13 UTC, Jonathan M Davis wrote:
> > On Saturday, June 16, 2018 22:59:24 DigitalDesigns via
> >
> > Digitalmars-d wrote:
> >> On Saturday, 16 June 2018 at 06:47:59 UTC, Jonathan M Davis
> >>
> >> wrote:
> >> > On Saturday, June 16, 2018 06:08:17 DigitalDesigns via
> >> >
> >> > Digitalmars-d wrote:
> >> >> When an identifier starts with __, allMembers does not
> >> >> return it. This is very bad behavior! It can silently
> >> >> create huge problems if the programmer is not aware of the
> >> >> bug.
> >> >>
> >> >> Instead, if you want to keep the current behavior at least
> >> >> create an error or warning rather than silently create a
> >> >> bug to waste the users time tracking it down.
> >> >
> >> > I would point out that identifiers that start with two
> >> > underscores are supposed to be reserved for the compiler.
> >> > Declaring them yourself is begging for trouble in general.
> >> >
> >> > - Jonathan M Davis
> >>
> >> So, you are saying that it's ok to beat me over the head and
> >> cause me trouble simply because I used __? Seriously? I
> >> thought D was all about making the programmers life easier and
> >> not more difficult! Thanks for letting me know! I guess it's
> >> time to go back to C#.
> >
> > There's no need to be melodramatic. All I'm saying is that
> > identifiers starting with __ are reserved by the language -
> > just like they are in many C-derived languages (I would expect
> > them to be reserved in C# as well, though I don't know for sure
> > if they are). Whether __traits stuff should ignore such
> > attributes, I don't know (I wouldn't have expected them to be
> > ignored, and that does seem weird), but either way, if you have
> > any variable names that start with two underscores (be they
> > member variables or local variables or whatever), you're
> > potentially declaring variables that are in conflict with stuff
> > that druntime or the compiler itself declare, and you risk bugs
> > by doing so. Even if __traits shouldn't just ignore those
> > identifiers, you shouldn't be declaring any variables that
> > start with two underscores. If you didn't know that before,
> > then I'm sorry, but the spec mentions it, and now you know.
> >
> > https://dlang.org/spec/lex.html#identifiers
> >
> > - Jonathan M Davis
>
> If this was a sane language constraint then any identifiers
> starting with __ that were not reserved would at least give a
> warning but particularly give an error! Not fail silently and
> break code in ways that cannot be determined otherwise. It's bad
> enough that the whole template system cannot be properly debugged
> but to silently ignore user identities that start with __ just
> because is ignorant, and it's not being melodramatic.
>
> I've seen you state several times that D does certain things to
> "protect" the user... and now you are justifying it attacking the
> user. You can't have it both ways, which is it?
>
> __ is not an uncommon identifier prefix and to reserve it then
> have code break in ways that are nearly undetectable and could
> cause major bugs in complex programs is not sane. How about
> reserve a special character for special identifiers and do a full
> rename on the master branch and simply do away with the arbitrary
> __ constraint?
>
> There are correct ways to solve the problem and incorrect.
> Justifying incorrect ways when they are clearly incorrect is not
> sane(but it is typical practice).

Identifiers that start with two underscores should be very uncommon, because
they're usually reserved in C-derived languages (though for better or worse,
most such languages don't actually flag them as errors even though they're
technically illegal). There's really nothing unique about D in that respect.
I suspect the reason that it's not a compiler error in D to start an
identifier with two underscores is because it would make it a pain for
druntime, because - being the runtime for the language - there are a number
of identifiers that it declares which start with two underscores, and if the
compiler tried to make that an error in general, it would either have to
know all of the identifiers that druntime used or somehow know that it was
compiling druntime (and at least some of those identifiers are extern(C) so
that they're not mangled based on the module they're in, making detecting
that something is part of druntime more complicated). It also may be that it
needs to be possible to declare identifiers that start with two underscores
in order to link against code from C's runtime. I don't know.

I don't disagree that it would be nice if the compiler made it an error if
anyone tries to do that in their program, but given how that would affect
druntime, I'm not sure how reasonable that really would be in practice even
if it really should happen in principle. And in practice, programs that have
identifiers that start with two underscores should be pretty rare anyway.
Outside of something like druntime or the compiler, I don't think that I've
ever seen it. So, I honestly would have expected this to be a completely
theoretical problem, though clearly, you weren't aware that identifiers that
start with two underscores are typically reserved by the language and used
them anyway and ran into problems as a result, so apparently, the problem
isn't completely theoretical. I do expect it to be pretty uncommon though.

Regardless, unless there's a reasonable way to have the compiler detect when
it's druntime that's declaring the variable that starts with two
underscores, it's going to be difficult to make it an error for anyone doing
it outside of druntime, and I have no clue how easy that is to do. It could
be that there are technical hurdles that prevent it, and it could be simply
that it was going to be enough of a pain that no one bothered on the
principle that it was highly unlikely to come up in practice. I don't know.
Feel free to create a bug report for it, and maybe the compiler can be
improved to actually treat it as illegal in the cases where it should be
illegal while letting druntime do its thing.

Now, whether __traits should be ignoring identifiers that start with two
underscores is another matter entirely, and I have no idea why that's
happening, but no user code should be being written that has identifiers
that start with two underscores (and that's far from unique to D).

- Jonathan M Davis