[OT] Unity's HPC#

[Manu]

On Thu, Feb 28, 2019 at 10:55 PM Nick Sabalausky (Abscissa) via
Digitalmars-d <digitalmars-d at puremagic.com> wrote:
>
> On 2/28/19 3:31 PM, Manu wrote:
> >
> > I've raised a bunch of issues I've been facing from this angle
> > recently; the `shared` stuff, dll stuff, inline stuff.
>
> My sympathies, I feel your pain :/
>
> But on a side note, this makes me think: We could *really* use a
> different format for all this argumentative stuff. Forums are too
> transient. What we need is something like a Wiki, but organized like this:
>
> - Ideas
>     - Arguments For/Against, and/or possible Cautions
>        - Rebuttals
>           - Counter-Rebuttals
>
> ...For everything, all in one canonical place. Wouldn't that be nice? I
> get so tired of everyone's arguments about everything having no viewable
> structure whatsoever...it's no wonder nobody's arguments ever get
> anywhere!!! This would be so much more practical as a standard
> base-of-operations for (hopefully) a meritocracy, don't you think?

Yeah, I dunno what format.
It's already fragmented, there's slack, there's discord, this forum,
and IRC, and there's subsets of the community on each one.
Reddit is a better forum, and stack-overflow needs to be more lively
for PR reasons.
There's already a lot.

> >> To be honest, I'm not entirely happy with how D compares:
> >>
> >> + Expressiveness: The only aspects where I think D has a clear advantage.
> >
> > Yes, and no. DIP1016 failed, and for all my colleagues, that is the
> > most annoying thing that appears, every single time, within minutes of
> > opening their code editors.
> > It's proven hard to keep people excited and motivated, when they're on
> > the back foot over something so trivial right off the mark.
> >
>
> Ugh, I had to look up "DIP1016"...It's rvalue references,
> and...just...ugh...
>
> See, this is something we should've had ten freaking years ago. It's so
> freaking basic, it's so freaking obvious. I mean, I'm all for the *idea*
> of some sort of DIP system, as long as it isn't too red-tape-y. Heck, I
> started working on a DIP rough draft recently and the need for technical
> details has really *helped* it, but...rvalue references?? That should
> never have even needed a DIP: You make a freaking temporary as
> necessary. Done! Worried about start/end lifetime of the temporary?
> Interactions? It's pretty freaking obvious what choices do/don't work,
> and if we *still* get it wrong, the test suite will tell us. That's the
> *point* of the test suite - to tell us when/how we've broken something.
> But NO! We've gotta define a manual process to soak up as much time as
> it takes to catch things BEFORE the test suite has a chance to!

I need to not think about it anymore. It's really bad for my mental health.

> >> - Cross-architecture: D has improved here from the past, but it's still
> >> nowhere near as mature on as wide a range of platforms as Unity is.
> >
> > I think this is actually okay in a practical sense. I don't think D
> > has an answer for PS4, but all other targets I'm aware of are okay.
>
> This surprises me actually. My perception was that the PS4/XB1 were very
> on-par with each other in terms of architecture: Both are x64-based with
> PC-like GPUs (and without all the hardware variation within a single
> platform, like on desktop/laptop). Wouldn't have expected any
> significant differences regarding a compiler's ability to generate
> machine code. Or is this more an OS issue than a hardware one? (Or not
> allowed to say?)

XBox runs windows, PS4 runs a modified FreeBSD. We have Win64
compilers. I don't know how much Sony have messed with their
LLVM/Clang.
Architecture isn't the problem, it's having a toolchain that targets
the os/system.

> >> - Machine code viewer: We have disassembly and such, but judging by
> >> their screenshot, no tool that's quite that nice.
> >
> > This is a function of VisualD having low man-power. VisualD has a nice
> > feature which will compile and disassemble a single source file and
> > focus on the code at your cursor location, so you can get a quick
> > disassembly of the code as you're editing it.
> > The implementation is a little big janky, it needs a little more
> > polish, but it could get there.
>
> I guess that sounds promising at least, y'know, considering. 'Least it's
> not something fundamental that requires changing any higher-ups minds ;)

Oh yeah, Rainer punches way above his weight, but it's such critical
tooling that it really needs extensive and ongoing investment.
It has to compare favourably against some of the best tooling out
there. C# tooling is off the charts, there's nothing even remotely
close. There's nothing about D structurally that should prevent a
similarly excellent experience.

> >> - Concurrency Safety: We're way beyond many languages, but we have
> >> nothing that goes nearly as far as the job scheduling system they describe.
> >
> > Are we? What do we have?
> > I argue that we have literally nothing. Shared is straight-up broken,
> > and `const` is often such that you can't use it... what else is there
> > to say on this story?
>
> I guess I was just thinking of TLS-by-default. Perhaps I overstated D's
> case.

Right, TLS by default is really helpful, but in C++ where you just
write a framework which implements that reality, then you're on the
same playfield.
The trouble comes when you DO interact with shared data. In C++ you
can't distinguish it from the non-shared data, so there's no way
beyond convention to do shared interaction.
In D, `shared` should help us, but it's completely unsafe, while
pretending to give the impression that it is. It should be an
excellent tool, but it's dangerously broken, and it turns out that a
totally broken feature is worse than if the feature didn't exist in
terms of public opinion.
We just need to fix it. It would be really easy to fix!
I suspect someone that knew how could do it in a couple hours.

> > Infact, in C++ we have const, which helps us a lot. In D we don't even
> > have `const`, because it falls over so often...
>
> I admit, I don't really use it much myself. Partly because of bad
> experiences attempting to do so.
>
> Do you think the lack of both tailconst and logical-const are the
> ultimate issues here, or does it go deeper than that?

I don't know. I do know that I've spent 10 years trying to defend D's
const, and also trying to use it in practise... but I've never
succeeded generally.
It always hits a wall somewhere along the way. I think you might be
able to make it work for small programs, when you're super thoughtful
and careful along the way. I don't think that's practical, or
particularly useful.

I think one of 2 things; a series of articles and demonstration
material needs to be produced to show how to successfully interact
with D's const, or it should be admit that it's a nice fantasy, and
relaxed. I'd prefer the former, but I'm not actually sure it's not
just fantasy as the project becomes 'real' in scale/scope.
I also think that many super-common designs and patterns are
effectively const-incompatible. We need to know that such designs are
essentially _incompatible with D_, and discourage those forms of
program. I don't know what they are. I only realise when I find myself
in the hole.

> >> Being basically a better C++, this is exactly the sort of software that
> >> D should excel at, so it's rather disappointing how D seems to stack up
> >> to HPC#. We *should* be absolutely demolishing it.
> >
> > I agree, we should we killing this. I've asked for help over the last
> > 6 months, we've made no progress.
> > `shared`s broken, inline's broken.
> > The thread about making -H (.di output) also able to output .h files
> > is a big part of it.
> > We have a lot of fundamental issues too, some are in progress like
> > copy construction, maybe move construction?
> >
>
> I honesty find D very frustrating these days. On one hand, it's still by
> far my favorite language, due to expressiveness, ranges, the low-level
> abilities it does have (unlike most languages these days), and all of
> the basic common-sense pragmatism that went into its early design about
> ten or so years ago. But OTOH, I feel the focus on pragmatism and common
> sense has been thrown straight out the window for the past many years,
> and its already accelerating down the same paths that led C++ to become
> the mess that it is today. I feel like D's obsessively doing *exactly*
> what Scott Meyers warned us against, in stark contrast to the early
> principles that made D worthwhile in the first place.

I'm frustrated beyond reason. I don't know how to be happy anymore.
Should I let it go? Sunk-cost fallacy...

It comes to this; I can only really engage with D as a toy, and that's
absolutely not what I want.
I want to use D professionally, that is all. If I can't use it
professionally, then I'm wasting my short lifetime here.

I've tried to push it in the direction that it can be useful to me for
a really long time, we've made ground, but we're still nowhere near
the mark, and the timeline involved is unreasonably long.
I've written and shipped like 5-6 gigantic projects in C++ in the
meantime. Those were real programs/games that did real things,
customers enjoyed, and made real money.
I don't know that if I'm honest about the trajectory that I'm able to
satisfy myself that we'll get there. So what should I do? :/

Truth is, we probably need to do what Unity did with HPC#; fork D, and
have a small group with a laser focus on making it fit for purpose.
I think that can only occur at a bold company. I suspect that's
chicken/egg stuff :/