[OT] Unity's HPC#

[Manu]

On Thu, Feb 28, 2019 at 9:10 AM Nick Sabalausky (Abscissa) via
Digitalmars-d <digitalmars-d at puremagic.com> wrote:
>
> On 2/28/19 7:05 AM, Dukc wrote:
> >
> > This is exactly where D has the largest advantage over C#. I find that
> > C# is generally good to code with, as long as you don't care about
> > performance beyond avoiding big O crimes. But when you try to do stuff
> > like using structs when you don't need polymorhism or trying to minimize
> > allocation when working with an array, it gets inpractical. With D, of
> > course you still need to put a bit more thought in, but it doesn't feel
> > like the language is fighting your optimization efforts.
>
> Yea, at one point, I used to be just as big a fan of C# as D, but
> eventually C#'s difficulty of low-level control and lack of
> expressiveness (compared to D) pushed me away. Although I do still
> consider it one of the few languages I don't actually hate.
>
> Nonetheless, Unity seems to have very much addressed the "low-level
> control" problem with plain-C#. As as for the expressiveness, well, like
> any major game engine developer, they're coming from C++, so as far as a
> D-like level of expressiveness, they wouldn't have had that before, so
> they're not really loosing much in that area.
>
> What I think is important for us to look at is: How does D's offering
> stack up to HPC#, and what could we do to improve?

Their work you describe is almost exactly the work I've been doing for
the last however long.
This is the best opportunity I've ever had for D to demonstrate
motivating value before me; there are *so many* ways we can make
compelling arguments in favour of C++, but somehow we're still just
not quite there.

I've raised a bunch of issues I've been facing from this angle
recently; the `shared` stuff, dll stuff, inline stuff.

> To be honest, I'm not entirely happy with how D compares:
>
> + Expressiveness: The only aspects where I think D has a clear advantage.

Yes, and no. DIP1016 failed, and for all my colleagues, that is the
most annoying thing that appears, every single time, within minutes of
opening their code editors.
It's proven hard to keep people excited and motivated, when they're on
the back foot over something so trivial right off the mark.

Implicit construction is also a major deficiency we have, and people
tend to seriously struggle with that. `alias this` as a solution often
feels awkward, and only goes so far.

> + Memory Layout: We're no worse, but HPC# seems to have caught up to us.

/agree, there's no meaningful distinction here. D has expressions
which can make working with buffers a little but nicer (slices in
particular).

> - "Performance is correctness": I could be wrong, and I hope I am, but
> I'm not sure D does much better than C++ in this area. Certainly not as
> well as HPC# appears to do.

No linear metric exists, but in practise, I think D often does worse
than C++ actually.
I have to work to write performant D code, but when I do work for it,
it's usually very easy to beat C++.

HPC# has a razor focus on this problem set, we can only compete by
developing frameworks which facilitate appropriate patterns for the
problem space.

...and that's hard, because my experience so far, is that we really
need to lean on `shared` to do a good job here, but `shared` is
broken.
I started a big thread on shared a while back, nothing came out of it.
If someone wants to fix shared (remove read/write access), that would
be absolutely mega!

> - Cross-architecture: D has improved here from the past, but it's still
> nowhere near as mature on as wide a range of platforms as Unity is.

I think this is actually okay in a practical sense. I don't think D
has an answer for PS4, but all other targets I'm aware of are okay.

> - Machine code viewer: We have disassembly and such, but judging by
> their screenshot, no tool that's quite that nice.

This is a function of VisualD having low man-power. VisualD has a nice
feature which will compile and disassemble a single source file and
focus on the code at your cursor location, so you can get a quick
disassembly of the code as you're editing it.
The implementation is a little big janky, it needs a little more
polish, but it could get there.

People need to care about VisualStudio to nail this checkmark. The
entire industry uses VS, 100's of thousands of native code developers,
there are practically no exceptions.
VS is not popular in this forum, but people NEED TO CARE about it if
they want to succeed generally, even if they don't use it themselves.
It's the most important piece of tooling by lightyears.

> + Memory Safety: D's probably about on par here.

Not really. It's almost all talk at the moment with little substance.
`shared` is a gaping safety violation, escape analysis isn't baked, we
still don't have ref-counting.

> - Concurrency Safety: We're way beyond many languages, but we have
> nothing that goes nearly as far as the job scheduling system they describe.

Are we? What do we have?
I argue that we have literally nothing. Shared is straight-up broken,
and `const` is often such that you can't use it... what else is there
to say on this story?

I write job scheduling systems of the kind they describe, that's my
job. I tried to work one in D. I started that thread about `shared`,
it went no where... I write it in C++ instead with no meaningful loss
of safety (ie, there is none either way, and we depend on convention).
Infact, in C++ we have const, which helps us a lot. In D we don't even
have `const`, because it falls over so often... so we have lost a key
typesafety tool to assist with concurrent work. We can use `const` to
statically enforce read-only access to shared data. In D, that often
doesn't work out.

> Being basically a better C++, this is exactly the sort of software that
> D should excel at, so it's rather disappointing how D seems to stack up
> to HPC#. We *should* be absolutely demolishing it.

I agree, we should we killing this. I've asked for help over the last
6 months, we've made no progress.
`shared`s broken, inline's broken.
The thread about making -H (.di output) also able to output .h files
is a big part of it.
We have a lot of fundamental issues too, some are in progress like
copy construction, maybe move construction?