[GSoC] 'Independency of D from the C Standard Library' progress and update thread

[Stefanos Baziotis]

On Friday, 28 June 2019 at 12:11:10 UTC, Stefanos Baziotis wrote:
> // NOTE(stefanos): Classes are not tested, more on that on the 
> Blockers.
>

=== Blockers ===

-- Blocker 1 - DMD --

The main blocker was that the project was targetted to DMD. The 
main problems are:
- The optimizer is limited.
- The code generated is a lot of times unpredictable (at least to 
me).
That is, both as far as performance is concerned but 
comprehensibility as well.
- Inline ASM can't be interleaved with pure D.

I want to stress that when writing such performance sensitive 
utilities, the language
is used as the tool to generate the ASM more effieciently (and 
with less errors) instead
of writing it yourself. This is a subjective opinion, but I guess 
that most people
having worked on such utilities will agree.
This is why these utilities are either written in ASM or in a 
language that is low-level
enough and with a good enough optimizer that will let them write 
in this more high-level language.

Now, I picked inline ASM as my preference because with pure D and 
DMD there was:
- Poor debugability. When the ASM is not hand-written, it is not 
as easily comprehensible.
To sacrifice that, the ASM generated from the compiler has to be 
predictable, which for me it wasn't.

- Poor tuning. One should not fight the optimizer. If I expect an 
optimization to be done
and it's not, then that's a problem.

- Poor scalabitliy. If a person after me comes and tries to 
optimize it further, I might have potentially created more 
problems with pure D than what I would have solved. For example, 
if I was that person and I did a compile and there was an 
unexpected load inside a loop that I can't get around by 
transforming the code, then that would be a problem.
Basically, if we go the pure-whatever-language-we-choose way, we 
must never, in the future, say "Better have written in ASM from 
the start". And my prediction was that that would be the case.

I can be a lot more specific on the reasons behind the pick of 
inline ASM, so feel free to ask.

Don't get me wrong, DMD is pretty good but, at least I, could not 
get it to the point
of hand-written ASM.
I want to say that this inline ASM I'm talking about is being 
minimized / removed and is replaced with pure D for various 
reasons.

-- Blocker 2 - Test suite --

In this month, I was working with a test suite that I had not 
examined carefully.
That was certainly my biggest mistake up until now. And that test 
suite was not good.
When I got advised to make a new test suite, that new suite 
revealed serious bugs in the code. That was both good and bad. 
The good thing was that I now had the chance to think
hard on the test suite and that of course the bug were revealed.
But the bad part was that Dmemcpy and Dmemmove had to almost be 
complete remade in 3 days.
It was done, but it was a serious blocker.

In that time, problems with Windows were revealed (specifically, 
the calling convention),
which were also solved, but that was a lot of spent time as well.

-- Blocker 3 - Classes --

The problem with classes is that it is mentioned that the 
compiler can change the layout
of the fields in a class / struct. Even if that means that the 
two hidden fields
(vptr and monitor) are still on the start, it still seems hacky 
to take the class
pointer, move forward 16 bytes and start the operations there 
(and the 16 bytes is not standard because the pointer size 
changes by the operating system). So, we decided
to leave it for now.
My guess is that classes probably will never be used directly in 
such low-level code.

-- Blocker 4 - SIMD intrinsics --

When I started writing Dmemset, I decided to go pure-D first. In 
that effort, there
were 2 ASM instructions that I was trying to get them work for 
about 4 hours. The ASM
instructions are:
         movd    XMM0, ESI;
         pshufd  XMM0, XMM0, 0;

I don't if more details on what I tried matter, but if anyone has 
an idea, please inform me.