[GSoC] 'Independency of D from the C Standard Library' progress and update thread

[Nicholas Wilson]

On Friday, 28 June 2019 at 12:14:13 UTC, Stefanos Baziotis wrote:
> On Friday, 28 June 2019 at 12:11:10 UTC, Stefanos Baziotis 
> wrote:
>> // NOTE(stefanos): Classes are not tested, more on that on the 
>> Blockers.
>>
>
> === Blockers ===
>
> -- Blocker 1 - DMD --
>
> The main blocker was that the project was targetted to DMD. The 
> main problems are:
> - The optimizer is limited.
> - The code generated is a lot of times unpredictable (at least 
> to me).
> That is, both as far as performance is concerned but 
> comprehensibility as well.
> - Inline ASM can't be interleaved with pure D.
>
> I want to stress that when writing such performance sensitive 
> utilities, the language
> is used as the tool to generate the ASM more effieciently (and 
> with less errors) instead
> of writing it yourself. This is a subjective opinion, but I 
> guess that most people
> having worked on such utilities will agree.
> This is why these utilities are either written in ASM or in a 
> language that is low-level
> enough and with a good enough optimizer that will let them 
> write in this more high-level language.
>
> Now, I picked inline ASM as my preference because with pure D 
> and DMD there was:
> - Poor debugability. When the ASM is not hand-written, it is 
> not as easily comprehensible.
> To sacrifice that, the ASM generated from the compiler has to 
> be predictable, which for me it wasn't.
>
> - Poor tuning. One should not fight the optimizer. If I expect 
> an optimization to be done
> and it's not, then that's a problem.
>
> - Poor scalabitliy. If a person after me comes and tries to 
> optimize it further, I might have potentially created more 
> problems with pure D than what I would have solved. For 
> example, if I was that person and I did a compile and there was 
> an unexpected load inside a loop that I can't get around by 
> transforming the code, then that would be a problem.
> Basically, if we go the pure-whatever-language-we-choose way, 
> we must never, in the future, say "Better have written in ASM 
> from the start". And my prediction was that that would be the 
> case.
>
> I can be a lot more specific on the reasons behind the pick of 
> inline ASM, so feel free to ask.
>
> Don't get me wrong, DMD is pretty good but, at least I, could 
> not get it to the point
> of hand-written ASM.
> I want to say that this inline ASM I'm talking about is being 
> minimized / removed and is replaced with pure D for various 
> reasons.

inline asm is generally very bad for the optimiser because is can 
have any side-effects and is completely opaque. It is possible to 
generate the asm with string mixins, see e.g. the BigInt routines 
in phobos.

You should test your work with LDC at some point which has an 
optimiser worth using, but note the bit about opaque inline ASM 
hurting performance.

> -- Blocker 2 - Test suite --
>
> In this month, I was working with a test suite that I had not 
> examined carefully.
> That was certainly my biggest mistake up until now. And that 
> test suite was not good.
> When I got advised to make a new test suite, that new suite 
> revealed serious bugs in the code. That was both good and bad. 
> The good thing was that I now had the chance to think
> hard on the test suite and that of course the bug were revealed.
> But the bad part was that Dmemcpy and Dmemmove had to almost be 
> complete remade in 3 days.
> It was done, but it was a serious blocker.
>
> In that time, problems with Windows were revealed 
> (specifically, the calling convention),
> which were also solved, but that was a lot of spent time as 
> well.
>
> -- Blocker 3 - Classes --
>
> The problem with classes is that it is mentioned that the 
> compiler can change the layout
> of the fields in a class / struct. Even if that means that the 
> two hidden fields
> (vptr and monitor) are still on the start, it still seems hacky 
> to take the class
> pointer, move forward 16 bytes and start the operations there 
> (and the 16 bytes is not standard because the pointer size 
> changes by the operating system). So, we decided
> to leave it for now.
> My guess is that classes probably will never be used directly 
> in such low-level code.

You should be able to get the offset of the first member with

int foo()
{
     static class A { int a; }
     return A.init.a.offsetof;
}

which will apply to any other non-nested class.

>
> -- Blocker 4 - SIMD intrinsics --
>
> When I started writing Dmemset, I decided to go pure-D first. 
> In that effort, there
> were 2 ASM instructions that I was trying to get them work for 
> about 4 hours. The ASM
> instructions are:
>         movd    XMM0, ESI;
>         pshufd  XMM0, XMM0, 0;
>
> I don't if more details on what I tried matter, but if anyone 
> has an idea, please inform me.

Take a look at https://github.com/AuburnSounds/intel-intrinsics

Keep up the good work!