D perfomance

[Joseph Rushton Wakeling]

On Sunday, 26 April 2020 at 11:40:49 UTC, John Colvin wrote:
> I understand that it was an annoying breaking change, but aside 
> from the difficulty of migrating I don't understand why a 
> custom type isn't the appropriate solution for this problem. I 
> think I heard "We want to use the built-in slices", but I never 
> understood the technical argument behind that, or how it 
> stacked up against not getting the desired behaviour.
>
> My sense was that the irritation at the breakage was 
> influencing the technical debate.

That's not entirely unfair, but I think it does help to 
appreciate the magnitude of the problem:

   * there's a very large codebase, including many different 
applications and
     a large amount of common library code, all containing a lot 
of functions
     that expect slice input (because the concept of a range was 
never in D1,
     and because slices were the only use case)

   * most of the library functionality shouldn't have to care 
whether its input
     is a reusable buffer or any other kind of slice

   * you can't rewrite to use range-based generics because that's 
D2 only and
     you need to keep D1 compatibility until the last application 
has migrated

   * there are _very_ extreme performance and reliability 
constraints on some
     of the key applications, meaning that validating D2 
transition efforts is
     very time consuming

   * you can't use any Phobos functionality until the codebase is 
D2 only, and
     even then you probably want to limit how much of it you use 
because it is
     not written with these extreme performance concerns in mind

   * all the time spent on those transitional efforts is time 
taken away from
     feature development

It's very easy to look back and say something like, "Well, if 
you'd written with introspection-based design from the start, you 
would have had a much easier migration effort", but that in 
itself would have been trickier to do in D1, and would have 
carried extra maintenance and development costs (particularly 
w.r.t. forcing devs to write what would have seemed like very 
boilerplate-y code compared to the actual set of use cases).

Even with the D1 compatibility requirement dropped, there still 
remains a big burden to transition all the reusable buffers to a 
different type.  IIRC the focus would probably have been on using 
`Appender`.

Note that many of these concerns still apply if we want to 
preserve a future for any of the (very well crafted) library and 
application code that Sociomantic open-sourced.  They are all now 
D2-only, but the effort required to rewrite around dedicated 
reusable-buffer types would still be quite substantial.