Voting For std.experimental.ndslice

[Ilya Yaroshenko via Digitalmars-d]

On Tuesday, 29 December 2015 at 21:19:19 UTC, Jack Stouffer wrote:
> On Tuesday, 29 December 2015 at 17:38:06 UTC, Ilya Yaroshenko 
> wrote:
>> On Tuesday, 29 December 2015 at 17:17:05 UTC, Jack Stouffer 
>> wrote:
>>> On Monday, 28 December 2015 at 14:42:43 UTC, Ilya Yaroshenko 
>>> wrote:
>>>> On Monday, 28 December 2015 at 11:13:57 UTC, Martin Nowak 
>>>> wrote:
>>>>> We're branching for 2.070 soon, would be nice if this can 
>>>>> make it, but only if it's really ready.
>>>>
>>>> Whooohooo!
>>>>
>>>> Thanks!
>>>>
>>>> Ilya
>>>
>>> I'm writing a blog post giving an overview of std.ndslice and 
>>> comparing and contrasting to Numpy, to be released when this 
>>> is merged, and I was wondering if I can have your permission 
>>> to include your great image processing example in the post?
>>
>> Thank you for doing this!
>
> First draft: http://jackstouffer.com/hidden/nd_slice.html
>
> Please critique.

Awesome!

Please find my notes below.

The Basics
======

About `iota.front -> slice.font -> user accessing the data`:

1.
 From the `sliced` documentation for `range` parameter: "only 
index operator auto `opIndex(size_t index)` is required for 
ranges".

See also recently added the last two examples for sliced 
http://dtest.thecybershadow.net/artifact/website-76234ca0eab431527327d5ce1ec0ad74c6421533-387174b023f8cb9612cfcddc76788896/web/phobos-prerelease/std_experimental_ndslice_slice.html#.sliced

The first one is for Input Range primitives, the second one is 
for Random Access Range primitives.

2.
Consequently `Slice` never invokes `front`, `popFront`, `empty`, 
and others range primitives. `Slice` invokes only 
`opIndex(size_t)`, and if `save` is defined, `slice.save` invokes 
`range.save`.

In addition, `sliced` can be used like Finite Random Access Range 
Constructor: user may define only `opIndex(size_t)` and `save` to 
make a Finite Random Access Range by calling `sliced`.

3.
If type of slice is `Slice!(3, Range)`, then `slice.front` do not 
invokes `range[indexToFront]`, it returns a slice type of 
`Slice!(2, Range)`.

In addition, `indexToFront` may not be zero, because primitives 
like `popFront` or operators like `reversed` may be called before.

For slices over arrays `indexToFront` is not defined because the 
pointer always refers to the first position: the pointer is moved 
when the slice changes.

For more details see also examples for Internal Binary 
Representation 
http://dtest.thecybershadow.net/artifact/website-76234ca0eab431527327d5ce1ec0ad74c6421533-387174b023f8cb9612cfcddc76788896/web/phobos-prerelease/std_experimental_ndslice_slice.html#.Slice

Getting Hands On
======
  - Please use "~>0.8.7" instead of "~>0.8.0". "~>0.8.7" has LDC 
0.17.0 alpha1 support.

Examples
======
  - A Basic Example With A Benchmark is not honest (difference 76 
is very large). To make it honest:
      1. Larger 100x1000 matrix can be used instead of 10x10, 
because numpy have a significant initialization overhead.
      2. Both `100_000.iota.sliced(100, 1000)` and 
`100_000.iota.array.sliced(100, 1000)` can be tested. The last 
one is with `std.array.array`. Memory access is expensive.
      3. Only iteration should be tested. Allocation and 
initilization should be separated from iteration both in D and 
Python.

    I expect smaller differnce in perfomance than 76 times.

  - You may want to test both DMD and LDC 0.17.0 alpha1 for 
bechmarks 
https://github.com/ldc-developers/ldc/releases/tag/v0.17.0-alpha1
    LDC 0.17.0 alpha1 works well with dip80-ndslice v0.8.7.

  - Python users love small code and they can be afraid to see 
large (template constraints)
    `mean` funciton.
    An example with lambda function can be added:
    ---
    auto means = 100_000.iota.sliced(100, 10000).rotated(3).map!(r 
=> sum(r) / r.length);
    ---
    One line!

  - Nitpick: `transposed` (without params for 2D slice) is more 
clear than `rotated(3)`, IMO

  - `dub --build=release`can be added where you have noted dmd 
optimisation flags.

Numpy's Main Problem, and How D Avoids It
=====
  - D version with allocation using `std.array.array` can be added 
(or just `new int[1000]`). The slice over `repeat` is not mutable 
(users from numpy world may expect that it is an array).

Ilya