Rectangular multidimensional arrays for D

[Stefan Frijters]

On Wednesday, 9 October 2013 at 08:30:11 UTC, Denis Shelomovskij 
wrote:
> 09.10.2013 7:55, Nick B пишет:
>> On Tuesday, 8 October 2013 at 17:26:46 UTC, Stefan Frijters 
>> wrote:
>> andrei wrote:
>>
>> I too are interesteed in this area as well. Dennis do you only 
>> plan to
>> focus on multidimensional arrays only, or will you incorporate 
>> the above
>> matrices as well  ??
>>
>> What features are you proposing ?
>
> I propose stuff for "multidimensional arrays only" as you 
> noted. And I plan just to revise my existing API [1] without 
> cardinal changes.
>
> I.e. all I propose is rectangular multidimensional arrays 
> slicing and iterating. For matrix and math specific tasks see 
> DScience [2] and SciD [3]. The latter started as a fork of 
> DScience but became a separate project and is in development. 
> See its wiki [4]. Also such math oriented libraries have to be 
> partially (and the are) wrapper around LAPACK.
>
> Also it will be interest to see features you (Stefan and Nick) 
> need e.g. as examples of code you want to compile with comments 
> if needed. Write down at least basic features for now.
>
> [1] 
> http://denis-sh.bitbucket.org/unstandard/unstd.multidimensionalarray.html
> [2] https://github.com/dscience-developers/dscience
> [3] https://github.com/kyllingstad/scid
> [4] https://github.com/kyllingstad/scid/wiki

Ok, off the top of my head, here are some of the points that 
would be great for me to have. I apologize in advance if any of 
them are trivial / irrelevant or out of scope; I have not had 
time to get my hands dirty on this subject. Even if they are not 
to be part of the generic multidimensional array (MDA) module, 
these are things that I would then like to build my own 
implementation of without having to work with instead of against 
the things that will be in Phobos.

- Many of my operations involve looping over the array in no 
particular order, so the first foreach example in your link #1 
will be very useful.
- Another very common operation is accessing a lattice site and 
looking at its neighbours to determine the outcome of the 
operation. Of course this is easy for nested for-loops as I can 
just nest one deeper and pre-calculate the neighbour offsets in 
another array, but I don't know if there's a canonical way to do 
this in terms of a foreach loop, and if this would add 
requirements to the MDA. As an example, Python's numpy seems to 
have 'generic_filter' for tasks like this[1]. In my testing it 
was very slow though.
- I will have multiple MDAs, containing information like local 
densities and velocities. These will affect each other in 
calculations and thus being able to use zip and friends would be 
very useful. This would require the MDA to be a range I guess?
- My code will use wrapped MPI[2] and HDF5[3] calls for 
parallelism and parallel IO, respectively, and because of that I 
will need some control over the memory layout. Nothing fancy, but 
the usual C-style pointer arithmetic would need to work I think, 
unless there's a nicer mechanism.

I hope these comments can be of some help.

Cheers,

Stefan

[1] 
http://docs.scipy.org/doc/scipy/reference/generated/scipy.ndimage.filters.generic_filter.html#scipy.ndimage.filters.generic_filter
[2] http://en.wikipedia.org/wiki/Message_Passing_Interface
[3] http://www.hdfgroup.org/HDF5/