data-oriented struct abstraction ?

[short2cave via Digitalmars-d-learn]

On Saturday, 30 May 2015 at 14:32:01 UTC, Anonymous wrote:
> This may be somewhat related.
>
> http://forum.dlang.org/thread/ckeqxhkqjyvmqodrfhhj@forum.dlang.org#post-m9rj0d:242m9e:243:40digitalmars.com
>
> https://github.com/economicmodeling/soa
>
>
> On Saturday, 30 May 2015 at 14:17:51 UTC, short2cave wrote:
>> On Saturday, 30 May 2015 at 13:57:28 UTC, Rikki Cattermole 
>> wrote:
>>> On 31/05/2015 1:49 a.m., short2cave wrote:
>>>> On Saturday, 30 May 2015 at 12:34:21 UTC, Rikki Cattermole 
>>>> wrote:
>>>>> On 30/05/2015 11:46 p.m., short2cave wrote:
>>>>>> Consider this struct:
>>>>>>
>>>>>> ---
>>>>>> Foo
>>>>>> {
>>>>>>  uint a,b,c;
>>>>>>  float d,e,f;
>>>>>> }
>>>>>> ---
>>>>>>
>>>>>> in a collection:
>>>>>>
>>>>>> ---
>>>>>> Foo[] foos;
>>>>>> ---
>>>>>>
>>>>>> Looping a particular member is not cache friendly
>>>>>>
>>>>>> ---
>>>>>> foreach(foo;foos){/*foo.a = ...*/}
>>>>>> ---
>>>>>>
>>>>>> but to write clear, understable, structured code, the 
>>>>>> struct is
>>>>>> necessary.
>>>>>>
>>>>>> So, is it possible to imagine an abstraction such as
>>>>>>
>>>>>> ---
>>>>>> FooData
>>>>>> {
>>>>>>  uint[] a,b,c;
>>>>>>  float[] d,e,f;
>>>>>>  Foo[] items(){}
>>>>>> }
>>>>>> ---
>>>>>>
>>>>>> that would allow to loop over the data in a cache-friendly 
>>>>>> way ?
>>>>>> You think that an item as the member but they are actually 
>>>>>> stored using
>>>>>> another memory layout ?
>>>>>>
>>>>>> A template for this would be particularly usefull:
>>>>>>
>>>>>> template DataFriendlyStruct(T) if (is(T==struct) && 
>>>>>> isPOD!T)
>>>>>> {
>>>>>> alias DataFriendlyStruct = ...
>>>>>> }
>>>>>
>>>>> What exactly are you doing that such a micro optimization 
>>>>> is necessary?
>>>>
>>>> It's just that i try to concrectly apply some things read in 
>>>> a paper a
>>>> few monthes ago. I won't be able to find the link again but 
>>>> the idea is
>>>> simple, here is a short summary:
>>>>
>>>>> structured types are dead, people gotta stup using classes 
>>>>> and
>>>>> structs. >They're not efficient.
>>> >
>>>> That's not a scoop. Programs are slow because of that but 
>>>> struct and
>>>> class allow a complexity in the design that you can't reach 
>>>> with globals.
>>>>
>>>> The problem if you have global declarations of data 
>>>> everywhere things
>>>> become quickly super heavy, for example:
>>>>
>>>> ---
>>>> float[] effect1_filter_1_omega;
>>>> float[] effect1_filter_2_omega;
>>>> float[] effect2_filter_1_omega;
>>>> float[] effect2_filter_2_omega;
>>>> float[] voice1_osc1_phasemod_source1
>>>> float[] voice1_osc2_phasemod_source2
>>>> float[] voice2_osc1_phasemod_source1
>>>> float[] voice2_osc2_phasemod_source2
>>>> ---
>>>>
>>>> efficient to loop but imagine that we talking about 8 
>>>> oscillators, 32
>>>> voices per oscillator, 2 effects per voice, etc...
>>>>
>>>> so an abstraction is required to give a 
>>>> "pseudo-structured-interface".
>>>
>>> There is one other cost to mention, the human cost. The cost 
>>> to which the developer takes to understand it. That's why I 
>>> called it a micro optimisation.
>>>
>>> But in this case, it will still depend upon what you are 
>>> doing.
>>>
>>> Food for thought. CPU caches are pretty weird now days. You 
>>> can't really predicate what will go into them and when.
>>>
>>> So while yes you would put everything into arrays to group 
>>> the data. You probably will be relying on an item from one 
>>> and an item for another.
>>> So a struct should be a unit of work.
>>>
>>> Also allocated the array in one go. Large as possible. It'll 
>>> make things a little bit happier.
>>>
>>> I can't really talk about much else.
>>
>> template, man, think template. Once it's done it can be 
>> instantiated at the ∞ with any struct used as model. A library 
>> template, a type cons. Once the template done, it has 0 cost.

Damn, that's it (soa)! Thx a lot for the link.