Self Optimizing Code

[crimaniak via Digitalmars-d]

On Tuesday, 2 August 2016 at 22:06:38 UTC, Mark "J" Twain wrote:
> Instead, a better solution would be to use variables:
>
> if (n*length > m*capacity) expand(l*length)

Some time ago I played with self-optimizing cache layer.
Problem: Time to obtain cache items is unknown and server 
dependant. For example, network can be involved in this. 
Sometimes is it localhost, sometimes server on another continent. 
Time to cache it is unknown too. For example if memcached 
extension is not installed on this server then fallback to disk 
backend will slow cache performance very much. So we don't know 
is it worst to cache.
Solution: cache measures own performance and acts accordingly.

So I make next things:
1. Functional interface instead of save/load type interface. 
cacheLevel.get(functor_to_get_item) allow to measure item 
obtaining time.
2. Implement all measuring/controlling logic in separate class 
with interface AbstractStatist, in CacheLevel class make just 
hooks for it. So changing AbstractStatist implementation I can 
change work mode to measure statistics and use it or work as 
usual cache (EmptyStatist for all empty hooks). I make 
implementation to skip all items not worst caching 
(calcTime/calcHits*totalHits-totalTime < 0) but it's possible to 
make more smart things (like caching only most efficient items 
not exceeding cache size).

In my experience I can draw some conclusions.
1. It need to separate measure mode and control mode. You can't 
have accurate statistics while changing system behavior according 
to current statistics state.
2. Statistics can be different for different applications but for 
specific application in specific conditions for most cases it can 
be approximated as constant.

So for array allocating strategy more realistic scenario the 
next, I think:
1. Application compiled in some 'array_debug' mode then some 
statist trait added to array, collect usage statistics and writes 
optimal constants at the application exit.
2. Programmer configure array allocator in application according 
to these constants.
3. Application builds in release mode with optimal allocation 
strategy and without any statist overhead and works fast. Users 
are happy.