Pandas like features

[data pulverizer]

On Thursday, 5 November 2020 at 20:30:03 UTC, jmh530 wrote:
> On Thursday, 5 November 2020 at 20:22:45 UTC, data pulverizer 
> wrote:
>> [snip]
>>
>> 2. Next I would use memory mapped i/o for storage. Usually 
>> memory mapped files are only accessible by one thread for 
>> security but I believe that this can be changed. For security 
>> you could use cryptographic keys to access the files between 
>> threads. So that memory written in one language can be access 
>> by another.
>>
>> [snip]
>
> One thread only? Sounds like GIL...

Not necessarily. The cryptographic keys are used to access the 
file not to lock it, I believe mmap files can be secured with a 
password, which should be generated cryptograhically as an 
alternative to manually entered and stored somewhere. It protects 
the file from unsanctioned access. Even though the file itself 
will probably only take a single password rather than some 
synchronized rotating mechanism. However it is done, the memory 
will need to be protected.

There should be no reason why multiple processes could not read 
from a file. Only writing would require a lock from other 
processes for obvious reasons.

As I said, I haven't even begun to properly plan an 
implementation yet, just something that I think about from time 
to time.