Generators in D

[Piotr Szturmaj]

Sean Kelly wrote:
> On May 17, 2011, at 7:37 AM, Piotr Szturmaj wrote:
>>
>> In regards to serializable fibers, I've found some interesting potential
>> usage - session handling in HTTP server. Fiber code could represent
>> session and could be saved to disk in the middle of execution, just
>> like sessions are saved in PHP. This is used to offload memory
>> when handling great number of sessions.
>>
>> If serialization will be clever enough, these fibers could even be shared
>> across different servers! This is really a requirement in load
>> balanced environments.
>
> The trick is serializing the map of dynamic objects referenced by the Fiber's
> stack.  And more generally, dealing with any pointers on the stack
> at all.  It's the same problem as pointers in a memory-mapped file.
> That said, you really shouldn't have to serialize Fibers just to offload
> memory.  That's what the virtual memory manager is for.

While I fully agree on your opinion about virtual memory manager, I 
really meant something different. Offloading memory is not the sole reason.

I didn't state that before, but I thought of fibers which do not 
reference any external data. They should be self-contained in terms of 
referenced memory.

All external data should be passed as arguments to fiber function. Fiber 
should make a copy of all passed data.

Other ideas involve "persistent" objects that should always be there. 
Upon deserialization Fiber should be "reconnected" to these object -- it 
will just get fresh references.

Offloading memory is useful in case of sessions that may be unused for a 
long time. This include web sessions but not only them. Great examples 
are online games Second Life and EVE Online.

"SecondLife required that code be suspended at any point in time and 
that its entire state be serializable into a format suitable for storage 
into a database. Serialized state could then be restored at a different 
point in time or on a different computer (for example while moving from 
node to node)."

EVE Online uses stackless python in similar manner.

So, I find it very interesting, especially if it would be possible to 
use with D :-)

Here are some interesting links I found, about serializing 
fibers/coroutines/continuations/etc.:

http://stackoverflow.com/questions/734638/language-that-supports-serializing-coroutines
http://mikehadlow.blogspot.com/2011/04/serializing-continuations.html
http://tirania.org/blog/archive/2009/Apr-09.html

Piotr