LuaD: creating a flexible data filter system

[Chris via Digitalmars-d-learn]

On Friday, 16 October 2015 at 09:01:57 UTC, yawniek wrote:
> hi,
>
> i'm reading in a stream of data that is deserialized into 
> individual frames.
> a frame is either of:
> a)  a specific D datastructure ( struct with a few 
> ulong,string,string[string] etc members), known at compile time
> b) json (preferably stdx.data.json)
>
> i now want to implement something where i can dynamically
> add lua filters that then get data out of these frames, create 
> a new lua object
> send it back to D code where its either sent to another lua 
> filter or at last
>  being serialized again to json und then being processed 
> further.
>
> ideally i would not like to copy all the data into a lua object 
> but directly access it
> from lua.
> is there an elegant approach to do this and support both a and 
> b cases?
>
> so far i did some benchmarks, mainly with string comparisons 
> and it turned out
> that luaD is about 10x faster than mruby and python D bridges.

You could use the Lua C API directly. I have a small project 
where I use DerelictLua[1] and access the Lua C API directly for 
fast processing (cf. http://www.lua.org/manual/5.3/ or 
http://www.lua.org/manual/5.2/ "C API").

Another performance trick is to compile lua functions and store 
them in memory, rather than having lua read and execute 
everything.

lua_State* L = luaL_newstate();

auto func = luaL_loadstring(L, "Lua code as C-string\0");  // Lua 
file compiled as a function

Later you call the function with the Lua C API like "lua_pcall(L, 
0, 1, 0);". It's a bit tricky to move things around on the Lua 
stack, but you'll get there! ;)

[1] https://github.com/DerelictOrg/DerelictLua