RPC and Dynamic function call
Denis Koroskin
2korden at gmail.com
Fri Nov 20 18:45:48 PST 2009
I am working on a prototype of PRC library and it involves a wide range of
techniques that I also implement. I'd like to share my code,
implementation details and issues I come across. I will split my report
into a few posts. This one is mostly an introduction.
RPC is a concept that allows to execute some code in a remote application.
To make it work, you must know function name, a set of arguments it
accepts, type it returns etc.
This is usually is not a problem if local and remote applications are same
or based on shared code.
The two applications may be ran in different environments (different
Operation Systems, different memory modes, x86/x64 etc).
In order to make a call a few components are required:
Function (one that we want to call), context (remote object reference; not
needed for static member functions and global/free functions), input/ouput
arguments.
A RPC requested is created locally. It contains everything in a list
above. Input arguments might be reference type, in which case they might
have references to other objects which in turn have references to other
objects etc. In general, everything accessible through a set of input
arguments might be accessed on remote side. This is why the whole object
hierarchy accessible through input arguments need to be saved, sent over
network and fully restored on remote side.
This is called serialization. The fastest and most compact form of
serialization is binary serialization. I'll talk about serialization in
one of the next posts.
Next thing we need to do is to somehow encode the function we are gonna
call in a way that remote application will understand and get a proper
function pointer out of this information. This has something to do with
reflection (although pretty simple mechanism is required).
Okay, now we received a RPC request that contains function pointer and a
set of input arguments. We also have some knowledge on ouput arguments.
All we need to do is to call that function and return back result.
At this point, it's worth to note that all we have at remote side is some
void[] array. No types. Once we step out of type system, there is no way
back. It means that you can't really call a function in a type-safe manner
anymore (well, one could create a set of trampolines for each of set of
types involved in a call, but I don't think it's reasonable or even
possible; I'll look into it, too, though). That's why you need to make a
dynamic call: iterate over some kind of input arguments' type information,
push values to stack, call function via funcptr, store the result etc.
I was the last piece to make RPC work, that's what I was implementing
today and that's what I will share in this post. The resulting code is
quite small and simple.
Note that all the asm it uses is just 3 constructs: naked, ret and call!
Everything else is implemented with a help of compiler (i.e. pushing
arguments, grabbing result, stack alignment etc). My code is most probably
not very portable (I didn't test it on anything other that Windows), it
doesn't account x64 specifics etc, but it shouldn't be hard to fix.
When could it be useful? A few applications include RPC, reflection
(possibly), binding to other languages (including scripting ones), code
injection/hijacking, run-time code generation etc.
Any suggestions, critics, comments are highly appreciated.
Full listing below (tested with DMD2.036, Windows XP / Windows 7):
module DynamicCall;
import std.traits;
import std.stdio;
enum ParamType
{
// No return value
Void,
// ST0
Float,
Double,
Real,
// EAX
Byte,
Word,
DWord,
Pointer,
// AArray, // isn't tested (merge with Pointer?)
//EDX,EAX
QWord,
DArray, // isn't tested
// Hidden pointer
Hidden_Pointer, // for returning large (>8 bytes) structs, not tested yet
}
struct Param
{
ParamType type; // type of value
void* ptr; // pointer to value
}
// This struct describes everything needed to make a call
struct Call
{
Param[] input; // set of input arguments
Param output; // result info
void* funcptr; // function pointer
}
// makes a call, stores result in call.ouput
void makeDynamicCall(Call* call)
{
switch (call.output.type) {
case ParamType.Void:
_makeCall!(void)(call);
break;
case ParamType.Pointer:
makeCall!(void*)(call);
break;
case ParamType.Byte:
makeCall!(byte)(call);
break;
case ParamType.Word:
makeCall!(short)(call);
break;
case ParamType.DWord:
makeCall!(int)(call);
break;
case ParamType.QWord:
makeCall!(long)(call);
break;
case ParamType.Float:
makeCall!(float)(call);
break;
case ParamType.Double:
makeCall!(double)(call);
break;
case ParamType.Real:
makeCall!(real)(call);
break;
}
}
// helper function to save some typing
void makeCall(T)(Call* call)
{
*cast(T*)call.output.ptr = _makeCall!(T)(call);
}
T _makeCall(T)(Call* call)
{
void* funcptr = call.funcptr;
void* argptr;
int numArgs = call.input.length;
if (numArgs != 0) { // this check is needed because last parameter is
passed in EAX (if possible)
Param* param = call.input.ptr;
// iterate over first numArgs-1 arguments
for ( ; --numArgs; ++param) {
/*
// the following doesn't work for some reason (compiles but lead to
wrong result in run-time)
// would be so much more elegant!
push!(arg)(param);
/*/
argptr = param.ptr;
switch (param.type) {
case ParamType.Byte: // push byte
arg(*cast(byte*)argptr);
break;
case ParamType.Word: // push word
arg(*cast(short*)argptr);
break;
case ParamType.Pointer:
case ParamType.DWord: // push dword
arg(*cast(int*)argptr);
break;
case ParamType.QWord: // push qword
arg(*cast(long*)argptr);
break;
case ParamType.Float: // push float
arg(*cast(float*)argptr);
break;
case ParamType.Double: // push double
arg(*cast(double*)argptr);
break;
case ParamType.Real: // push real
arg(*cast(real*)argptr);
break;
}
//*/
}
// same as above but passes in EAX if possible
/*
push!(lastArg)(param);
/*/
argptr = param.ptr;
switch (param.type) {
case ParamType.Byte:
lastArg(*cast(byte*)argptr);
break;
case ParamType.Word:
lastArg(*cast(short*)argptr);
break;
case ParamType.Pointer:
case ParamType.DWord:
lastArg(*cast(int*)argptr);
break;
case ParamType.QWord:
lastArg(*cast(long*)argptr);
break;
case ParamType.Float:
lastArg(*cast(float*)argptr);
break;
case ParamType.Double:
lastArg(*cast(double*)argptr);
case ParamType.Real:
lastArg(*cast(real*)argptr);
}
//*/
}
asm {
// call it!
call funcptr;
}
}
// A helper function that pushes an argument to stack in a type-safe manner
// extern (System) is used so that argument isn't passed via EAX
// does it work the same way in Linux? Or Linux uses __cdecl?
// There must be other way to pass all the arguments on stack, but this
one works well so far
// Beautiful, isn't it?
extern (System) void arg(T)(T arg)
{
asm {
naked;
ret;
}
}
// A helper function that pushes an argument to stack in a type-safe manner
// Allowed to pass argumet via EAX (that's why it's extern (D))
void lastArg(T)(T arg)
{
asm {
naked;
ret;
}
}
// Compare it to my older implementation:
/+
T _makeCall(T)(Call* call)
{
void* funcptr = call.funcptr;
void* argptr;
int i = call.input.length;
int eax = -1;
foreach (ref param; call.input) {
--i;
argptr = param.ptr;
switch (param.type) {
case ParamType.Byte:
// passing word
asm {
mov EDX, argptr;
mov AL, byte ptr[EDX];
}
if (i != 0) {
asm {
push EAX;
}
} else {
asm {
mov eax, EAX;
}
}
break;
case ParamType.Word:
// passing word
asm {
mov EDX, argptr;
mov AX, word ptr[EDX];
}
if (i != 0) {
asm {
push EAX;
}
} else {
asm {
mov eax, EAX;
}
}
break;
case ParamType.Pointer:
case ParamType.DWord:
// passing word
asm {
mov EDX, argptr;
mov EAX, dword ptr[EDX];
}
if (i != 0) {
asm {
push EAX;
}
} else {
asm {
mov eax, EAX;
}
}
break;
case ParamType.QWord:
// pushing word
asm {
mov EDX, argptr;
mov EAX, dword ptr[EDX+4];
push EAX;
mov EAX, dword ptr[EDX];
push EAX;
}
break;
case ParamType.Float:
// pushing float
asm {
sub ESP, 4;
mov EAX, dword ptr[argptr];
fld dword ptr[EAX];
fstp dword ptr[ESP];
}
break;
case ParamType.Double:
// pushing double
asm {
sub ESP, 8;
mov EAX, qword ptr[argptr];
fld qword ptr[EAX];
fstp qword ptr[ESP];
}
break;
}
}
asm {
mov EAX, eax;
call funcptr;
}
}
+/
// I was trying to move out common code to a separate function, but
failed. It doesn't work for reasons unknown to me
/+
void push(alias fun)(Param* param)
{
switch (param.type) {
case ParamType.Byte:
fun(*cast(byte*)param.ptr);
break;
case ParamType.Word:
fun(*cast(short*)param.ptr);
break;
case ParamType.Pointer:
case ParamType.DWord:
fun(*cast(int*)param.ptr);
break;
case ParamType.QWord:
fun(*cast(long*)param.ptr);
break;
case ParamType.Float:
fun(*cast(float*)param.ptr);
break;
case ParamType.Double:
fun(*cast(double*)param.ptr);
break;
case ParamType.Real:
fun(*cast(real*)param.ptr);
break;
}
}
+/
// Convenient templates to map from type T to corresponding ParamType enum
element
template isStructSize(T, int size)
{
enum isStructSize = is (T == struct) && T.sizeof == size;
}
template ParamTypeFromT(T) if (is (T == byte) || is (T == ubyte) || is (T
== char) || isStructSize!(T, 1))
{
alias ParamType.Byte ParamTypeFromT;
}
template ParamTypeFromT(T) if (is (T == short) || is (T == ushort) || is
(T == wchar) || isStructSize!(T, 2))
{
alias ParamType.Word ParamTypeFromT;
}
template ParamTypeFromT(T) if (is (T == int) || is (T == uint) || is (T ==
dchar) || isStructSize!(T, 4))
{
alias ParamType.DWord ParamTypeFromT;
}
template ParamTypeFromT(T) if (is (T == long) || is (T == ulong) ||
isStructSize!(T, 8) || is (T == delegate))
{
alias ParamType.QWord ParamTypeFromT;
}
template ParamTypeFromT(T) if (is (T == float))
{
alias ParamType.Float ParamTypeFromT;
}
template ParamTypeFromT(T) if (is (T == double))
{
alias ParamType.Double ParamTypeFromT;
}
template ParamTypeFromT(T) if (is (T == real))
{
alias ParamType.Real ParamTypeFromT;
}
template ParamTypeFromT(T) if (is (T == void))
{
alias ParamType.Void ParamTypeFromT;
}
template ParamTypeFromT(T) if (isPointer!(T))
{
alias ParamType.Pointer ParamTypeFromT;
}
Test case:
import DynamicCall;
import std.stdio;
Param createParam(T)(T value)
{
//T* ptr = new T; // BUG! Doesn't work for delegate type
T* ptr = cast(T*)(new void[T.sizeof]).ptr;
*ptr = value;
Param param;
param.type = ParamTypeFromT!(T);
param.ptr = cast(void*)ptr;
return param;
}
struct b1
{
char c;
}
struct b2
{
wchar w;
}
struct b4
{
dchar d;
}
real foo(byte b, short s, int i, long l, float f, double d, real r, b1 c,
b2 w, b4 d2, int function() func, int delegate() dg, int* ptr)
{
writeln(b + s + i + l + f + d + r + c.c + w.w + d2.d + func() + dg() +
*ptr);
return 13;
}
int func()
{
return 42;
}
void main()
{
int dg()
{
return 14;
}
Call call;
call.funcptr = &foo;
int* i = new int;
*i = 128;
call.input ~= createParam!(byte)(cast(byte)1); // BUG! cast is mandatory
to compile
call.input ~= createParam!(short)(200);
call.input ~= createParam!(int)(4);
call.input ~= createParam!(long)(cast(long)8);
call.input ~= createParam!(float)(16.0f);
call.input ~= createParam!(double)(32.0);
call.input ~= createParam!(real)(64.0);
call.input ~= createParam!(b1)(b1(1));
call.input ~= createParam!(b2)(b2(2));
call.input ~= createParam!(b4)(b4(4));
call.input ~= createParam!(int function())(&func);
call.input ~= createParam!(int delegate())(&dg);
call.input ~= createParam!(int*)(i);
call.output.type = ParamType.Real;
call.output.ptr = new real;
makeDynamicCall(&call);
writeln(*cast(real*)call.output.ptr);
}
-------------- next part --------------
A non-text attachment was scrubbed...
Name: DynamicCall.d
Type: application/octet-stream
Size: 7813 bytes
Desc: not available
URL: <http://lists.puremagic.com/pipermail/digitalmars-d/attachments/20091121/d3ab5e10/attachment.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: DynamicCall_test.d
Type: application/octet-stream
Size: 1505 bytes
Desc: not available
URL: <http://lists.puremagic.com/pipermail/digitalmars-d/attachments/20091121/d3ab5e10/attachment-0001.obj>
More information about the Digitalmars-d
mailing list