Trying to reproduce Node.js async waterfall pattern.. (Meta-programming)

Christian Beaumont via Digitalmars-d-learn digitalmars-d-learn at puremagic.com
Mon Jun 23 12:39:55 PDT 2014


Hi,

I just started learning D, and thought I'd throw myself in at the 
deep end with some meta-programming, trying to write the 
equivalent of the commonly used, async waterfall, and also, 
because I'd like to use it...

If you aren't familiar with it, waterfall is a function that is 
passed a sequence of functions as its arguments that are to be 
executed in order... (it's a pattern for async programming).

Here is an example in Node.js...

waterfall(
   function(asyncCallback){
     asyncCallback(null, "one");
   },
   function(lastResult, asyncCallback){
     // lastResult equals "one"
     asyncCallback(null, "two");
   },
   function(lastResult, asyncCallback){
     // lastResult equals "two"
     asyncCallback(null, "done");
   }
   ,
   // final callback
   function (error, finalResult)
   {
      // result equals "done"
   }
);

Each function is given a callback, that when called, steps the 
waterfall forward on to the next function to process.  If an 
error is passed to the callback (instead of null), then the 
waterfall stops processing and calls the final callback at the 
end of the chain.

This is how I have it implemented so far, but as you can see, 
there are some issues...

import std.stdio;
import std.conv;
import std.exception;

alias Callback = void delegate(Exception error, string result);
alias AsyncFunc = void function(Callback cb);

static Waterfall(TLastResult, Funcs...)(Funcs funcs, Callback 
finalCallback, TLastResult lastResult)
{
	static if (funcs.length)
	{
		auto cb = (Exception error, string result)
		{
			if (error is null)
				Waterfall(funcs[1 .. $], finalCallback, result);
			else
				finalCallback(error, result);
		};

		funcs[0](cb);
	}
	else
		finalCallback(null, lastResult);
}

static Waterfall(Funcs...)(Funcs funcs, Callback finalCallback)
{
	static if (funcs.length)
	{
		auto cb = (Exception error, string result)
		{
			if (error is null)
				Waterfall(funcs[1 .. $], finalCallback, result);
			else
				finalCallback(error, result);
		};

		funcs[0](cb);
	}
	else
		finalCallback(null, lastResult);
}

void main()
{
	Waterfall(
		(Callback cb) { writeln("fn0"); cb(null, "one"); },
		(Callback cb) { writeln("fn1"); cb(null, "two"); },
		// (Callback cb) { writeln("fnError"); cb(new Exception("Bad 
joojoo"), "two"); },
		(Callback cb) { writeln("fn2"); cb(null, "done"); },
		(Exception error, string result)
		{
			if (error !is null)
				writeln("Error = " ~ error.to!string);

			if (result !is null)
				writeln("Result = " ~ result.to!string);
		}
	);
}

1) I can't see any way to get the compiler to deduce the type of 
"Funcs...".  I had an urge to somehow specialize the variadic 
"Funcs..." but I couldn't figure out any syntax to do that.  
Well, because of that, I have to repeatedly say (Callback cb) 
instead of (cb).

2) I can't see a path to flow the output type of the previous 
callback to the input "TLastResult" for the next... so it's stuck 
on "string" :(

3) I had to use a specialization to deal with the "head" case; 
(the first function doesn't have an input from a previous 
result). Minor, but niggly.

Any input on how to proceed would be great!

thanks!
Christian

BTW, you can read more about async-waterfall here...
https://www.npmjs.org/package/async-waterfall


More information about the Digitalmars-d-learn mailing list