openMP

[David Nadlinger]

On Wednesday, 3 October 2012 at 23:02:25 UTC, dsimcha wrote:
> So the "process which creates the future" is a Task that 
> executes in a different thread than the caller?  And an 
> alternative way that a value might become available in the 
> future is e.g. if it's being retrieved from some slow I/O 
> process like a database or network?

Yes.


>> For example, let's say you are writing a function which 
>> computes a complex database query from its parameters and then 
>> submits it to your query manager/connection pool/… for 
>> asynchronous execution. You cannot use std.parallelism.Task in 
>> this case, because there is no way of expressing the process 
>> which retrieves the result as a delegate running inside a 
>> TaskPool.
>
> Ok, I'm confused here.  Why can't the process that retrieves 
> the result be expressed as a delegate running in a TaskPool or 
> a new thread?

Because you already have a system in place for managing these 
tasks, which is separate from std.parallelism. A reason for this 
could be that you are using a third-party library like libevent. 
Another could be that the type of workload requires additional 
problem knowledge of the scheduler so that different tasks don't 
tread on each others's toes (for example communicating with some 
servers via a pool of sockets, where you can handle several 
concurrent requests to different servers, but can't have two task 
read/write to the same socket at the same time, because you'd 
just send garbage).

Really, this issue is just about extensibility and/or 
flexibility. The design of std.parallelism.Task assumes that all 
values which "becomes available at some point in the future" are 
the product of a process for which a TaskPool is a suitable 
scheduler. C++ has std::future separate from std::promise, C# has 
Task vs. TaskCompletionSource, etc.


>> The second problem with std.parallelism.Task is that your only 
>> choice is polling (or blocking, for that matter). Yes, 
>> callbacks are a hairy thing to do if you can't be sure what 
>> thread they are executed on, but not having them severely 
>> limits the power of your abstraction, especially if you are 
>> dealing with non-CPU-bound tasks (as many of today's "modern" 
>> use cases are).
>
> I'm a little confused about how the callbacks would be used 
> here.
>  Is the idea that some callback would be called when the task 
> is finished?  Would it be called in the worker thread or the 
> thread that submitted the task to the pool?  Can you provide a 
> use case?

Maybe using the word "callback" was a bit misleading, but it 
callback would be invoked on the worker thread (or by whoever 
invokes the hypothetical Future.complete(<result>) method).

Probably most trivial use case would be to set a condition 
variable in it in order to implement a waitAny(Task[]) method, 
which waits until the first of a set of tasks is completed. Ever 
wanted to wait on multiple condition variables? Or used select() 
with multiple sockets? This is what I mean.

For more advanced/application-level use cases, just look at any 
use of ContinueWith in C#. std::future::then() is also proposed 
for C++, see e.g. 
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2012/n3327.pdf.

I didn't really read the the N3327 paper in detail, but from a 
brief look it seems to be a nice summary of what you might want 
to do with tasks/asynchronous results – I think you could find 
it an interesting read.

David