<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html; charset=UTF-8" http-equiv="Content-Type">
<title></title>
</head>
<body bgcolor="#ffffff" text="#000000">
I've started thinking about how to make ParallelFuture jive with D's
new threading model, since it was designed before shared and
std.concurrency were implemented and is basically designed around
default sharing. (core.thread takes a non-shared delegate, and allows
you to completely bypass the shared system, and from what I remember of
newsgroup discussions, this isn't going to change.) <br>
<br>
I've re-read the concurrency chapter in TDPL and I'm still trying to
understand what the model actually is for shared data. For example,
the following compiles and, IIUC shouldn't:<br>
<tt><br>
shared real foo;<br>
<br>
void main() {<br>
foo++;<br>
}</tt><br>
<br>
I guess my high-level question that I'm <b>still</b> not quite getting
is "What is shared besides a piece of syntactic salt to make it harder
to inadvertently share data across threads?"<br>
<br>
Secondly, my parallel foreach loop implementation relies on sharing the
current stack frame and anything reachable from it across threads. For
example:<br>
<tt><br>
void main() {<br>
auto pool = new TaskPool;<br>
uint[] nums = fillNums();<br>
uint modBy = getSomeOtherNum();<br>
<br>
foreach(num; pool.parallel(nums)) {<br>
if(isPrime(num % modBy)) {<br>
writeln("Found prime number: ", num % modBy);<br>
} <br>
}<br>
}</tt><br>
<br>
Allowing stuff like this is personally useful to me, but if the idea is
that we have no implicit sharing across threads, then I don't see how
something like this can be implemented. When you call a parallel
foreach loop like this, **everything** on the current stack frame is
**transitively** shared. Doing anything else would require a complete
redesign of the library. Is calling pool.parallel enough of an
explicit asking for "here be dragons" that the delegate should simply
be cast to shared? If not, does anyone see any other reasonable way to
do parallel foreach?<br>
<br>
On 7/31/2010 7:31 AM, Andrei Alexandrescu wrote:
<blockquote cite="mid:B4EE6B53-87A3-4FC9-90AF-CAD9477164B8@erdani.com"
type="cite">
<div>Hello,</div>
<div><br>
</div>
<div>Here's a belated answer to your question (hectic times prevented
me from tending to non-urgent email).</div>
<div><br>
</div>
<div>I think a parallel library would be great to have as indeed
phobos is geared at general concurrency. Such a lib would also expose
bugs and weaknesses in our model and its implementation. </div>
<div><br>
</div>
<div>Andrei<br>
<br>
Sent by shouting through my showerhead.</div>
<div><br>
On May 30, 2010, at 12:54 PM, David Simcha <<a moz-do-not-send="true"
href="mailto:dsimcha@gmail.com">dsimcha@gmail.com</a>> wrote:<br>
<br>
</div>
<blockquote type="cite">
<div>I have a few questions/comments about the possible inclusion
of a library for parallelism in Phobos:<br>
<br>
1. What is the status of std.concurrency? It's in the source tree,
but it's not in the documentation or the changelogs. It appears to
have been checked in quietly ~3 months ago, and I just noticed now.<br>
<br>
2. From reading the description of std.concurrency in TDPL it seemed
more geared toward concurrency (i.e. making stuff appear to be
happening simultaneously, useful for things like GUIs and servers)
rather than parallelism (i.e. the use of multiple CPU cores to increase
throughput, useful for things like scientific computing and video
encoding). It seems fairly difficult (though I haven't tried yet) to
write code that's designed for pull-out-all-stops maximal performance
on a multicore machine, especially since immutability is somewhat of a
straight jacket. I find implicit sharing and the use of small
synchronized blocks or atomic ops to be very useful in writing parallel
programs.<br>
<br>
3. Most code where parallelism, as opposed to concurrency, is the goal
(at least most that I write) is parallelized in one or two small,
performance critical sections, and the rest is written serially.
Therefore, it's easy to reason about things and safety isn't as
important as the case of concurrency-oriented multithreading over large
sections of code.<br>
<br>
4. I've been eating my own dogfood for awhile on my ParallelFuture
library. (<a moz-do-not-send="true"
href="http://cis.jhu.edu/%7Edsimcha/parallelFuture.html">http://cis.jhu.edu/~dsimcha/parallelFuture.html</a>;
<a moz-do-not-send="true"
href="http://dsource.org/projects/scrapple/browser/trunk/parallelFuture/parallelFuture.d">http://dsource.org/projects/scrapple/browser/trunk/parallelFuture/parallelFuture.d</a>)
It's geared toward throughput-oriented parallelism on multicore
machines, not concurrency for GUIs, servers, etc. and is higher level
than std.concurrency. Is there any interest in including something
like this in Phobos? If so, would we try to make it fit into the
explicit-sharing-only model, or treat it as an alternative method of
multithreading geared towards pull-out-all-stops parallelism on
multicore computers?<br>
<br>
One last note: Walter claimed a while back on the NG that
Parallelfuture doesn't compile. I use it regularly and it compiles for
me. Walter, can you please point out what the issue was?<br>
</div>
</blockquote>
<blockquote type="cite">
<div><span>_______________________________________________</span><br>
<span>phobos mailing list</span><br>
<span><a moz-do-not-send="true" href="mailto:phobos@puremagic.com">phobos@puremagic.com</a></span><br>
<span><a moz-do-not-send="true"
href="http://lists.puremagic.com/mailman/listinfo/phobos">http://lists.puremagic.com/mailman/listinfo/phobos</a></span></div>
</blockquote>
<pre wrap="">
<fieldset class="mimeAttachmentHeader"></fieldset>
_______________________________________________
phobos mailing list
<a class="moz-txt-link-abbreviated" href="mailto:phobos@puremagic.com">phobos@puremagic.com</a>
<a class="moz-txt-link-freetext" href="http://lists.puremagic.com/mailman/listinfo/phobos">http://lists.puremagic.com/mailman/listinfo/phobos</a></pre>
</blockquote>
<br>
</body>
</html>