Multithreaded file IO?

[Jerry Quinn]

Jonathan M Davis Wrote:

> On Friday, September 23, 2011 23:01:17 Jerry Quinn wrote:
> 
> A direct rewrite would involve using shared and synchronized (either on the 
> class or a synchronized block around the code that you want to lock). However, 
> the more idiomatic way to do it would be to use std.concurrency and have the 
> threads pass messages to each other using send and receive.

I'm trying the direct rewrite but having problems with shared and synchronized.

class queue {
  File file;
  this(string infile) {
    file.open(infile);
  }
  synchronized void put(string s) {
    file.writeln(s);
  }
}

queue.d(10): Error: template std.stdio.File.writeln(S...) does not match any function template declaration
queue.d(10): Error: template std.stdio.File.writeln(S...) cannot deduce template function from argument types !()(string)

Remove the synchronized and it compiles fine with 2.055.


> So, what you'd probably do is spawn 3 threads from the main thread. One would 
> read the file and send the data to another thread. That second thread would 
> process the data, then it would send it to the third thread, which would write 
> it to disk.

I think that would become messy when you have multiple processing threads.  The reader and writer would have to handshake with all the processors.


> Unfortunately, I'm not aware of any good code examples of this sort of thing 
> online. TDPL has some good examples, but obviously you'd have to have the book 
> to read it. Given some time, I could probably cook up an example, but I don't 
> have anything on hand.

std.parallelism actually looks the closest to what I want.  Not sure if I can make it work easily though.