The other thing is, how many people are really going to use readln(char[] buf) directly instead of using the version that returns a newly allocated string or using byLine()? If they're using the version that returns a newly allocated string, this is a non-issue. If they're using byLine(), then a lot of the logic for recycling buffers could be moved there, where it is safer because it is well-encapsulated.<br>
<br><div class="gmail_quote">On Thu, Feb 4, 2010 at 12:02 PM, Andrei Alexandrescu <span dir="ltr"><<a href="mailto:andrei@erdani.com">andrei@erdani.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
So it seems like due to the low-level APIs that readlnImpl calls, it flies under the radar of Steve's detection.<br>
<br>
I think using an Appender with the useExistingBuffer primitive would work, but would be suboptimal - each read can only use as many characters as read the last time, which would trigger quite a few allocation (whenever the line size increases). Please let me know whether this assessment is correct.<br>
<br>
A possible solution would be to cache the last buffer and its maximum length in static variables inside readlnImpl. I think it's reasonable to say that readln is the owner of the char[] and you shouldn't take portions of it and expect they won't be changed. However, it will not trump over existing arrays.<br>
<br>
Andrei<br>
<br>
David Simcha wrote:<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"><div><div></div><div class="h5">
<br>
<br>
On Thu, Feb 4, 2010 at 12:10 AM, Andrei Alexandrescu <<a href="mailto:andrei@erdani.com" target="_blank">andrei@erdani.com</a> <mailto:<a href="mailto:andrei@erdani.com" target="_blank">andrei@erdani.com</a>>> wrote:<br>
<br>
David Simcha wrote:<br>
<br>
I recently filed bug 3673<br>
(<a href="http://d.puremagic.com/issues/show_bug.cgi?id=3763" target="_blank">http://d.puremagic.com/issues/show_bug.cgi?id=3763</a>) and started<br>
looking at possible fixes. This is a bug in<br>
std.stdio.readlnImpl(). However, the more I think about it the<br>
more I think the function needs to be completely rethought, not<br>
just patched. I'm not sure if this is a high priority given<br>
that it has nothing to do with the language spec and TDPL but<br>
it's a pretty embarrassing quality of implementation issue.<br>
<br>
Anyhow, readlnImpl() takes a ref char[] and tries to recycle the<br>
memory to read in another line. In doing so, it queries<br>
GC.capacity for the relevant memory block and resizes the array<br>
to the size of the memory block. This is arguably unsafe in the<br>
general case because, if someone passes in a slice that starts<br>
at the beginning of a GC block to use as the buffer, everything<br>
else in the same GC block can get overwritten. This massively<br>
violates the principle of least surprise. On the other hand,<br>
when encapsulated in the higher level API of byLine(), it's both<br>
safe and efficient.<br>
<br>
<br>
Could you please give an example?<br>
<br>
import std.stdio;<br>
<br>
void main() {<br>
auto writer = File("foo.txt", "wb");<br>
foreach(i; 0..1000) {<br>
writer.write('a');<br>
}<br>
writer.writeln();<br>
writer.close();<br>
<br>
auto chars = new char[500];<br>
chars[] = 'b';<br>
auto buf = chars[0..100];<br>
<br>
auto reader = File("foo.txt", "rb");<br>
reader.readln(buf);<br>
writeln(chars[$ - 1]); // a<br>
assert(chars[$ - 1] == 'b'); // FAILS.<br>
}<br>
<br>
<br></div></div>
------------------------------------------------------------------------<div class="im"><br>
<br>
_______________________________________________<br>
phobos mailing list<br>
<a href="mailto:phobos@puremagic.com" target="_blank">phobos@puremagic.com</a><br>
<a href="http://lists.puremagic.com/mailman/listinfo/phobos" target="_blank">http://lists.puremagic.com/mailman/listinfo/phobos</a><br>
</div></blockquote><div><div></div><div class="h5">
_______________________________________________<br>
phobos mailing list<br>
<a href="mailto:phobos@puremagic.com" target="_blank">phobos@puremagic.com</a><br>
<a href="http://lists.puremagic.com/mailman/listinfo/phobos" target="_blank">http://lists.puremagic.com/mailman/listinfo/phobos</a><br>
</div></div></blockquote></div><br>