[phobos] readlnImpl (Again)

[Steve Schveighoffer]

This is a tricky situation.  I don't think assumeSafeAppend is the right call.  Imagine something like this:

char[] data = new char[50];
data[40..45] = "hello";

f.readln(data[0..40]);

now, if the line extends past 40 characters, it will overwrite data[40..45].

What I think the safe thing to do is to not use appending until the buffer is exhausted.  However, even in that case, I don't think append is the right move.  Appending one char at a time, while maybe faster when the buffer can append in place, is still extremely slow compared to just setting a char in an array.  A better solution is to simply increment the length of the buffer when needed.  A handy function from the runtime would extend an array to the full capacity it could contain.

A quick (uncompiled) version of the relevant part:

                uint used = 0;
                char[4] encbuf;
                uint enclen;
                for (int c; (c = FGETWC(fp)) != -1; )
                {
                    if ((c & ~0x7F) == 0)
                    {
                        enclen = 1;
                        encbuf[0] = cast(char)c;
                    }
                    else
                    {
                        enclen = std.utf.encode(encbuf, cast(dchar)c);
                    }
                    if(used + enclen > buf.length)
                    {
                        /* this should be a runtime function */
                        buf.length = used + enclen;
                        buf.length = buf.capacity;
                    }
                    buf[used..used+enclen] = encbuf[0..enclen];
                    if (c == terminator)
                        break;
                }
                if (ferror(fps))
                    StdioException();
                return used;


This is where a D-specific buffering solution would be extremely useful.  Instead of using FGETC, you could simply gain access to the buffer, and search for the terminator using std.algorithm.  Tango uses this kind of thing.

-Steve


>
>From: David Simcha <dsimcha at gmail.com>
>To: Discuss the phobos library for D <phobos at puremagic.com>
>Sent: Mon, April 19, 2010 11:30:43 AM
>Subject: [phobos] readlnImpl (Again)
>
>I've begun to realize that Steve's array stomping patch breaks some assumptions that std.stdio.readlnImpl made about how array appending works, leading to it being slower than molasses again.  Should I just put some assumeSafeAppend() calls in there to make it behave as it used to, even though the old behavior was unsafe, or do we want to completely redesign this code now that we want to guarantee no stomping?
>


      
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.puremagic.com/pipermail/phobos/attachments/20100419/26b2a7c1/attachment-0001.html>