stdio line-streaming revisited

[Andrei Alexandrescu (See Website For Email)]

kris wrote:
> Andrei Alexandrescu (See Website For Email) wrote:
>> kris wrote:
>>
>>> Last week there were a series of posts regarding some optimized code 
>>> within phobos streams. A question posed was, without those same 
>>> optimizations, would tango.io be slower than the improved phobos [1]
>>>
>>> As these new phobos IO functions are now available, Andrei's 
>>> "benchmark" [2] was run on both Win32 and linux to see where tango.io 
>>> could use some improvement.
>>
>> [snip]
>>
>> On my machine, Tango does 4.3 seconds and the following phobos program 
>> (with Walter's readln) does 5.4 seconds:
> 
> On Win32, the difference is very much larger. As noted before, several 
> times faster. Those benefits will likely translate to linux going forward.

If I understand things correctly, it looks like the hope is to derive 
more speed from further dropping phobos and C I/O compatibility, a path 
that I personally don't consider attractive.

Also, the fact that the tango version is "more than twice as efficient 
as the fastest C version identified" suggests a problem with the testing 
method or with the C code. Are they comparable? If you genuinely have a 
method to push bits through two times faster than the fastest C can do, 
you may want as well go ahead and patent it. Your method would speed up 
many programs, since many use C's I/O and are I/O bound. It's huge news. 
I'm not even kidding. But I doubt that that's the case.

>> Also, the Tango version has a bug. Running Tango's cat without any 
>> pipes does not read lines from the console and outputs them one by 
>> one, as it should; instead, it reads many lines and buffers them 
>> internally, echoing them only after the user has pressed end-of-file 
>> (^D on Linux), or possibly after the user has entered a large amount 
>> of data (I didn't have the patience). The system cat program and the 
>> phobos implementation correctly process each line as it was entered.
> 
> If you mean something that you've written, that could presumeably be 
> rectified by adding the isatty() test Walter had mentioned before. That 
> has not been added to tango.io since (a) it would likely make programs 
> behave differently depending on whether they were redirected or not. 
> It's not yet clear whether that is an appropriate specialization, as 
> default behaviour

What is absolutely clear is that the current version has a bug. It can't 
read a line from the user and write it back. There cannot be any 
question that that's a problem.

>, and (b) there has been no ticket issued for it
> 
> Again, please submit a ticket so we don't forget about that detail. We'd 
> be interested to hear if folk think the "isatty() test" should be 
> default behaviour, or would perhaps lead to corner-case issues instead

I was actually pointing out a larger issue: incompatibility with phobos' 
I/O and C I/O. Tango's version is now faster (thank God we got past the 
\n issue and bummer it's not the default parameter of nextLine) but it 
is incompatible with both phobos' and C's stdio. (It's possible that the 
extra speed is derived from skipping C's stdio and using read and write 
directly.) Probably you could reimplement phobos and bundle it with 
Tango to give the users the option to link phobos code with Tango code 
properly, but still C stdio compatibility is lost, and phobos code has 
access to it.


Andrei