[Issue 14256] Poor IO performance on 64-bit dmd 2.066 (OS X)

[via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=14256

Ivan Kazmenko <gassa at mail.ru> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|RESOLVED                    |REOPENED
         Resolution|WORKSFORME                  |---

--- Comment #11 from Ivan Kazmenko <gassa at mail.ru> ---
I've just checked with a more recent version (dmd 2.074.1), and the picture is
the same as two years ago here (Win64, file is on SSD), so reopened.

In the readln program, I allocated buffer of 1100000 bytes (the posted version
had 10000, and so failed on test 3).

The current table:

Entry                                       test1     test2     test3
Number of lines                           1000000     10000       100
Length of each line                           100     10000   1000000
------------------------------------------------------------------------
Python 2.7.5 x32:                            0.68      0.44      0.36
Python 2.7.10 x64:                           0.55      0.36      0.33
DMD 2.074.1 byLine -m32:                     0.27      0.73      1.05
DMD 2.074.1 byLine -m64:                     1.45      1.31      1.43
DMD 2.074.1 readln -m32:                     0.25      0.63      1.00
DMD 2.074.1 readln -m64:                     1.55      1.54      1.46
DMD 2.074.1 read+splitLines -m32:            0.35      0.39      0.31
DMD 2.074.1 read+splitLines -m64:            0.41      0.31      0.32

The times of 1 second or above are clearly problematic.

In Python, string storage is low-level but number of lines affects the Pythonic
part, so test1 is slower.

In D -m32, the byLine and readln versions are slower when the length of lines
grows, possibly due to reallocation when constructing a string.  I'd say 3x
slower than Python on large strings feels like too much.

In D -m64, the byLine and readln versions still take 1.3+ seconds on all tests,
more than 2x slower than Python, which is sad.

As earlier, the read+splitLines version is the fastest on all tests in both
-m32 and -m64, so speed is definitely possible, just not as out-of-the-box as
the other two versions.

Ivan Kazmenko.

--