Strange Bug in LDC vs DMD

[H. S. Teoh via Digitalmars-d-learn]

On Fri, Jun 30, 2017 at 07:57:22PM +0000, FoxyBrown via Digitalmars-d-learn wrote:
[...]
> The only way this can happen is if the rename command is somehow
> feeding back in to the algorithm. Since the problem goes away when I
> pre-compute dirEntries, it suggests that dirEntries is being lazily
> computed.

Um... the docs explicit say that dirEntries is lazy, did you not see
that?

	https://dlang.org/phobos/std_file#dirEntries

"Returns an input range of DirEntry that *lazily* iterates a given
directory, ..." [emphasis mine]


[...]
> I'm pretty sure that the analysis above is correct, that is,
> dirEntries is lazy and ends up picking up the renamed file. This is
> sort of like removing an element in an array while iterating over the
> array.

This is certainly what it looks like; however, it doesn't explain a
couple of things:

1) Why the DMD version appears to be unaffected, since as far as I can
tell from the code, it is also a lazy iteration;

2) On Linux at least, renaming a file does not move the location of the
entry in the directory, so whether dirEntries is lazy or not shouldn't
even matter in the first place.

Does Windows reorder the directory when you rename files? E.g., if you
set the folder to sort alphabetically, does it actually sort the
directory, or does it only sort the GUI output?  My guess is that the
sort order only affects the GUI output, as it would be grossly
inefficient to actually sort the directory. But you never know with
Windows...  In any case, if Windows *does* physically sort the folder,
that could explain how rename() affects dirEntries.  However, this still
doesn't explain the discrepancy between DMD and LDC.

Actually, now that I think of it... Linux may do the same thing if the
new filename is longer and doesn't fit in the old slot. So that could
explain (2).  However, why the difference between DMD and LDC?  It
doesn't make sense to me, if you tested both on the same OS.

Here's a way to rule out (2): instead of using the current working
directory, change the code to create a fresh copy of the directory each
time.  Does DMD / LDC still show a difference?  The idea here is that it
may have been a coincidence that you saw LDC having the problem and DMD
not, since whether or not a renamed file gets moved depends on how big
the current slot for its name is in the directory, and if you've already
done a bunch of operations on the directory, some slots will be bigger
and some will be smaller, so some renames will happen in-place whereas
others will cause a reordering.  It could be you just got unlucky with
LDC and caused a reordering, whereas you got lucky with DMD and the
existing slots were already big enough so the problem isn't visible.

OTOH, this still doesn't explain why calling the OS functions directly
fixes the problem.  If there is a bug in LDC's version of dirEntries
somewhere, we'd like to know about it so that we can fix it.


T

-- 
Without outlines, life would be pointless.