Strange Bug in LDC vs DMD

[FoxyBrown via Digitalmars-d-learn]

On Friday, 30 June 2017 at 17:32:33 UTC, H. S. Teoh wrote:
> On Fri, Jun 30, 2017 at 12:50:24PM +0000, FoxyBrown via 
> Digitalmars-d-learn wrote:
>> I am using dirEntries to iterate over files to rename them.
>> 
>> I am renaming them in a loop(will change but added code for 
>> testing).
>> 
>> 
>> In DMD the renaming works but in LDC the renaming fails. It 
>> fails in a way that I can't quite tell and I cannot debug 
>> because visual D is not working properly for LDC.
>> 
>> The code essentially look like the following:
>> 
>> 
>> 	auto dFiles = dirEntries(loc, mask, _mode);
>> 
>> 	foreach (d; dFiles)
>> 	{
>> 
>>            auto newName = Recompute(d.name)
>>            writeln(newName);
>>            rename(d.name, newName);
>>         }
>> 
>> but when I comment out rename, it works under LDC.
>> 
>> The funny thing is, newName is printed wrong so Recompute is 
>> effected by the rename.
>> 
>> This shouldn't occur.
> [...]
>
> This sounds very strange.  What exactly do you mean by "newName 
> is printed wrong"? Do you mean that somehow it's getting 
> affected by the *subsequent* rename()?  That would be truly 
> strange.  Or do you mean that newName doesn't match what you 
> expect Recompute to do given d.name? Perhaps you should also 
> print out d.name along with newName just to be sure?
>
> Do you have a reduced code example that's compilable/runnable?  
> It's rather hard to tell what's wrong based on your incomplete 
> snippet.
>
>
> T

No, if I simply comment out the rename line, then the writeln 
output changes. Simple as that. No other logic changes in the 
code.

This means that the rename is affecting the output. The recompute 
code gets the filename, does a computation on it, then returns 
it.. prints it out, then renames that file to the newly computed 
file name.

The only way this can happen is if the rename command is somehow 
feeding back in to the algorithm. Since the problem goes away 
when I pre-compute dirEntries, it suggests that dirEntries is 
being lazily computed. If that is the case, then the problem is 
easily understood: The file gets renamed, dirEntries reiterates 
over the file, then it gets recomputed again, but this time the 
result is bogus because it is a double recompute, which is 
meaningless in this program.

I'm pretty sure that the analysis above is correct, that is, 
dirEntries is lazy and ends up picking up the renamed file. This 
is sort of like removing an element in an array while iterating 
over the array.

The odd thing is, is that DMD does not produce the same result. I 
do not know if there is a different in the LDC vs DMD dirEntries 
code(or lazily evaluated code in general) or if it has to do with 
speed(possibly the renames are cached and do not show up 
immediately to dirEntries with the slower DMD?).

I do not have any simplified code and I'm moving on from here. It 
should be easy to mock something up. The main thing to do is to 
rename the files based on something in the file name.

e.g., suppose you have the files 1,2,3,4,5 (that is there names)

and extract and multiply the filenames by 10. (that is your 
recompute function).

You should end up with 10,20,30,40,50.

But if the cause of issue I'm describing is in fact true, one 
don't necessarily get that because some files will be iterated 
more than once. e.g., maybe 10, 100, 1000, 20, 200, 30, 40, 50, 
500.

I am doing it over a lot of files btw, but that is essentially 
what is going on.  The example above should be easy to do since 
one can simply to!int the filename and then multiply it by 10 and 
then rename that.

I have moved on to avoid dirEntries completely and simply use the 
os directory listing function manually to extract the data but 
this should be investigated as it if it the behavior is what I am 
describing, a serious bug exists somewhere. (if someone could 
confirm that dirEntries is a lazy range, then it would explain 
the problem, but not necessarily why dmd and ldc differ, (dmd 
seeming to function as expected)).