[phobos] periodic test failures + auto-testing

[Jonathan M Davis]

On Sunday, December 18, 2011 20:59:11 Brad Roberts wrote:
> Since I started the pull tester, there's essentially always a build running.
>  That's not a problem.  What is a problem is the tests that fail for
> reasons that have nothing to do with the changes and non-deterministically.
>  The most common example, that I've seen so far, is this one:
> 
> [2011-Dec-18 18:07:48] [2011-Dec-18 18:07:53] [2011-Dec-18 18:07:52.9008638]
> [-4 secs, -900 ms, -863 μs, and -8 hnsecs] [99 ms, 136 μs, and 2 hnsecs]
> core.exception.AssertError at std.file(778): unittest failure
> 
> It's shown up 22 times in the last 3 days.  So far, it's been exclusive to
> the freebsd/64 run.  An example:
> http://d.puremagic.com/test-results/pull.ghtml?runid=2654
> 
> Has anyone seen this and looked into it at all?

*Sigh* It's trying to verify that the functions which get the various types of 
file time are correct, and unfortunately, there's no way to test the times 
_exactly_, so the tests have some leeway in them. I thought that we'd managed 
to make the leeway big enough, but it looks like it's still not enough in some 
cases. Maybe the the process ends up sleeping too long or there's something 
about the machine that makes it vary mroe. I don't know, but I guess that I'll 
have to increase the leeway again.

Still, it's a bit weird that the access time and the modification time aren't 
the same - assuming that the access time isn't actually being set when the file 
is accessed (and that's usually disabled). And if they were different, I'd have 
expected the _access_ time to be newer, not the file modification time. So, 
maybe something else is going on. I'll poke at it a bit tomorrow evening, 
though I suspect that I'll just end up increasing the leeway.

- Jonathan M Davis