[phobos] FreeBSD test machine

[Jonathan M Davis]

On Wednesday, September 28, 2011 11:31 Sean Kelly wrote:
> On Sep 25, 2011, at 11:45 PM, Jonathan M Davis wrote:
> > On Sunday, September 25, 2011 23:39:00 Brad Roberts wrote:
> >> Frankly, if the tests are that sensitive, the problem is the tests. A
> >> Joe Random downloader of the source should be able to expect the code
> >> to compile and past tests on a reasonably modern os. I'd argue that
> >> 8.x is modern enough (more so given that last I checked, 8.2 was the
> >> most recent 8.x release). The job of the unit tests is to test the
> >> code, not how up to date the distribution is.
> > 
> > True, but the problem is that in order to test the correctness of the
> > code, the tests must be quite exact. If the tests were Posix-only, it
> > would be relatively easy to do. However, Windows is seriously messed up
> > with regards to its DST information. It's almost always off outside of
> > the US - especially with dates that are farther back - so it becomes
> > quite difficult to test DST switches across OSes. Windows generally
> > seems to do a better job with the current year, which is why I ended up
> > testing 2011 for America/Santiago. I guess that I'll just have to see if
> > I can rework the tests a bit. Worse comes to worst, Windows will get its
> > own set of tests for DST, but I'm loathe to do that if I can avoid it.
> 
> I don't suppose you can just verify that the computations are correct for
> the timezone data provided? It shouldn't matter how out of date my input
> is. What I care about is whether the math sorts out correctly.

No. The Proper processing of that data is essentially what's being tested, so 
you'd more or less end up with code testing itself. I'm forced to give the 
exact dates and times of DST switches (as well as which direction a switch is) 
in order to test that they're handled correctly. For the most part, that's 
fine, but in order to be appropriately thorough, I need a time zone in the 
Southern and Western hemispheres which has DST, and there's a shortage of 
those. If you combine that with the fact that Windows is downright pathetic at 
DST correctness, it makes it very difficult to find a time zone which can be 
reliably tested. America/Santiago was the best that I was able to find, but 
it's been changing its DST on more or less a yearly basis of late. I might be 
able to make it work better if I don't try and test the exact same dates and 
time zones on Windows and Posix, but I'd like to avoid that if I can. For now, 
I've just commented out the tests for America/Santiago, since I know that it 
works.

An any case. Yes, in principle, it would be fantastic to be able to test this 
stuff generically without caring how accurate the time zone information on the 
system is, but since it's time zone stuff that's being tested - including the 
accuracy of the code that reads in the time zone data - there's not really any 
code which could be used to obtain the information generically, except for the 
code which is being tested. So, hard coding the information is the best way 
that I can think of dealing with it. And as long as the time zone data on a 
system is up-to-date (which they _usually_ should be), the tests should be 
fine. Unfortunately, Chile's situation is making the problem worse than it 
normally would be.

- Jonathan M Davis