[Issue 11783] Make std.datetime unittesting faster

[d-bugmail at puremagic.com]

https://d.puremagic.com/issues/show_bug.cgi?id=11783



--- Comment #8 from Andrei Alexandrescu <andrei at erdani.com> 2013-12-22 03:56:22 PST ---
(In reply to comment #7)
> There are lots of corner cases in date-time calculations, so without being
> thorough, bugs can easily go unnoticed where the code works in almost all cases
> but not with a particular date or time. So, the tests try and be very thorough.
> And yes, that cost adds up. But I really don't see any way around it without
> risking bugs. The odds of such bugs are much lower now that the code has
> already been written and tested, but any refactoring that occurs would risk
> reintroducing bugs, and without thorough tests, the odds are much higher that a
> corner case won't be tested well enough, and bugs will creep in. For instance,
> at one point in the development process of std.datetime, there was a bug
> relating to years in B.C. that ended with 99, and it just so happened that I
> hadn't tested such a year, so I hadn't caught the bug. It's frustratingly easy
> to have a particular date that doesn't work when almost every other date does,
> so while I agree that the running time for std.datetime's unit tests needs to
> be reduced, I'm very leery about reducing them and would be inclined to find
> ways to make them more efficient without reducing how much is actually being
> tested (or worst case, make it so that some of the tests are disabled normally,
> though I don't particularly like that idea). I'm sure that improvements could
> be made in that area, but I haven't had time to spend on it any time recently.
> 
> The problem with the exception tests is that they take far, far more time than
> the other tests, so if you have many of them, the time balloons quite quickly,
> whereas you have to add a lot more other tests to get the same kind of increase
> in execution time. So, in order to keep their cost low, you're forced to do far
> fewer tests for the failure cases, which is very frustrating. I don't know what
> percentage of the execution time the exception tests currently are and am
> certainly not trying to claim that the std.datetime unit tests take as long as
> they do simply because exceptions are too slow in D, but due to how expensive
> they are, they do make the situation worse, and it's quite possible that a
> poorly chosen loop which had assertThrown is adding several seconds to the
> execution time. It could still very well be the case that almost all of the
> time is spent on other tests. I pointed it out because it is a known area which
> can cause an inordinately large increase in the excecution time and thus could
> be low hanging fruit, not because I was trying to claim that std.datetime's
> tests are slow simply due to bug# 9584.
> 
> And worst case, if std.datetime's unit tests are too big a problem for you, you
> can comment out a bunch of them or remove the line which sets the version
> testStdDateTime, which would disable many of the tests (many of them are still
> enabled via that since I haven't finished removing it yet). It probably won't
> affect much until someone actually makes changes to std.datetime, which doesn't
> happen very often. So, while I think that that's a bad strategy long term, it
> could be a quick way to reduce how much of a problem they're causing until I
> have the chance to refactor them in a month or two.

Jonathan, I seem to have difficulty getting my point across and the longer the
responses become the larger the disconnect.

Your three long paragraphs seem to convey the following points, one per
paragraph:

1. Dates and times are special in ways that make them impossible to test in
less than an ungodly long amount of time.

2. Again, it's the exceptions that make things so slow; there's no way to
improve timing without cutting down on essential tests.

3. It seems I am the only person who has a problem with a 38 seconds test on
modest functionality. Furthermore, the engineering solution proposed is, again,
to remove some tests with the note that that actually is a bad thing.

They fail to discuss or even so much acknowledge the simple matter that
std.datetime has its own grave issues. I very much disagree with this view and
the general attitude that the problem is in various other places than simply
inside std.datetime and the way its unittests are engineered. If one were to
take at face value these increasingly long replies, the only logical conclusion
one could draw is that std.datetime is perfectly engineered and the only way to
make its unittests any faster is to improve the compiler implementation.

-- 
Configure issuemail: https://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------