|
Test-driven regression finding
|
|
Posted by: stak
Tags: mozilla
Posted on: 2012-09-16 12:58:51
I was going through the conference proceedings for ICSE 2012 to see if there was anything particularly interesting. One of the short papers I found was called "An Integrated Bug Processing Framework" and described a tool to automatically bisect code and find regressions based on bug reports. Although it was a two-page paper and very short on implementation details, it seemed like a good idea to incorporate at Mozilla.
Tracking down changesets that introduced regressions is harder to do on the Mozilla codebase than at most other places I've worked at, because of the size and complexity of the code. Finding the regressing changeset almost always involves bisecting nightlies or code, as opposed to doing a quick code inspection and hg blame. I recently had to do this for a Fennec bug, and it was rather tedious. (I actually ended up updating mozregression to work with Fennec, and submitted a pull request which is still pending, but even so, testing each bisection build for the error was annoying).
What would be awesome is if we modified our approach to fixing regressions slightly, and made it more test-driven. Instead of bisecting the regression, patching the code, and writing a test, we could write the test first, then use that in an automated bisection script to find the regressing changeset, and then patch the code. The test would also (by definition) be a test for the patch, and could be submitted to the repository along with the fix. This would make it much easier to track down the regressing changeset, and also ensure that patches have tests to go with them.
|
|
(c) Kartikaya Gupta, 2004-2024. User comments owned by their respective posters. All rights reserved.
You are accessing this website via IPv4. Consider upgrading to IPv6!
|
The bug is fixed and the testcase is checked into the testsuite to ensure that it doesn't happen again.
An example: https://bugzilla.mozilla.org/show_bug.cgi?id=783441