With the release of Fedora 22, it's about time I published the contributor stats for Fedora 22! We'll be taking a look at 3 aspects of the efforts of the QA team for Alpha, Beta and Final: Updates Testing, Validation Tests and Bug Reports. These aren't complete stats for everything QA does, just three facets that are easy to measure.
Updates Testing
Fedora is comprised of thousands of packages, each of these being updated constantly. Each of these updates needs to be tested for regressions before being released into the stable Fedora repositories. Our system, Bodhi, helps to facilitate this testing by letting users leave comments confirming an update works as intended.
Test period: Fedora 22 Alpha (2015-02-10 - 2015-03-10)
Testers: 63
Comments: 234
| Name | Updates commented |
|---|---|
| Carlos (Morel-Riquelme) (iddnna) | 63 |
| Hans Müller (cairo) | 30 |
| Parag Nemade (pnemade) | 16 |
| yuwata | 13 |
| Adam Williamson (adamwill) | 10 |
| Christopher Meng (cicku) | 10 |
| Carlos (Morel-Riquelme) (ipomoeba) | 8 |
| Alexander Kurtakov (akurtakov) | 5 |
| Lukas Brabec (lbrabec) | 4 |
| Thomas Spura (tomspur) | 4 |
| nonamedotc | 4 |
| flo | 3 |
| Arthur Scott Poore (spoore) | 3 |
| Peter Smith (smithp) | 3 |
| ...and also 48 other reporters who created less than 3 reports each, but 58 reports combined! | |
We saw a slight drop in the number of testers between Fedora 21 and Fedora 22 from 108 to 63. Not to worry though, Alpha typically has fewer testers than any of the later milestones. Again, Carlos led the stats with 63 comments posted, followed by Hans and Parag with 30 and 16 respectively.
Validation Tests
Validation testing is done to ensure the milestone (Alpha, Beta and Final) releases conform to the Release Criteria. We currently track these results in the Fedora Wiki. The stats below reflect all the results reported against our validation test cases.
Test period: Fedora 22 Alpha
Testers: 20
Reports: 424
Unique referenced bugs: 18
| Name | Reports submitted | Referenced bugs1 |
|---|---|---|
| satellit | 84 | 1030708 1187725 1192688 1195899 1197135 1199364 1199460 (7) |
| pwhalen | 56 | 1183807 (1) |
| pschindl | 37 | 1196676 (1) |
| adamwill | 36 | 1192688 1194682 1197135 1199242 (4) |
| roshi | 31 | 1199398 (1) |
| kushal | 25 | |
| boblfoot | 17 | 1139380 1144010 1185136 1197412 1200613 (5) |
| nonamedotc | 15 | |
| sgallagh | 10 | |
| cmurf | 8 | |
| lbrabec | 6 | |
| lnie | 6 | |
| mccann2 | 5 | |
| robatino | 5 | |
| jreznik | 4 | 1199399 (1) |
| kevin | 3 | |
| kinokoio | 2 | |
| zuser | 1 | |
| prabhakarpujeri | 1 |
Coverage for this release was very close to what it was for Fedora 21. We had 3 less testers and almost the same amount of reported results. This time around, however, Satellit was able to edge out Pwhalen by 28 reports. Pschindl moved up to 3rd from his 6th place slot for Fedora 21. Great work guys!
Bug Reports
This section is fairly straight forward - how many bugs were filed before the Alpha release?
Test period: Fedora 22 Alpha (2015-02-10 - 2015-03-10)
Reporters: 155
New reports: 413
| Name | Reports submitted | Excess reports | Accepted blockers |
|---|---|---|---|
| Tim Waugh | 23 | 0 (0%) | 0 |
| awilliam at redhat.com | 21 | 3 (14%) | 5 |
| Jaroslav Reznik | 18 | 0 (0%) | 0 |
| Joachim Frieben | 16 | 1 (6%) | 1 |
| lejeczek | 14 | 2 (14%) | 0 |
| Rex Dieter | 12 | 0 (0%) | 0 |
| Lubomir Rintel | 10 | 1 (10%) | 0 |
| Chris Murphy | 9 | 1 (11%) | 1 |
| Carlos (Morel-Riquelme) | 9 | 2 (22%) | 0 |
| Peter Robinson | 9 | 0 (0%) | 0 |
| satellitgo at gmail.com | 9 | 1 (11%) | 0 |
| Stephen Gallagher | 8 | 0 (0%) | 2 |
| Márcio | 8 | 0 (0%) | 0 |
| Ralf Corsepius | 8 | 0 (0%) | 0 |
| Joachim Backes | 7 | 0 (0%) | 0 |
| Conley Moorhous | 6 | 0 (0%) | 0 |
| Fabian Deutsch | 6 | 2 (33%) | 0 |
| Mikhail | 6 | 1 (16%) | 0 |
| Alexander | 5 | 0 (0%) | 0 |
| Leslie Satenstein | 4 | 2 (50%) | 0 |
| Martin Kho | 4 | 0 (0%) | 0 |
| Mike FABIAN | 4 | 1 (25%) | 0 |
| Mike Ruckman | 4 | 2 (50%) | 0 |
| Peter Laursen | 4 | 1 (25%) | 0 |
| Petr Hracek | 4 | 0 (0%) | 0 |
| Scott Talbert | 4 | 0 (0%) | 0 |
| Sylvain Réault | 4 | 0 (0%) | 0 |
| Lukas Slebodnik | 3 | 0 (0%) | 1 |
| Pavel Šimerda (pavlix) | 3 | 0 (0%) | 1 |
| Petr Schindler | 3 | 1 (33%) | 1 |
| Dennis Gilmore | 3 | 0 (0%) | 0 |
| Felipe van Schaik Willig | 3 | 0 (0%) | 0 |
| Flóki Pálsson | 3 | 0 (0%) | 0 |
| Jan Sedlák | 3 | 1 (33%) | 0 |
| Jens Petersen | 3 | 0 (0%) | 0 |
| Mamoru TASAKA | 3 | 0 (0%) | 0 |
| miguel rodriguez | 3 | 0 (0%) | 0 |
| Mikko Tiihonen | 3 | 0 (0%) | 0 |
| Orion Poplawski | 3 | 0 (0%) | 0 |
| Robert Lightfoot | 3 | 0 (0%) | 0 |
| Severin Gehwolf | 3 | 0 (0%) | 0 |
| Trevor Davenport | 3 | 0 (0%) | 0 |
| ...and also 113 other reporters who created less than 3 reports each, but 132 reports combined! | |||
For Fedora 22, we had about half the reporters and reports we did for Fedora 21. I think this is evidence we've been able to work through some of the growing pains from the Fedora.next transition. Tim Waugh headed up the pack with 23 reports with Adam and Jaroslav in a close 2nd and 3rd. Great work guys!
Conclusion
All in all, the Alpha release of Fedora 22 went pretty well. During this phase we modified our blocker process a bit, which should mean we find and fix bugs even faster for the other milestones through a release. Thanks to everyone who helped to make the release of Fedora 22 Alpha such a success! If you'd like to get involved with Fedora QA, check out our wiki page and drop by the #fedora-qa channel on Freenode, we'd love to help you get started!