Roshi's Ramblings

Documenting things I document

With the release of Fedora 22, it's about time I published the contributor stats for Fedora 22! We'll be taking a look at 3 aspects of the efforts of the QA team for Alpha, Beta and Final: Updates Testing, Validation Tests and Bug Reports. These aren't complete stats for everything QA does, just three facets that are easy to measure.

Updates Testing

Fedora is comprised of thousands of packages, each of these being updated constantly. Each of these updates needs to be tested for regressions before being released into the stable Fedora repositories. Our system, Bodhi, helps to facilitate this testing by letting users leave comments confirming an update works as intended.

Test period: Fedora 22 Alpha (2015-02-10 - 2015-03-10)

Testers: 63

Comments: 234

Name Updates commented
Carlos (Morel-Riquelme) (iddnna) 63
Hans Müller (cairo) 30
Parag Nemade (pnemade) 16
yuwata 13
Adam Williamson (adamwill) 10
Christopher Meng (cicku) 10
Carlos (Morel-Riquelme) (ipomoeba) 8
Alexander Kurtakov (akurtakov) 5
Lukas Brabec (lbrabec) 4
Thomas Spura (tomspur) 4
nonamedotc 4
flo 3
Arthur Scott Poore (spoore) 3
Peter Smith (smithp) 3
...and also 48 other reporters who created less than 3 reports each, but 58 reports combined!

We saw a slight drop in the number of testers between Fedora 21 and Fedora 22 from 108 to 63. Not to worry though, Alpha typically has fewer testers than any of the later milestones. Again, Carlos led the stats with 63 comments posted, followed by Hans and Parag with 30 and 16 respectively.

Validation Tests

Validation testing is done to ensure the milestone (Alpha, Beta and Final) releases conform to the Release Criteria. We currently track these results in the Fedora Wiki. The stats below reflect all the results reported against our validation test cases.

Test period: Fedora 22 Alpha

Testers: 20

Reports: 424

Unique referenced bugs: 18

Name Reports submitted Referenced bugs1
satellit 84 1030708 1187725 1192688 1195899 1197135 1199364 1199460 (7)
pwhalen 56 1183807 (1)
pschindl 37 1196676 (1)
adamwill 36 1192688 1194682 1197135 1199242 (4)
roshi 31 1199398 (1)
kushal 25
boblfoot 17 1139380 1144010 1185136 1197412 1200613 (5)
nonamedotc 15
sgallagh 10
cmurf 8
lbrabec 6
lnie 6
mccann2 5
robatino 5
jreznik 4 1199399 (1)
kevin 3
kinokoio 2
zuser 1
prabhakarpujeri 1

Coverage for this release was very close to what it was for Fedora 21. We had 3 less testers and almost the same amount of reported results. This time around, however, Satellit was able to edge out Pwhalen by 28 reports. Pschindl moved up to 3rd from his 6th place slot for Fedora 21. Great work guys!

Bug Reports

This section is fairly straight forward - how many bugs were filed before the Alpha release?

Test period: Fedora 22 Alpha (2015-02-10 - 2015-03-10)

Reporters: 155

New reports: 413

Name Reports submitted Excess reports Accepted blockers
Tim Waugh 23 0 (0%) 0
awilliam at redhat.com 21 3 (14%) 5
Jaroslav Reznik 18 0 (0%) 0
Joachim Frieben 16 1 (6%) 1
lejeczek 14 2 (14%) 0
Rex Dieter 12 0 (0%) 0
Lubomir Rintel 10 1 (10%) 0
Chris Murphy 9 1 (11%) 1
Carlos (Morel-Riquelme) 9 2 (22%) 0
Peter Robinson 9 0 (0%) 0
satellitgo at gmail.com 9 1 (11%) 0
Stephen Gallagher 8 0 (0%) 2
Márcio 8 0 (0%) 0
Ralf Corsepius 8 0 (0%) 0
Joachim Backes 7 0 (0%) 0
Conley Moorhous 6 0 (0%) 0
Fabian Deutsch 6 2 (33%) 0
Mikhail 6 1 (16%) 0
Alexander 5 0 (0%) 0
Leslie Satenstein 4 2 (50%) 0
Martin Kho 4 0 (0%) 0
Mike FABIAN 4 1 (25%) 0
Mike Ruckman 4 2 (50%) 0
Peter Laursen 4 1 (25%) 0
Petr Hracek 4 0 (0%) 0
Scott Talbert 4 0 (0%) 0
Sylvain Réault 4 0 (0%) 0
Lukas Slebodnik 3 0 (0%) 1
Pavel Šimerda (pavlix) 3 0 (0%) 1
Petr Schindler 3 1 (33%) 1
Dennis Gilmore 3 0 (0%) 0
Felipe van Schaik Willig 3 0 (0%) 0
Flóki Pálsson 3 0 (0%) 0
Jan Sedlák 3 1 (33%) 0
Jens Petersen 3 0 (0%) 0
Mamoru TASAKA 3 0 (0%) 0
miguel rodriguez 3 0 (0%) 0
Mikko Tiihonen 3 0 (0%) 0
Orion Poplawski 3 0 (0%) 0
Robert Lightfoot 3 0 (0%) 0
Severin Gehwolf 3 0 (0%) 0
Trevor Davenport 3 0 (0%) 0
...and also 113 other reporters who created less than 3 reports each, but 132 reports combined!

For Fedora 22, we had about half the reporters and reports we did for Fedora 21. I think this is evidence we've been able to work through some of the growing pains from the Fedora.next transition. Tim Waugh headed up the pack with 23 reports with Adam and Jaroslav in a close 2nd and 3rd. Great work guys!

Conclusion

All in all, the Alpha release of Fedora 22 went pretty well. During this phase we modified our blocker process a bit, which should mean we find and fix bugs even faster for the other milestones through a release. Thanks to everyone who helped to make the release of Fedora 22 Alpha such a success! If you'd like to get involved with Fedora QA, check out our wiki page and drop by the #fedora-qa channel on Freenode, we'd love to help you get started!

comments powered by Disqus