Ubuntu logo

Developer Summit

Improve community kernel SRU testing

2011-11-01 12:00..13:00 in Antigua 2

SRU testing is about validating that package updates don't introduce problems on stable releases. The testing includes regression testing performed by the QA team, security testing by the Security team, certification testing by the Certification team and verification by the Kernel team. There is also some testing performed by the Ubuntu community coordinated by the QA team, but this is mostly achieved on a best effort basis. The purpose of this blueprint is to increase community participation and include their successful testing rather than just bug reports.


The current process for getting community participation is by commenting on the bugs fixed by the SRU when the package is building for the proposed archive. The comment describes how to enable this archive where the package will be ready in a few hours and then asks the subscribers to please test with feedback. Another process specific for kernel packages is for development releases to be announced on voices.canonical.com and proposed kernels may also be announced eventually. As can be seen from the pending SRU reports, this has not resulted in much participation and the responsibility then falls upon the QA team.


The first objective is to increase community participation for proposed kernels. The most significant value is to increase the probability that verification testing is performed by someone in the community within a reasonable delay. As a side effect, another value is to relieve the QA team from some testing which can otherwise result in additional delays. The SRU release team would benefit from having more people testing with minimal delays to push a new kernel from proposed to updates.


The second objective is to include successful testing from the community rather than just bug reports. This will enable the SRU release team to also consider the number of people having tested the proposed kernels in addition to their existing reports. This should result in greater confidence before pushing a new kernel from proposed to updates.


This is the story to achieve both these objectives:
1. A kernel is added to the proposed archive, so the user is notified that a new kernel is available;
2. The user opens Update Manager where she is informed that the kernel can be tested after rebooting;
3. After booting for the first time with a proposed kernel, Checkbox prompts the user to run the SRU suite;
4. If a test fails, Apport is invoked and tags the bug with "regression-proposed" for the current reports;
5. After testing, all test results are submitted to Launchpad where additional reports can be generated.