Ubuntu logo

Developer Summit

Continuous Integration Kernel Testing in LAVA

2011-11-01 10:00..10:45 in Curacao 5

Supporting large scale testing of kernel trees and defconfig in the LAVA lab is a high priority engineering effort across all Linaro engineering and Landing Teams for Q4.10. This organization wide effort imposes a growing list of requirements that LAVA team has started to work on during Q3 and will focus on driving forward during the Q4.2011.

This effort involves improvements to the lava dashboard to ensure that the build and test results data can be efficiently consumed by kernel engineers improvements to the lava dashboard to make it easy to follow every step of a submitted job from the scheduler to the job submissions step. increase of capacity for all LEB board types deployed in the lava-lab to a level suitable to continuously track 20+ kernel branches with multiple defconfigs each. improvements to lava job submissions tools that make it easy for services like the kernel CI service to hook link from their build page to the scheduler job and the results.

The goal of this session is to discuss user stories around LAVA components for driving continuous integration testing and results tracking through LAVA.

Session Notes: Currently the kernel is cross built in ec2 by jenkins -- build results are submitted to the dashboard and if successful a test is run in LAVA. http://validation.linaro.org/lava-server/kernel-ci-views/index attempts to convey a summary of the results. Current process of getting your own tree is manual - talk to Deepti to get your tree included. Wishlist:

Want to able to use the set up topic trees for temporary testing.

Maybe need to make it easier for KWG devs to set up trees for themselves

We might want to use per-user configs so that users can select the trees/configs they care about

Per-user reports and a web form so they can submit git tree, branch and config for a one-time or regular testing of their tree

per-tree specification of tests to run

Determine which tests to run with the new kernel: ACTION: mwhudson to submit mp to change this

ltp

hw enablement tests

PM functional tests

Store the kernel version (uname) so that we can cross-reference this

Build results notification (by email, RSS feed)

Can just do this in Jenkins for now

Want to do this in LAVA

RSS ok for now

Weekly build-status digest email for the community?  Deepak to do this manually for now

John's builds don't appear on the tree view

Waterfall view perhaps not appropriate for one-time builds

Should use tags for grouping results (now that tags exists)

deepti to change jenkins jobs to use tags

Add tagging support to dispatcher so the results get tagged similarly

View for test/build combination

Easy way to see last successful build for a test/build

Waterfall-ish view of test suite vs test/build history

Also improve bundle view

Anti-wishlist:

Builds of out of tree configs

Basically, developer should commit the Kconfig changes to the tree

ACTION: start tagging build and test results (with LAVA tags) Figure out how our read-only tags look like (since tags cannot be changed afterwards). Mouse-over git-describe: show the shortlog