Continuous integration isn’t something that many people in the Asterisk project think about but it is a critical part of the development of Asterisk. It provides assurances on changes that go in and allows us to find problems faster, usually before a release occurs.
The first introduction someone may have to our continuous integration is the following post on a change they submitted for review:
This is a comment posted by Jenkins, the software we use for continuous integration. It listens for events from Gerrit to know when things happen (such as posting a change for review) and triggers itself. The initial jobs that run on a change are called checks. They do a basic check on the change to make sure it is okay. For Asterisk this involves building it under 32-bit and 64-bit (some people accidentally write code that only builds on 64-bit) and running the unit tests. Unit tests cover internal Asterisk operations and interfaces. If this fails then the job in question will have a FAILURE result instead of SUCCESS. If a FAILURE result occurs further investigation needs to be done to see what failed. Full logs are available by clicking on the link for the job. The same credentials you use for Gerrit will work for Jenkins.
In the case of investigating a build failure you will want to look at the Console Output which can be found by clicking “Console Output” on the left hand menu. In the case of a unit test failure you will want to look at the “Test Result” list at the bottom.
Once a change passes the check jobs it is reviewed. If changes are done the check jobs are executed again on the new change. After multiple people have reviewed the change then someone will mark the change with a +2. This is an indication to Jenkins (by an event again) that the change is ready to be merged.
The merge time jobs are more complex. They run a subset of the actual Asterisk testsuite which runs functional tests. These cover PJSIP, SIP, ARI, IAX2, Local channels, and external MWI support. This takes about an hour. If we were to run the full testsuite on the change it would take over 3 hours. When these jobs are done a comment like the following is posted:
In this case all of the jobs passed and immediately after Jenkins automatically merged the change. Much like the check jobs if a failure occurs then the job will have a status of FAILURE and further information can be obtained by clicking on the link for the job.
For investigating gates you will want to look at the Build Artifacts.
These are the log files from the Asterisk instances that were used to run the test, logs from the testsuite itself, and the configuration used to build Asterisk. Clicking the “Build Artifacts” link itself will allow you to browse the directory structure.
As well as jobs that are executed on individual changes we also have scheduled jobs that run nightly. Since a full testsuite run takes some time and changes are generally against multiple branches we opt to run the full testsuite nightly instead. This means that it can take a bit longer to find more complex problems uncovered by other tests but they are generally uncovered overnight. These are referred to as periodics and can be examined on the Jenkins server itself.
If you’re interested in how the continuous integration system is configured including the scripts that run it we keep it all in the infrastructure project on Gerrit and accept changes against it.
