Let’s talk some more about testing, understanding the test framework for Asterisk, and building better tests. An exciting topic I know! In a previous post, we discussed how unit testing Asterisk worked. Here, though we’ll be talking a bit about the Asterisk Test Suite.

The Asterisk Test Suite is a way to write automated, functional, “black-box” tests that exercise the interactions in and between modules, and the core. For this write-up we will be focusing on some of the more useful parts of the Test Suite framework, specifically test objects and pluggable modules.

Background

The Asterisk Test Suite started, as most things do, with a very basic framework. An implemented test just needed to have an executable file called run-test, and a test-config.yaml within its directory for it to be run. This meant, and still means today, that you can write a test in a variety of programming languages. As things progressed, and as more tests were added libraries were written in order to re-use code, tasks and functionality. This of course has made it easier and faster to write tests for the Test Suite, but has consequently somewhat limited the multi-language side of things. Most tests written today either use a fully yaml configurable backed class, or are written using Python since it is the language with the most added library support. It’s still possible to write a Test Suite test using some other programming language like Lua, or even Bash. However it is not recommended since you’d have to re-implement a lot of functionality to do so.

Test Objects

A test object is the singular main driver and entry point for a test. It’s usually responsible for setting up, initializing, and managing tasks shared by tests within an applicable context.

As tests were written it became obvious that there were a lot of common tasks tests were [re]implementing, thus the TestCase class was created. Written in Python this class provides base utilities and shared functionality for Asterisk and AMI instance management, registrable event notifications, and reactor operations. Many, if not all, current test object implementations inherit from this class. Test objects, in themselves, implement a lot of common functionality for a given test context and are often completely configurable from the test-config.yaml. Below are a few typically used test objects:

• ari.AriTestObject – manages ARI requests and event notifications.
• sipp.SIPpTestCase – manages SIPp driven test scenarios.
• test_case.SimpleTestCase –  orginates a call into the dialplan and waits for a configurable user event.
• test_case.TestCaseModule – yaml configurable TestCase  class.

Note, the above is not a full list of all currently implemented test objects.

Pluggable Modules

A pluggable module implements common functionality that gets plugged into, or injected into a test object. Where a test object is used to set up and drive the base test, a pluggable module is used to execute and/or monitor test specific behavior. Multiple pluggable modules can be used in conjunction with a test object. Below are some commonly used pluggable modules:

• ami.AMIEventModule – listens for, and matches on AMI events configured by the test.
• ari.WebSocketEventModule – listens for ARI websocket events, and calls into a configured method handler on a match.
• pluggable_modules.Originator – originates a call in Asterisk.
• pluggable_modules.HangupMonitor – monitors channels and stops the test once all matching channels have hung up.
• pluggable_modules.EventActionModule – links registered event listeners with registered action handlers.

When a test is instantiated, the test object gets created, and then each associated pluggable module is given a reference to the test object as well as the pluggable module configuration that was specified in the test-config.yaml.

EventActionModule

A pluggable module itself, the EventActionModule has further abstracted frequently used test functionality. For the most part it even supersedes several other commonly used pluggable modules. It’s used to link registered events with registered actions. Any test that monitors for a particular event (AMI, ARI, test start, etc.) and then executes some action (originate a call, raise some other event, stop the test, etc.) should consider using this module. Below is a list of current usable events:

• test-start – triggered upon test start.
• ari-start – triggered once ARI is up and running.
• ari-events – monitors and matches on ARI events.
• ami-start – triggered once AMI is up and running.
• ami-events – monitors and matches on AMI events.
• ami-restart – triggered when asterisk is fully booted and AMI reconnects.

Below is a list of current usable actions:

• logger – logs a message.
• validate-log  – checks to make sure a log file exists.
• callback – calls into a given “module.method”.
• stop-test – stops the test.
• pjsua-phone – perform a pjsua phone module action.
• ari-requests – sends a request using ARI.
• ami-actions – executes an AMI action.

Building Better Tests

A prior post already went over how to install, execute, and write a basic test for the Test Suite, so we’ll skip those details here. However, keep in mind when writing a test that a lot of the heavy lifting has already been done for you.

When writing a test first choose an appropriate test object to initialize and drive the test. Rarely should you need to write you own test object, but if you do then likely you’ll want to inherit it from TestCase, or one of the other test objects. Mostly, though you can just use an already written test object.

Next, choose one or more applicable pluggable modules for the test. Again often times there is already a module written for you to use. If not, and it makes sense to do so then write a new one. When writing a pluggable module keep in mind the class’s constructor must be similar to the following:

def __init__(self, config, test_object):

Where config is the pluggable module configuration specified in the test-config.yaml and test_object is the associated test object driver. You can find plenty implementations of pluggable modules in Test Suite source directory lib/python/asterisk/pluggable_modules.py to pattern your own after.

It also may make sense to write a new event or action for the EventsActionModule. Again, you can find working examples in lib/python/asterisk/pluggable_modules.py. Just look for classes that are registered either in the PLUGGABLE_EVENT_REGISTRY or PLUGGABLE_ACTION_REGISTRY.

Be sure to also check out the sample yaml configurations found under the sample-yaml/ Test Suite directory. Here you’ll find sample yaml configurations for both test objects and pluggable modules. Of note the phones-config.yaml.sample has a nice example of using the EventsActionModule .

Lastly, of course the Test Suite itself is full of working examples. A lot of times you can simply find a test doing something similar and use that as a base for your new test.

Concluding Remarks

For all modifications to Asterisk it is highly recommended to have some kind of automated test, or tests, written that exercise the changes in some fashion. In some cases a test is even required before the code is allowed to be submitted to the code base. Whether is it through unit testing and/or the Test Suite automated testing makes Asterisk better. More documentation on installing and running the Test Suite can be found on the Asterisk wiki.

Happy test writing!