As part of the transition to Python3 at the end of last year, we introduced a Python Virtual Environment, or venv for the Asterisk TestSuite.  This was done to standardize the Python package requirements in an effort to help make both the transition and future running easier.  However, we still see that there is some confusion about what a Python venv is and why it’s worth using.

Let’s start with what a Python venv is.  The fact that it is called a Virtual Environment could lead one to think that this is using virtualization, but it’s not.  According to peps.python.org, a venv is:

[A] mechanism for lightweight “virtual environments” with their own site directories, optionally isolated from system site directories. Each virtual environment has its own Python binary (allowing creation of environments with various Python versions) and can have its own independent set of installed Python packages in its site directories, but shares the standard library with the base installed Python.

So there is no virtualization or docker style containers, but rather a local copy of the Python binary and libraries and some configuration that ties these together.  When we activate or deactivate the virtual environment we are setting or un-setting environment variables that tell Python to stay within the bounds of the venv.  The Virtual Environment is limited to the Python binary and dependencies.  When you modify the Python installation from an activated state (ie; change version or install a package via pip) it does not affect the Python installation on the base OS or for any other venvs.

So why are we using it?  The idea is to make using the Asterisk TestSuite easier.  We do this by standardizing and isolating the packages that the Asterisk TestSuite needs from the base OS.  To accomplish this, it is common practice with venvs to use a requirements.txt file to define the local Python package dependencies.  Once the venv has been created and activated, the packages can be automatically installed by pip via:

python -m pip install -r ./requirements.txt

The format of the requirements file can be found here, but we generally use two formats, one for public packages and one for local ones.   We break these up into two files, requirements.txt and extras.txt.  The requirements.txt includes entries like:

numpy>=1.19.5
pycparser==2.21

Each line defines a package name known to pip as well as a version requirement.  In general we try to use the >= operator in requirements.txt so that systems aren’t forced to upgrade unless they have to.  The extras.txt includes direct zip downloads like:

https://github.com/asterisk/starpy/archive/refs/heads/1.1.zip

As long as the .zip file contains an installable Python package, pip can install it for us.

So that’s what the venv is and why we use it.

If you’re still not convinced, you don’t actually have to deal with any of this directly.  Included in the Asterisk TestSuite is a runInVenv.sh script.  This script checks the venv, checks requirements.txt and extas.txt and both manages and activates/deactivates the venv for you.  If the venv has not been created, the runInVenv.sh script will create it and install the correct Python packages.

If there is a change to requirements.txt or extas.txt since the last time runInVenv.sh was run, it will clean the venv, re-create it and install the new packages.  If you use the runInVenv.sh script, you do not have to manually maintain the venv.

Using the runInVenv.sh script is straight forward.  If previously you were to run:

python runtests.py -t tests/channels/pjsip

You would now run:

./runInVenv.sh python runtests.py -t tests/channels/pjsip

The script will deactivate the venv upon exit, so you never have to interact with the venv directly.

If you wish to create your own bespoke tests, the venv still can be used.  If you have your own Python requirements, you can add them to the existing txt config files and the script will make sure they are installed in the venv.  Removing the venv is as simple as deleting the .venv folder that the runInVenv.sh script creates in the base Asterisk TestSuite directory.

Hopefully this helps clear up both what a Python Virtual Environment is, and why it makes running and maintaining the Asterisk TestSuite better.