Introduction

Packet loss can be an annoying problem when dealing with real time communication, especially when dealing with video. It’s very noticeable when the screen freezes for multiple seconds, then the footage resumes with everything in a completely different position than it was originally. We’ve all seen this before. Packet loss is inevitable, but it can be dealt with. That’s what we’re going to touch on today, with a focus on video.

What’s packet loss? How does it happen?

Most people probably know what packet loss is, but for those who don’t, packet loss occurs when a packet of data traveling over the network fails to reach its destination. This could be caused by something like bad network connectivity, which would lead to packets being occasionally dropped. In the case of real time communication and Asterisk, this can occur in different forms. Let’s take a look at a very basic overview of Asterisk’s RTP structure. A call is started between two people. There will be a RTP instance to keep track of it. Asterisk will continuously receive data (packets) from the other end. It will also send packets to the other end. If one of these packets gets lost along the way, then we’ve got packet loss. Asterisk did not have a way to deal with video packet loss previously, but there has been a lot of work done to change that. Let’s take a look at how!

Could you send that again?

There are two different sides to packet loss. In this section, let’s talk about what needs to be done when we receive a request for retransmission from the remote end. This is called negative acknowledgement, or NACK for short. But first, we need a mechanism to keep up with packets we are sending out! Any data structure that can contain the payload works. These packets are all stored by their sequence number (seqno) and are retrieved the same way. Whenever Asterisk sends a packet out to the remote end, we place it inside of this data structure first. This can’t be an ever-growing structure, however, so once it’s full, kick the oldest packet out of there. Then, when we receive a NACK request, it will provide us with information on which packets have been lost. Within this request, we can see the packet ID (pid) of the initial lost packet, followed by a bitmask (blp) of other packets after the pid that haven’t been received. The blp starts at the least significant bit. All of this information is stored in the feedback control information (FCI) section of the NACK request. There can be multiple entries in the FCI, so we need to make sure we check for multiple requests! Once we have this information, we can use our handy packet storing structure to retrieve the lost packets if they are still present, and send them back to the requesting party. That’s it for part one; let’s move on to how to detect packet loss and what to do from there!

Let’s say we receive a RTP packet, and this packet will have a seqno. The seqno for this particular packet happens to be 1000. The next packet we would expect to receive would be 1001. But maybe you’re experiencing some bad network connectivity, and you don’t get packet 1001. Maybe you don’t get packets 1002, 1003, or any packets up until 1100 either! This can create some delays in our video experience. Another data storage structure needs to be defined that will keep track of out of order packets (packets that aren’t the expected ones). We assume we are experiencing packet loss if the structure becomes too large. That’s when we send out a NACK request. In this request, there are a few important things to keep track of (see https://tools.ietf.org/html/rfc4585 for a full description). Let’s focus on the length, the synchronization source (SSRC), and the FCI sections. We need to put the length of the NACK request in here. This will include all the typical header elements, as well as a variable number of FCI fields. There are two different SSRC fields; we care about the media SSRC, which will tell the remote end which instance this NACK request is coming from. Finally, in the FCI field, we populate the pid field with the seqno of the packet we last expected to receive, followed by the blp, the bitfield of packets that were also determined to be lost. We can keep track of which packets we are missing with another data structure that stores the missing seqno inside. If this structure becomes too large, however, it’s probably not worth saving; just wipe it and start fresh! One important thing to note here is that a packet cannot be assumed to have been received. In this case, the blp would suggest that no packets after 1000 had been received. It’s also important to give the remote end time to send those packets back to us, as well as time for us to arrange those packets in our structure. Then, we take the packets that we have available and roll with it from there until we need to request another retransmission. That’s pretty much all there is to it!