Codec negotiation in Asterisk has been one of its deepest darkest secrets.  It’s been around since the beginning and over the past two decades it’s grown and mutated into one of the least understood parts of Asterisk.  With Advanced Codec Negotiation that’s about to change!  One of the Asterisk team’s goals for 2020 was to dig into the black art of codec negotiation, understand how it’s currently implemented, create a plan for improvement, get the improvements into Asterisk 18, and finally, document the results.

Areas for Improvement:

• Process Control: Today you can specify a list of codecs that can be used by an endpoint but that’s about it.  We need ways to specify how we use that list when examining an incoming SDP offer, creating an outgoing offer, examining an incoming answer and creating an outgoing answer.  Which list do we prefer? The configured one or the list from the SDP?  What operation do we perform on the lists?  Merge?  Use only common?   Should we allow transcoding?, etc.
• Answer Feedback: When an SDP answer is received on an outgoing call leg, we don’t have a way to communicate that back to the incoming call leg.  When Alice calls Bob and Bob sends back his SDP answer, it’ll probably have a subset of the codecs offered to him or maybe just one.  When we send the answer to Alice however, we include all the codecs we got on her incoming offer and in her endpoint’s allow list.  If Alice then starts sending media using a codec that wasn’t on Bob’s answer, we’re going to have to force transcoding.  That’s not an ideal solution.
• Code Organization: The code to perform the current process is spread out over several modules including app_dial, chan_pjsip, res_pjsip_session, res_pjsip_sdp_rtp, etc.  It’s also duplicated such that a typical incoming call would actually try to find compatible codecs two or more times.

Where do we start?

Before jumping into implementation, we really needed to identify what kind of control we wanted over the process.  First, we determined that we’ll always have two sets of codecs to operate on.  For instance, on an incoming call we have the list from the SDP offer and the “allow” list from the endpoint.  On an incoming answer, we have the list from the offer we sent and the list from the answer received.  We then determined that there are 4 control points where we can operate on those two lists:

• SDP Offers: 
  • Right before the incoming call channel driver sends the call to the core/dialplan.
  • Right after the outgoing channel driver receives the call from the core/dialplan.
• SDP Answers:
  • Right before the outgoing channel driver passes the answered call back to the core/dialplan.
  • Right after the incoming channel driver receives the answered call from the core/dialplan.

Before we can perform any operations on the lists, we need to know which one is preferred.  For instance, on an incoming call, do we prefer the list from the SDP offer, or the list configured on the endpoint?

• Pending:  This is the list usually found on the SDPs.
• Configured: This is the list usually configured on the endpoint.

Next we identified the operations we can perform on the two lists at each of the control points:

• Intersect:  Only use the codecs that appear in both lists with those from the “preferred” list appearing first.
• Merge:  Use all the codecs from both lists with those from the “preferred” list appearing first.
• Only Preferred:  Only use the codecs from the preferred list.
• Only Non Preferred: Only use the codecs from the non-preferred list.

Now we need to decide what codecs from the resulting list to keep:

• Keep All:  Keep all the codecs in the resulting list.
• Keep First:  Keep only the first codec in the resulting list and toss the rest.

Finally, transcoding:

• Allow: Allow this call to use transcoding.
• Prevent: Prevent this call from using transcoding.

With all of that decided, it was time to do the…

Implementation:

Because the impact for this project is pretty large, we broke it up into several pieces:

• Analysis:  Before we wrote any new code, we really needed to understand the current flow through all of the modules.  To help with this, we implemented a new analysis tool called Scope Tracing.  This allowed us to instrument the existing code and produce a “call graph” of the current process.  The result was pretty scary. 🙂  This is a generic facility and has been committed to all Asterisk branches except 13.
• Create the core negotiation functionality:  The code that performs the operations listed above resides in the core “streams” module.  This made better sense than implementing it in the channel driver.  We also made changes to app_dial and the bridging code to pass answers from outgoing call leg back to the incoming call leg.
• Create the configuration:  We then created the configuration framework for the pjsip channel driver endpoint options.  Here’s an example of the new endpoint options:

codec_prefs_incoming_offer = prefer: configured, operation: intersect, keep: all, transcode: prevent
codec_prefs_outgoing_answer = prefer: pending, operation: intersect, keep: first, transcode: prevent
codec_prefs_incoming_answer = prefer: pending, operation: intersect, keep: first, transcode: prevent
codec_prefs_outgoing_offer = prefer: pending, operation: intersect, keep: all, transcode: prevent

• PJSIP channel driver implementation:  This is actually the hardest part because of the multiple modules involved.  Most of the new code is in chan_pjsip.c but a good part of the work was just stripping out the old convoluted code from the other modules while making sure we didn’t break anything.

Status:

Much of the work is done but the PJSIP channel driver implementation still needs more work before we’re comfortable with it.  There are two reasons for this…  First, we have to make absolutely sure we didn’t miss anything or break anything and we’re not there yet.  Second, in the beginning of August, we got reports that conference bridges using SFU video were not generating correct SDPs on re-INVITES, or not generating any re-INVITES, when a participant joined or left the bridge.  Since the creation of SDPs and re-INVITEs are a critical part of the codec negotiation process, we needed to sort out those issues.  We got the SDP and re-INVITE issues sorted but not in time for the PJSIP channel driver implementation to make it into the 18.0.0 release.  We’re shooting for 18.1 or 18.2 for the first usable implementation.

More Info:

There’s more info on the PJSIP Advanced Codec Negotiation page on the Asterisk Wiki.