I've been working on a concept over the past several months, which I call "adaptive capabilities". Some of you are already familiar with the original module that started this work, called cdr_adaptive_odbc. The idea behind this is rather simple, but the result is very powerful: instead of relying on database tables to be static representations of what Asterisk designers originally intended, make Asterisk adapt to the layout of a database table.

In terms of the CDR, this means that at startup time, we read in the layout of the table and cache it. Then we consider that layout when a CDR is posted. For additional columns that the designers had not thought about, we populate them if there is a CDR variable with the same name. For standard CDR variables that don't exist in the backend table, we omit them from the insert.

Additionally, several CDR backends (including ODBC and MySQL) support the idea of aliases; that is, to permit an Asterisk administrator to decide that she wants to rename a standard CDR field to something else in the backend table and have it work correctly. So no longer does the callerid field need to be named "clid" in the database -- it could be named "callerid" or even something else entirely.

So, in other words, Asterisk is now adapting to other people's database designs, rather than forcing database administrators to adapt to Asterisk designs.

What is coming up in Asterisk 1.6.1, though, is even more adaptive capability. We are adding this capability to our Asterisk Realtime Architecture. For now, you can't rename realtime columns, but what you can do is allow Asterisk to manage the layout of your realtime tables. The realtime tables are actually a bit adaptive already -- if a column that corresponds to a standard option in sip.conf, for example, does not exist in the backend table for sipusers or sippeers, or the column is set to NULL, then we assume the default value for that field name.

However, the problem of a field not existing is a bit more complicated. For querying and updating realtime, there are simply certain fields which must exist in Asterisk 1.4 Realtime, or it will not work at all. These are the fields that we either use in the WHERE clause of a SELECT or UPDATE, or in the UPDATE clause. The trouble in the past is that when either of these failed, it failed in a way that was very confusing to the Asterisk administrator. If a SELECT failed, then a record either would not be found, or, due to a failover query, the wrong record could be found. If an UPDATE failed, it usually did so silently, and the administrator was similarly confused about why data was missing from the database.

So what is changing is that, now, since the table structure is known, we will now take a special action during startup, where each realtime table in use will tell the core which columns it expects to be there. The action it takes will be one of several capabilities. I'm going to cite the PostgreSQL backend, as the functionality there is rather complete. The default choice is simply to warn the user. You will get a nice WARNING message at startup, which will also go to your logs, and you'll be able to find that message, read it and realize that Asterisk needs a column with the name "regseconds" and it should be an INT4 or equivalent (CHAR(5) would also be seen as acceptable). For fields that are used in WHERE clauses, you will also get a warning on each attempt to query the database if the column is missing.

The other special action comes in a choice of two types: "createchar" or "createclose". This is a special option that will allow Asterisk to ALTER the table, to make it look like the table that it expects to see. This means that you will get every piece of information that Asterisk expects to be able to send to the database. The difference between these two options will the type of column that Asterisk chooses to create for you. If you select "createclose", Asterisk will create the best fit column type for storing the data that it expects to find. If you select "createchar", then Asterisk will only create CHAR type columns of sufficient length.

I already hear the paranoid cry that the application should never alter the database. We agree that in the default case, it shouldn't, and it does not. This additional ability is something that the Asterisk administrator will need to consciously turn on, through an option (and the sample configs also default to the behavior off).

There is also one more important distinction to this adaptive functionality in Asterisk Realtime Architecture. If the table is not altered, and we try to update a column that does not exist, in previous version, the whole UPDATE query would fail and none of the information would be updated. Starting in 1.6.1, that is no longer the case. If a column is missing that Realtime is asked to update, then that column will be dropped from the query, so that the rest of the updates may succeed.

I am very excited about this new functionality, as it means that we will be able to add requested columns to update in the backend table, without the risk of breaking existing installations. Sure, you may get an extra warning at startup, but nothing should break realtime during an update. We've held off on a few features that meant changing the backend realtime tables, but I feel that this new functionality will ensure a good balance between those new features and existing compatibility.