Friday, October 31, 2014

Kamailio 4.2 Tips: #10 - Lightweight retransmission detection

UDP is still most common transport used for VoIP (hopefully that will change soon -- for proper security and privacy, TLS must be used), but by its nature of unreliable transport layer, applications must retransmit SIP packets over UDP if the next hop is not sending a response in very short time (typically, the first retransmission is done after 500ms, if no response was received). On mobile networks or congested paths, retransmissions happen quite often. The ability to detect quickly and handle them efficiently can improve the performances on SIP server side.

With the old versions, the retransmission of a request was handled by tm module, creating the transaction, then same requests were matching the transaction and processed appropriately. As early the transaction was created during the config file execution, as quick the retransmission was detected.

But a transaction is quite big structure (has space to store incoming request, its outgoing branches as well as relevant responses), creates timer tasks and uses shared memory, the last two require synchronised access, thus having impact on performances. Also, in many cases, requests are replied locally (e.g., destination user is not online, the request has to be authenticated), but the decision to reply can take a bit (e.g., a database query has to be performed for authentication).

Creating the transaction and destroying it quickly, because the request was not forwarded, doesn't sound the most optimal approach. Processing a retransmitted request till it is about to relay it stateful and see at that moment it is a retransmission, it is also affecting performances (e.g., extra database queries are done).

Kamailio 4.2 introduces a lightweight layer, which can be used between receiving a request and creating the transaction, for detecting and handling retransmissions. TMX module can keep track of requests processed at a moment by all SIP worker processes. Instead of a full transaction structure in memory for each request, it uses a very small structure per worker process in memory, keeping only the attributes needed to match the retransmission -- again, it is not a structure per SIP request, but per SIP worker process, given that at the end of configuration file execution, the worker will create the transaction if the request is relayed.

To benefit of this feature, you have to use t_precheck_trans() function. Default kamailio.cfg for v4.2 includes it -- the handling of retransmission is done by:

    # handle retransmissions
    if(t_precheck_trans()) {
        t_check_trans();
        exit;
    }
    t_check_trans();

The logic behind the above config snippet:

  • if the request is handled by another process (t_precheck_trans() returns true), then use t_check_trans() to send the appropriate reply in the eventuality the transaction was created by that process and exit
  • if the request is not handled by the another process (t_precheck_trans() returns false), then use t_check_trans() because the transaction might have been created before

Within a configuration file that creates other large structures for calls, such as dialog, this feature becomes even more useful.

Enjoy it!

Wednesday, October 22, 2014

Kamailio 4.2 Tips: #9 - Asynchronous Route Execution - DNS

I got dragged back in the asynchronous loop by several folks here at AstriCon, so again another tip for it.

Kamailio had a lot of asynchronous processing options for many years (including the transport layer TCP /TLS). Kamailio 4.2 brought in more options or simplified some of the existing alternatives. See also some of the previous posts in this series.

The subject of this post is asynchronous config route block execution. Technically it is about delegating immediate execution of a group of actions from configuration file to another process than the SIP worker and can be used for various needs, not tied to something specific. For that reason, the example is about doing sort of asynchronous DNS handling (dedicated to Torrey, developer of sipt module, following the discussion after AstriDevCon day, at this moment busy with the hackatron).

There is a lot of SIP traffic that doesn't require DNS -- for example: traffic terminated on the server, like REGISTER, presence requests, requests within dialog, calls between users registered locally (location service). Likely to have an hostname as next hop address are: requests sent to foreign SIP providers or calls sent to PSTN termination providers. Depending on the type of the service, there can be more or less percentage of SIP requests that need to do DNS lookup. However, DNS query can take a while, so it is an obvious benefit of avoiding such operation in a SIP worker.

How to do it: first, the number for asynchronous workers has to be set via global parameter: async_workers. Then, the async module provides a new config function: async_task_route("rname").

The missing piece would be how to decide whether forwarding of the message involves DNS or not. That is simple to do with:
  • $nh(d) - variable returning the next hop domain
  • is_ip(...) - function exported by ipops module which can detect if an address is IP or not
Next are the relevant config snippets:

request_route {
...
    if(!is_ip("$nh(d)")) {
        if(!async_task_route("DNSRELAY")) {
            send_reply("500", "Server error");
        }
        exit;
    }
...
}

route[DNSRELAY] {
    if (!t_relay()) {
        t_reply("500", "Server error");
    }
    exit;
}


Enjoy it!


Tuesday, October 21, 2014

Kamailio 4.2 Tips: #8 - Asynchronous Accounting to Database

One of frequent database operations done by Kamailio is inserting accounting records. Some of database servers or additional software (sql proxy) offer solutions to speed up such operations, but typically they come with limitations. Such mechanism would INSERT DELAYED offered by MySQL, but it doesn't work for InnoDB tables.

Kamailio 4.2 introduces the option to do the accounting to database using the asynchronous framework from core, provided that database connector module has support for it. This feature can be enabled by setting the acc module parameter db_insert_mode to 2.

async_workers=4
...
modparam("acc", "db_insert_mode", 1)

At this moment, db_mysql module is able to do asynchronous queries, so it has to be used for accounting. Expect other database drivers to implement this feature in the near future. If db_insert_mode is 2 and database connector module doesn't support asynchronous query, then normal insert is performed.

Enjoy it!

Monday, October 20, 2014

Kamailio 4.2 Tips: #7 - Increment CSeq for Proxy Authentication Done by Server

Using uac module, Kamailio was able to do authentication to the next hop for quite some time. For example, this could be needed when sending traffic to a PSTN termination provider which require to authenticate by username and password.

But there was a limitation: CSeq was not incremented for the INVITE sent with credentials - which was making authentication not-fully RFC compliant. The authentication worked if the other party was using Kamailio (or SER/OpenSER).

The dialog module in Kamailio 4.2 has now the capability of detecting that the authentication was done by Kamailio (via uac_auth() from uac module) and increment the CSeq for the INVITE with credentials header. Subsequent messages in the same dialog will get the CSeq value adjusted accordingly.

This feature has to be enable by setting dialog module parameter 'track_cseq_updates':

modparam("dialog", "track_cseq_updates", 1)

Note that the call (starting with initial INVITE) has to be tracked with dialog module (i.e., use dlg_manage() for requests belonging to that call).

Enjoy it!

Sunday, October 19, 2014

Kamailio 4.2 Tips: #6 - Multi-part body management

It was not very common to have SIP messages with multi-part body, but in context of SIP-I/T or instant messaging and presence, or even emergency services, such bodies can occur.

Kamailio 4.2 is a big step forward to simplify the management of multi-part body from configuration file. Textops module introduced three dedicated functions:
  • set_body_multipart( [txt, content_type] [, boundary])
  • append_body_part(txt, content_type [,  content_disposition])
  • remove_body_part(content_type)
All together, they give the option to convert to a multi-part body, set or add a part and remove a part from the body.

Here are some examples:

append_body_part("7e Od 04 55 75 69 20 4d 61 6b 65 43 61 6c 6c",
        "application/vnd.cirpack.isdn-ext",
        "signal;handling=required");

Resulting content in the body is:

...
Content-Type: application/vnd.cirpack.isdn-ext
Content-Disposition: signal;handling=required

7e Od 04 55 75 69 20 4d 61 6b 65 43 61 6c 6c

--unique-boundary-1
...

Removing some part, can be done with:

remove_body_part("application/vnd.cirpack.isdn-ext");

Enjoy it!

Saturday, October 18, 2014

Kamailio 4.2 Tips: #5 - Read configuration file from standard input

Kamailio is reading configuration file by default from Linux standard paths:

  • /etc/kamailio/kamailio.cfg - when installed via packages (e.g., deb, rpm)
  • /usr/local/etc/kamailio/kamailio.cfg - when installed from souces
A custom path to a configuration file can be given using -f command line parameter:

kamailio -f /path/to/kamailio.cfg

Actually it doesn't even matter the file name, can be anything, not only kamailio.cfg. As a side note, with this option, you can run multiple instances of Kamailio on same box, from same installation file, just be sure you have configs with different listen socket (e.g., different IP address or port or protocol).

Kamailio 4.2 adds the option to read the configuration file from standard input: the value for -f parameter has to be - (dash line). It is very unlikely than one would like to type the configuration file via keyboard, but can be useful to pipe the output of another application to Kamailio. That application has to print to terminal (standard output) a Kamailio configuration file) to Kamailio.

This can be useful for:
  • keep configuration files on a centralided system and get it only when starting Kamailio -- could increase the security by not storing locally a file that has credentials to connect to database
  • write an application that embeds kamailio.cfg and prints it based on a key -- the key can be used also for encrypting the embedded kamailio.cfg, increasing the security
For example, next is the command to get latest kamailio.cfg from Kamailio github.com repository with wget and start Kamailio without storing the configuration file locally:

wget -qO- --no-check-certificate \
   https://raw.githubusercontent.com/kamailio/kamailio/master/etc/kamailio.cfg \
   | kamailio -f -

Note: the command expects Kamailio to be installed from source code (i.e., with modules in /usr/local/lib/kamailio/modules).

Enjoy it!

Friday, October 17, 2014

Kamailio 4.2 Tips: #4 - Asynchronous SQL Query

SQLOPS is the module giving the tools to perform SQL queries to different databases from configuration file and have the result in variables. But in many cases the result doesn't matter, think about pushing statistics to database in order to have a web application rendering some graphs or displaying various reports.

With older versions, you could use a combination of mqueue, rtimer and sqlops to do kind of asynchronous database operations.

Now, Kamailio 4.2 has a new function - sql_query_async(...) - that delegates the database operation to be done in one of the processes from async pool (new in 4.2 as well). The SIP worker process is immediately ready to process next task in the config or handle next packet from the network. The config file is simpler and performances are higher.

modparam("sqlops","sqlcon","ca=>mysql://username:password@dbhost/dbname")
...
sql_query_async("ca","INSERT INTO regstats(username, time) VALUES('$au','$Ts'");

Enjoy it!



Thursday, October 16, 2014

Kamailio v4.2.0 Released

October 16, 2014: Kamailio v4.2.0 is out –  a new major release, collecting new features and improvements added during about eight months of development and one and a half month of testing.
In short, this major release brings 10 new modules and enhancements to more than 60 existing modules, plus components of the core and internal libraries. Detailed release notes are available at:
Development for next major release, 4.3.0 (expected to be out around mid of 2015) has started already. Watch the project’s webside closely for further updates and news about evolution of Kamailio.
Enjoy SIP routing in a secure, flexible and easier way with Kamailio v4.2.0!
Thank you for flying Kamailio!

Wednesday, October 15, 2014

Kamailio 4.2 Tips: #3 - Send arbitrary data packet from config

Ever needed to send a custom 'notification' when some event happened while routing SIP packets?

UAC module provided for many releases the mechanism of sending new SIP requests at any point in configuration file, but sometimes SIP headers can be unnecessary overhead (but of course, that is balanced by all the other benefits of SIP, like flexible routing, re-using Kamailio as receiver, a.s.o.).

Anyhow, for simplified needs and not only, with version 4.2.0, corex module introduces send_data(uri, content) config function. The URI is still a SIP address, allowing in this way to choose any of supported transport layers (UDP, TCP, TLS or SCTP). The content, while is limited to text format if specified in config file, can be structured the way it fits the best (e.g., xml, json, line-oriented).

Here are some examples:

send_data("sip:myserver.com:4545;transport=sctp",
      "{\"caller\":\"$fU\",\"callee\":\"$rU\",\"time\":$Ts}");

send_data("sip:myserver.com:4545", "$fU,$rU,");

send_data("sip:1.2.3.4:4545;transport=tcp",
      "Caller: $fU\nCallee: $rU\nTime: $ts\n\n");

Enjoy it!

Kamailio 4.2 Tips: #2 - Hash table iterator

Htable module is among most popular ones when coming to building flexible and customised routing policies in kamailio.cfg. So far the access to the content of an hash table was possible by knowing the key of the item.

Starting with Kamailio 4.2.0, you have the possibility to walk through htable content via an iterator, accessing the key and value for all items. Three new functions (to initialise the iterator, go to next item and end iteration) along with two new script variables (to retrieve the key and the value for current item) give the tools for using this functionality. Next snippet presents how to print the items (key, value) for htable h1:

sht_iterator_start("i1", "h1");
while(sht_iterator_next("i1")) {
    xlog("h1[$shtitkey(i1)] is: $shtitval(i1)\n");
}
sht_iterator_end("i1");

More details about this feature can be found in the readme of htable module:


Enjoy it!

Tuesday, October 14, 2014

Kamailio 4.2 Tips: #1 - Log message prefix

Quite long time since I haven't posted here on technical subjects related to Kamailio. Being just around the corner of a new major version release, 4.2.0, I am going to have a small series of short posts to reveal some of the new tiny bits that can improve a lot your experience managing Kamailio.

Tip #1 - global parameter: log_prefix

It can be used to specify the text to be prefixed to each log message printed by Kamailio while processing the SIP message.

An immediate benefit could be for log messages added in kamailio.cfg via xlog module functions. If you want to match the log message with packets from a network trace, you have to print several details from SIP packet. For example, such list of details can be:
  • sip message type
  • source ip
  • sequence number
  • sip method type
  • call-id
In addition, the xlog message would have some extra text to describing its role.

With older versions, that was possible by listing those elements in each xlog, like:

xlog("$mt $si $cs $rm $ci - extra text one\n");
xlog("$mt $si $cs $rm $ci - extra text two\n");

With 4.2, it can be done like:

log_prefix="$mt $si $cs $rm $ci - "
...
xlog("extra text one\n");
xlog("extra text two\n");

An useful effect is that log messages printed from inside the C code will be prefixed as well. Many of the error messages in C code don't print any details of the SIP message processed at that moment, now you can overcome it without touching the C code.

As a bonus, remember that you can have different log level for core and each of loaded modules by using debugger module and setting the appropriate mod_level parameter, even in v4.1.x.

Enjoy it!

Wednesday, October 8, 2014

Kamailio Git Branch 4.2 Created

The 4.2 branch on Kamailio GIT repository  has been created. This is the official branch to be used for releasing any of the 4.2.x versions. The first one, v4.2.0, is expected to be released in one week.
The version used now in master branch is prefixed with 4.3.0, which is expected to be the next future major release. From this moment,  developers are free again to commit new features and changes to the master branch, code that will be part of the 4.3.0 major release.
Branch 4.2 is an official stable branch and will only be the target for bug fixing and documentation improvements commits. If you want to install Kamailio from branch 4.2, follow the guidelines at:
Report any issue to .
Your feedback is very appreciated and you will help getting Kamailio v4.2.0 as stable as possible at the time of release.