Tuesday, December 11, 2018

Performance Tests For KEMI Scripting Languages

Two editions ago, Kamailio (with version 5.0) introduced the scripting languages embedded interface (short name KEMI) that enabled using Lua, Python, JavaScript or other scripting languages to build SIP routing logic. During Kamailio v5.1.x cycle, this framework became more mature and started to show its benefits: besides ability to reload the routing scripts at runtime, each of these languages come with tons of extensions and offer a more flexible programming eco-system.
When preparing the latest major release of Kamailio (the v5.2.0) and the days after, I run some tests to compare the performances of using native scripting versus Lua and Python (v2). The tests were not focused on measuring the capacity of Kamailio, but to see the difference in executing similar SIP routing logic with different scripting languages.
The results and conclusions are collected in a wiki page:
It was pleasant to discover that native scripting, Lua and Python perform more or less the same, the differences are so small that can be just a side effect of what other applications are running at that moment in the system. The tests were related to processing registrations with user authentication, using MySQL as a backend.
A test consisted of 20 000 registrations for 10 users being sent at a rate of 4000 requests/second, with a limit of 10 000 at the same time. Note that each registration was challenged for authentication and resent with credentials, then authenticated (this performs a MySQL query) and replied with 200ok. The average of running the routing script ranged form 60 to 80 microseconds, most of them being around 70 microseconds (for a better understanding, that means more than 10 000 authenticated registrations per second).
The wiki page includes the sipp scenario, used configs and tools to perform the tests, therefore anyone can try to run and check the results.
A bit of a surprise was to see that Python has really good results. I ran couple of basic tests during the development of KEMI framework for Kamailio 5.0 and using Python seemed slower. Not this time, so I will have to re-run the tests just to be sure I haven’t forgotten something.
Anyhow, during the development of Kamailio 5.2, there were couple of additions to KEMI to improve the performances. Among the most relevant were to export several functions that are equivalent to some native scripting language conditions, such as matching From/To URIs against myself keyword or matching the SIP methods.
The plan is to run the tests for the other scripting languages supported by KEMI at this moment, respectively Python (v3), JavaScript, Ruby and Squirrel. It would be also interesting to see the results of using Lua scripting with LuaJIT (an addition added for Kamailio v5.2 as well).
Should you run the tests from the wiki, let us know the results for your environment via sr-users@lists.kamailio.org mailing list.
Thanks for flying Kamailio!

No comments:

Post a Comment