Testing Asterisk SIP/IAX2 performance

Asterisk Asterisk General
1

I have a plan to compare SIP and IAX2 channel performance on Asterisk versions 1.4.x and 1.6.x. Digium bragged that they improoved the SIP performance on Asterisk 1.6 for a big margin, so I would like to test this out :smile:

So the plan is to use one PC, where Asterisk 1.4 and Asterisk 1.6 would be installed and tested (not simultaniously afcoarse). In the test I would stress the Asterisk box with couple 100 concurrent calls and monitor the CPU and Network load. For that I would also need a couple more Asterisk boxes, that would serve as call origin/destination simulators. So the setup would be something like this:

(Asterisk boxes for call originatination)------------Test Asterisk box---------------- (Asterisk boxes for call termination)

The border Asterisk boxes would be connected to the central Asterisk box via SIP or IAX2 trunk (depends on the test), all the boxes would be on the same LAN. The central Asterisk box would only do call routing and proxying of RTP packets, the border Asterisk boxes would take care of call origination and termination. I would initiate calls with a simple bash script, that would generate Call Files and I would terminate calls within Asterisk dialplan on another border Asterisk box.

I already did one test, and these are the dialplan configs that I used. An example is for a call from ext. 100 to ext. 150, where a call is initiated with a Call File on a border Asterisk box and is routed to the central Asterisk box, that routes it to another border Asterisk box, where the call is anwered.

extensions.conf context for Asterisk server where the call-origination takes place:
[call-origination]
exten => 100,1,Answer()
exten => 100,n,Wait(30)
exten => 100,n,Hangup()

extensions.conf context for Asterisk server where the call-termination takes place:
[call-termination]
exten => 150,1,Answer()
exten => 150,n,Echo()
exten => 150,n,Hangup()

All sounds OK in theory (at least to me :smile:), but in the actual test I found very low CPU consuption and low network load with multiple hundreds simulated simultanious calls. I do not know what is the exact problem, but I suspect that the Wait()/Echo() applications are not generating any network traffic (even though both legs of a SIP call are opened, no voice traffic is passing through), so the central Asterisk server is not stressed at all. Am I right? Would I need to use any other dialplan applications to simulate the actual audio stream that is generated on an actual call? Or do you have any additional proposition about my test scenario?

I did a lot of internet searching about the Asterisk performance testing and I got to the SIPp application, but that does not help me out too much, because I need a test tool that supports SIP and IAX2, so that is why I am trying to make my own custom test bed. Any help is greatelly appreciated.

PS.
I have a lot of Asterisk experience and can do a bit of programming, so don’t be afraid to be too technical for me :smile:

2

Checkout blogs.digium.com/2009/02/18/s-prize/

Ian

3

Why not use a single box running HP SSIP tool. You can easily generate 10,000 call legs and set it to ECHO mode. Have Asterisk answers the call and just loop a playback audio file and ssipp echoing this sound. I use this to test transcoding performance of 150 simultaenous calls. once the system is in full testing mode, you can dial in yourself or dial a special extension where you can RECORD the audio so that way you can demonstrate the quality to everyone that needs to QA your system under heavy load. I am sure you can script it to generate 10,000 call legs from 2nd asterisk box, but there’s no need for it in my opinion and especially 2 boxes.

4

If I am not mistaken, with SIPp I can only test the performance of the SIP channel and can not also test the performance of IAX2 channel. Am I correct?

One bad thing with running a test on a single server is, that all the network traffic is not sent to the NIC and put on the network, so the test on a single machine does not take take into consideration the load that sending traffic to other peers takes to the CPU.

Secondly, I would only like to test the Asterisk call routing performance and the max. number of concurrent calls that a system can handle, so I would not like to use any dialplan applications on the “central test Asterisk box” that do anything else then routing calls (Dial() appplication). I think that running Playback() application on the central test server is not OK, because playing a file l takes up a lot more CPU resources then handling SIP/IAX2 and RTP traffic. So I think that I would not be able to avoid some custom scripting and using multiple border Asterisk test boxes, but that is not a big problem.

So if I make a script that creates call files and I modify my extensions.conf to the following configuration, will 64k traffic be transfered in both direction on an active call?

extensions.conf context for Asterisk server where the call-origination takes place:
[call-origination]
exten => 100,1,Answer()
exten => 100,n,Playback(couple_minutes_alaw file)
exten => 100,n,Hangup()

extensions.conf context for Asterisk server where the call-termination takes place:
[call-termination]
exten => 150,1,Answer()
exten => 150,n,Echo()
exten => 150,n,Hangup()

If SIPp can also be used for testing IAX2 channel, please give me some good links, that you must have found when you were doing a research for your test setup. Thanks.

5

Howdy,

If you’re interested in performance, join the asterisk-dev IRC channel on irc.freenode.net. Russell Bryant (rbryant) and Stefan Schmidt (schmidts) have been doing a lot of work lately to improve SIP performance. Much of that work made it into 1.8.0. More work is being done.

Cheers

6

Ok, just to give you an update on this thread.

In the end I did a perfomance test comparison between the asterisk versions 1.6.16.1 and 1.8.2.2 on SIP calls (test for IAX2 calls is postponed to later time because of time constraints). I created a similar setup as I described in previous threads. In the test I had:

  • 1x central test server (registered 1000 SIP peers and handled SIP calls)
  • 2x traffic generator Asterisk server that simulated 500 SIP peers that register to the central test server
  • 1x traffic generator Asterisk server that played a 7 minute file (Answer(), Playback(), Hangup ()) - CO (call originate)
  • 1x traffic generator Asterisk server that received a call and did Echo() (Answer(),Echo(),Hangup()) - CE (call terminate)

All the traffic generators were connecting to central test server. The way I generated traffic was via Call Files, so the calls started at CO server, ran through central test server and was answered by the CE server. Because of the Playback() and Echo() applications, actual RTP traffic was generated during an active. With this setup I was able to test the load of 300 concurrent calls before the traffic generators servers started experiencing problems :smile:
In the test, I generated the wanted amount of calls sequentially in 0,5 s intervals. Each call got established and ran for 7 minutes (the duration of the sound file that Playback application was playing), after that the call dropped. I was able to monitor the resource consumption during the time that the calls were active via top and iptraf command.

So the central test server had to handle:

  • Register and qualify traffic for 1000 qualify peers (SIP REGISTER and OPTIONS messages)
  • handle SIP signalling for generated calls that run between the CO and CE server
  • handle RTP traffic for all the active calls running between the CO and CE server

The test showed that Asterisk 1.8 burns the same or more CPU time then 1.6 and a lot more memory. To be totally honest, I was expecting the opposite result. That is why I would like to research the main things that were introduced in 1.8 that might confirm the test result. Does anyone have any idea about that? Do you know where to start digging to find the cause for this?
Since a lot of resource consumption is due to RTP packet bridging, is there any changes to RTP traffic handling in 1.8 as apposed to 1.6?

7

Doesn’t anyone have any idea on how to explain bigger resource consumption on 1.8 when compared to 1.6? Can someone atleast point me in the direction to look for the cause?

Is anyone here interested in a test report of my testing? I am prepared to write a detail report and also share my call generator and sip.conf file generator scripts that others can use in their own performance test. But I do not plan to do this if there will be no interest from the community …

8

Why kind of server do you need, to handle 1000 registered extensions (using g711 and two outbound trunks).
I guess it has to handle about 1000 concurrent calls.

Thank you

9

[quote=“dejanst”]Doesn’t anyone have any idea on how to explain bigger resource consumption on 1.8 when compared to 1.6? Can someone atleast point me in the direction to look for the cause?

Is anyone here interested in a test report of my testing? I am prepared to write a detail report and also share my call generator and sip.conf file generator scripts that others can use in their own performance test. But I do not plan to do this if there will be no interest from the community …[/quote]

Howdy,

The asterisk-dev mailing list (lists.digium.com/mailman/listinfo/asterisk-dev) or the #asterisk-dev IRC channel (irc.freenode.net) are going to provide better feedback than the forums. The forums usually get more “users” kind of questions and answers.

10

Found a nice developer that helped me out and we found the problem. It’s described here:

reviewboard.asterisk.org/r/1066/

When I tried the patch, the memory consumption on 1.8.2.2 dropped by more then a half :wink:

To be totally honest, I am a bit dissapointed at the forum, since more advance topics are brushed off, since not a lot of developers read this. Even my preparation to publish an official test report and publish Call File generator and sip.conf generator Bash scripts got ignored. I know that the scripts are not complex at all, but it would save the tester atleast an hours time to write and try them out by himself…
I am actively working with Asterisk and am willing to do a lot of beta testing and debugging, but I am not totally sure if all the things/errors that I get are for putting on the bug tracker and the mailing list. So I try the forum before I go there :smile: But thanks for all the guys that are trying to spread the Asterisk good reputation here on the forums :wink:

@sebek72
Before you go off and set up a server for 1000 users, first find out about the difference between the number of registered users and the number and concurrent calls. It’s very rare that they are the same, the number of registered users is usually much much higher then the number of concurrent calls (if you are not trying to build a call center :smile:)
About the server strenght - in my testing I found that Asterisk is quite lightweight, so any mid-range server would do the job more or less. Just be sure to do a lot of testing before putting this in production :wink:

11

Thank you…

It is hard to predict concurrent calls. So I rather say 1000, but on average it would be around 300. (a lot off phones would connect over vpn). Is it possible to route internal calls from one location on internal LAN not over asterisk (phone -> VPN -> asterisk -> VPN -> phone 2) to exclude unnecessary traffic?

I was thinking a 2x quad core xeon with 100GB ram - it this reasonable sizing? The users may increase to 1200 in time.

Thank you

12

dejanst - id be very interested in your script if you were happy to make it available?

13

@sebek72
If you plan to make a big Asterisk implementation, then you need to do some performance test on your main server that you plan to use for your system. I think that your planned server spec. is a bit overkill, all depends on what else beside Asterisk you want to run on the Asterisk server (AGI scripts, Databases …)

@garnathan
Currently I do not have my test report in English language - I plan to do a translation in about a month’s time. But you are definitly welcome to use the Bash scripts I made. Please send me your test requirements on PM and let’s continoue our discussion there :wink:

14

[quote=“dejanst”]@sebek72
If you plan to make a big Asterisk implementation, then you need to do some performance test on your main server that you plan to use for your system. I think that your planned server spec. is a bit overkill, all depends on what else beside Asterisk you want to run on the Asterisk server (AGI scripts, Databases …)

I would be a plain asterisk with freepbx (CDR database). Nothing special.

15

Just to sum up my test results, I used this HP server as my test Asterisk server and it was able to run about 400 concurrent calls without any issues. I also simulated 1000 registered extensions to increase the load while loading the server with concurrent calls. The test setup was just compiled Asterisk, no databases and no advance dialplan scripting was used.

I hope that this can give you an idea of the spec. for the server that you need for your setup.

16

Thank you very much.

17

dejanst - thanks for your reply and offer. In the end i wrote a simple perl script that takes 3 parameters, a destination DDI, # of channels to test, and delay. All it does is generate as many “call” files as you require and drops them into /var/spool/asterisk/outgoing/ . It plays a 3 minute wav file down the channel. Ive found it to be perfect for what i require to test. It gives me 3 minutes to monitor the CLI, CPU and RAM loads. It can also be quite useful when trying to find bad or dead channels in a PRA. If anyone wants it, just PM me