Looking for Asterisk CPU Usage Differences between two VM's with same SO, HW and Asterisk configuration

Asterisk Asterisk Hardware
1

Dear Community,

Hi!
I’ve been having some CPU consumption differences related to Asterisk service in two “identical” virtualized machines (PVE VM’s)

Both instances were “raised” as Clones of a SO Template where Asterisk has /etc files and realtime configuration for pjsip driver exactly the same for both VMs,

rpm -qa shows the same amount of installation packages

Asterisk CLI shows the “same” (IP Differences ofc) information when using “Show” commands for every command

I’ve tried exporting thread debugging but I’m not very sure about the process that i did.

Any ideas where to look differences for?
Anything will be useful at this point so do not hesitate to advice something

Warm Regards

2

What is the actual difference of CPU consumption between machines?

3

Its about 1:3 difference,
“Normal” asterisk instace has 10% and the high consumption one has about 25-45%

4

They are both just sitting there idlily? Are they running services that are exactly the same?

5

I have the exactly same amount of services and open ports per VM

Using:

systemctl list-units --type=service --state=running

And

netstat -tulnp

6

That wasn’t my question at all. I want to know what they are doing when you are seeing this. Are they just idling? Are they in production? I can setup three different servers on the same hardware, same configs and put them in three different locations. They will all use CPU differently because they have different use patterns.

7

Oh… Sorry
Those two VMs are in a productive environment and have the same amount (or almost the same) of calls.
50/50 load share.

Hope you get the idea
Thanks for replying
Kind Regards

8

And one thing that i’ve noticed is that,
The more calls the servers have, the bigger the difference
Not only 1:3 CPU usage ratio between servers, but 1:4

9

Right. More calls means more processing. So a server with higher call volumes will see more CPU usage. A busier server will always consume more resources.

10

I think the OP is saying the offered load is the same for both and that the 1:3 is not the result of a higher offered load on one, and that the discrepancy increases as the offered load is increased.

I can’t work out the hosting environment, but, if different hosts are involved, there may well be significant differences as the result of the other work on the machines, e.g there may be thrashing of some levels of the processor memory cache on one host.

11

are they running on the same VM host
if not try swap them around to see if it is VM host related

12

witch CPU parameter differ from the two instances? system, user, wait, load, steal (or other)? have you gatered some info from iostat, vmstat?

13

Does Vmware show the same info in the performance info as the os is reflecting?

Do you have memory ballooning occurring?

Make sure to have the latest vmware tools.

14

Exactly,
Currently i have two PVE environments with the same hardware,
Ill try to check what you stated:

“there may well be significant differences as the result of the other work on the machines, e.g there may be thrashing of some levels of the processor memory cache on one host.”

Thanks!

15

Thanks for your reply Mark,
I’ll try to debug at VM host level
Kind Regards

16

Dear @sioux1977 thanks for your reply,
I haven’t checked those parameters yet,
After debugging PVE Hosts ill check them
Thanks for the advices
Kind Regards

17

I have Proxmox environment on both VM instances, with same Hardware but different VMs,
As David said, it could be that the VMs are affecting the CPU
Warm Regards

18

As an update,
I’ve seen almost 50/50 call traffic, and a ratio from 1:4 to 1:6 in CPU interrupts via:

mpstat

Someone knows how to debug this?
Kind Regards

19

So at a basic level look for high %iowait, and %steal time. If you have lots of IOwait time that means the cpu is spending time on IO. (IO can be Disk, memory, network time) and Steal time is the time that is being allocated to other vms that should have been to this VM in its time slice.

20

If you find a different in iowait, you can install iotop and run that to get a picture of whats consuming disk time.

But vmstat will also help on cpu metrics. mpstat has a lot information but when 2 vms should be same do they look the same at a more high level. or sar with the 10 min. avg. does usage look simular over the day in resources or are they different?