Memory grows up extremely

Hi everyone,

we use asterisk in a control center with about 2000 calls per day. Additionally we have a coupling to tetra radio, configured in asterisk as confbridges interworking with clients speaking/listening to the tetra radio groups by frequently muting / unmuting the channels and confbridges per AMI.
Using this scenario rises the memory usage of asterisk about 20 GB per day until there is no more memory and asterisk restarts.
We have tested Version 13.18-cert3 LTS and 16.8-cert3 LTS. Activating Compiler Flag “MALLOC_DEBUG” helps to see the memory usage, especially frame.c gets large in Size: 436,782,887 Bytes 9533921 objects
Strangely we have in messages
[Aug 16 16:43:24] ERROR[17041] astobj2.c: Excessive refcount 100000 reached on ao2 object 0x13255f0
[Aug 16 16:43:24] ERROR[17041] astobj2.c: FRACK!, Failed assertion Excessive refcount 100000 reached on ao2 object 0x13255f0 (0)
but memory info says:
Sun Aug 16 16:44:01 CEST 2020
memory show summary

258200 bytes in 3227 allocations in file astobj2.c

I’am confused. How to find out the reasons for the upgrowing memory ?

Best regards

The easiest option is to try to reproduce it in a controlled environment and then eliminate parts to determine where the leak may be coming from. With the current state of things it’s not as easy. Are you using any third party modules?

After some tests we have got a reproducable scenario in Versions 13.21-cert6 and 16.8.cert3, where the value of frame.c increases permanently.

The following commands are sent over the AMI-Interface (commented)
ami_commands.txt (1.5 KB)

confbridge.txt (1.2 KB)

exten => _berta,1,Answer()
exten => _berta,2,Set(JITTERBUFFER(fixed)=20,500)
same => n, NoOp(${CALLERID})
same => n, Set(VOLUME(RX)=2)
same => n, Set(VOLUME(TX)=2)
same => n, ConfBridge(berta)

After sendig actionid: 1705904476_335#berta-ddab9ea6-dcb4-4e83-b9bf-ff5e44325110 asterisk increases the value of frame.c continously as displayed by memory show summary. Asterisk had been compiled with MALLOC_DEBUG and “cache_media_frames = no” in asterisk.conf
For one channel the value increases about 32 per second; for two channels the value increases about 64 per second.

Does this happen in 16-13-0 and 13-36-0? I believe Sangoma will only maintain certified versions for paying customers and only to the extent that the intended behaviour is certified.

If you are a paying customer, you should use their commercial support channels. If you want to minimise the bugs, you should use the latest sub-version that is not a release candidate.

Using 16-13-0 or 13-36-0 shows the same behaviour. I am not a paying customer. Which is the right channel for reporting the probable bug ?, using the latest sub-version, of a current version.

Note that Asterisk 13 has only just over a month’s support life left, so the chances of a fix being applied to that are small.