Single ConfBridge stuck on large server

I have a bunch of zombied channels stuck in a ConfBridge. ConfBridge list reports 12 users in the conference, but core show channels shows about 160 channels attached to the bridge. New users cannot join the conference. Using ‘confbridge kick all’ just crashes the CLI, and individually requesting hangup on the channels doesn’t work. I can’t reboot the server.

Any help? This is racking up quite a SIP bill :frowning:

Asterisk 16.6.2

Have you tried “channel request hangup” on them?

Yes, the channels are still zombied and present

If “channel request hangup” doesn’t work then there’s nothing you can do except restart Asterisk, as something is deadlocked.

That’s what I didn’t want to hear. Any way I can help debug?

You could grab a deadlock backtrace[1]. It would show if anything is deadlocked, but it won’t solve it.

[1] https://wiki.asterisk.org/wiki/display/AST/Getting+a+Backtrace

This happened again today after restarting asterisk and is now a recurring issue. It seems that confbridge is not properly tearing down calls once they are hung up.

This is only affecting one specific ConfBridge on the entire server. Any attempt to manipulate that specific conference or the channels in it crashes the CLI. But everything else works, including new calls and conferences. I don’t know if that qualifies as a deadlock?

It could be a deadlock, it’s hard to say with certainty at face value.

No core dumps are available, (and it’s compiled with DONT_OPTIMIZE) and this is constantly recurring. I’m at a total loss of where to start.

Core dumps only occur if a crash occurs. You’d want to follow the “Getting Information For A Deadlock” section.

OK, got it, but this is above my pay grade. I defenatly see that the threads towards the bottom of the file indicate that they are waiting for exit from confbridge:

#0  0x00007fc4179444ed in __lll_lock_wait () from /lib64/libpthread.so.0
#1  0x00007fc41793fde6 in _L_lock_941 () from /lib64/libpthread.so.0
#2  0x00007fc41793fcdf in pthread_mutex_lock () from /lib64/libpthread.so.0
#3  0x000000000050d5fa in __ast_pthread_mutex_lock ()
#4  0x000000000045e254 in __ao2_lock ()
#5  0x00007fc3d3ddce76 in leave_conference () from /usr/lib64/asterisk/modules/app_confbridge.so
#6  0x00007fc3d3ddf246 in confbridge_exec () from /usr/lib64/asterisk/modules/app_confbridge.so
#7  0x000000000053bf50 in pbx_exec ()
#8  0x0000000000527a86 in pbx_extension_helper ()
#9  0x000000000052b7a8 in ast_spawn_extension ()
#10 0x000000000052c462 in __ast_pbx_run ()
#11 0x000000000052dc39 in pbx_thread ()
#12 0x00000000005bd449 in dummy_start ()
#13 0x00007fc41793ddd5 in start_thread () from /lib64/libpthread.so.0
#14 0x00007fc4169de02d in clone () from /lib64/libc.so.6

But I have no idea what to do next. I can develop within Asterisk, but I don’t know anything about C compiling and thread management.

It is waiting on the conference, the full backtrace would be needed. You can file an issue[1] but there is no time frame on when it would be resolved if there is an issue.

[1] https://issues.asterisk.org/jira

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.