Error : Previous SIP reload not yet done

Hi expert,

Having severe problem with sip channel. Now its happening almost everyday. When I execute sip reload command I am getting the error “Previous SIP reload not yet done”.

I have tried changing some of the parameter in sip.conf but no help. Even executing core restart command also does not work. It start working only after restart asterisk server.

Any idea to resolve the issue. By the way, my asterisk running on virtual server.

Thank You

Virtual servers cause media quality problems, not this sort of problem.

My guess is that you have a mild deadlock.

You didn’t say which version, but you should ensure that you are using the most recent subversion of 1.8 or 11. If that doesn’t help, you will need to make sure Asterisk is built with optimisation disabled and thread debugging enabled. Then, when the problem presents, run “core show locks” and use gcore and gdb to get backtraces. (Googling “asterisk wiki backtrace” should lead you to more information this process.)

Thanks david. I will check your steps, hope will resolve the issue. By the way, I am using the latest version 11.2.1 .

Was there any resolution to this? I have a similar problem. I found the following deadlocks (dump is below.) There seems to be a deadlock between lock requests at pbx_lua.c line 1209 (lua_reload_extensions) and a held lock at pbx.c line 11809 (ast_rdlock_contexts).

I am trying to solve it by trying to acquire the ast_wrlock_contexts lock before trying to get the lock on the config files… But, since I only vaguely understand asterisk’s mutexes, this [a] might not work and [b] is almost certainly a horrible hack.

Would anyone suggest a better solution? Thanks.


Asterisk 11.2.1 built by root @ on a x86_64 running Linux on 2013-10-29 17:19:17 UTC

=======================================================================
=== Currently Held Locks ==============================================

===
=== <lock#> (): (times locked)

=== Thread ID: 0x7ff2b5931700 (logger_thread started at [ 1225] logger.c init_logger())
=== —> Lock #0 (logger.c): RDLOCK 1039 logger_print_normal &(&logchannels)->lock 0x8372f0 (1)
/usr/sbin/asterisk(ast_bt_get_addresses+0xe) [0x4f566e]
/usr/sbin/asterisk(__ast_rwlock_rdlock+0xb5) [0x4f24d5]
/usr/sbin/asterisk() [0x4f5f63]
/usr/sbin/asterisk() [0x4f9acf]
/usr/sbin/asterisk() [0x5761bb]
/lib64/libpthread.so.0(+0x7851) [0x7ff2b3e02851]
/lib64/libc.so.6(clone+0x6d) [0x7ff2b537694d]
=== -------------------------------------------------------------------

=== Thread ID: 0x7ff2900cf700 (do_monitor started at [28728] chan_sip.c restart_monitor())
=== —> Lock #0 (chan_sip.c): MUTEX 28137 handle_request_do &netlock 0x7ff270b329a0 (1)
/usr/sbin/asterisk(ast_bt_get_addresses+0xe) [0x4f566e]
/usr/sbin/asterisk(__ast_pthread_mutex_lock+0xb4) [0x4f3be4]
/usr/lib/asterisk/modules/chan_sip.so(+0x91b70) [0x7ff270903b70]
/usr/lib/asterisk/modules/chan_sip.so(+0x93b86) [0x7ff270905b86]
/usr/sbin/asterisk(ast_io_wait+0x82) [0x4eb582]
/usr/lib/asterisk/modules/chan_sip.so(+0x6c623) [0x7ff2708de623]
/usr/sbin/asterisk() [0x5761bb]
/lib64/libpthread.so.0(+0x7851) [0x7ff2b3e02851]
/lib64/libc.so.6(clone+0x6d) [0x7ff2b537694d]
=== —> Lock #1 (chan_sip.c): MUTEX 9038 sip_pvt_lock_full pvt 0x7ff29a23e990 (1)
/usr/sbin/asterisk(ast_bt_get_addresses+0xe) [0x4f566e]
/usr/sbin/asterisk(__ast_pthread_mutex_lock+0xb4) [0x4f3be4]
/usr/lib/asterisk/modules/chan_sip.so(+0x126bf) [0x7ff2708846bf]
/usr/lib/asterisk/modules/chan_sip.so(+0x91ba0) [0x7ff270903ba0]
/usr/lib/asterisk/modules/chan_sip.so(+0x93b86) [0x7ff270905b86]
/usr/sbin/asterisk(ast_io_wait+0x82) [0x4eb582]
/usr/lib/asterisk/modules/chan_sip.so(+0x6c623) [0x7ff2708de623]
/usr/sbin/asterisk() [0x5761bb]
/lib64/libpthread.so.0(+0x7851) [0x7ff2b3e02851]
/lib64/libc.so.6(clone+0x6d) [0x7ff2b537694d]
=== —> Lock #2 (pbx.c): MUTEX 11809 ast_rdlock_contexts &conlock 0x8392a0 (1)
/usr/sbin/asterisk(ast_bt_get_addresses+0xe) [0x4f566e]
/usr/sbin/asterisk(__ast_pthread_mutex_lock+0xb4) [0x4f3be4]
/usr/sbin/asterisk() [0x52bf5a]
/usr/sbin/asterisk(ast_exists_extension+0x34) [0x52c824]
/usr/lib/asterisk/modules/chan_sip.so(+0x3b95c) [0x7ff2708ad95c]
/usr/lib/asterisk/modules/chan_sip.so(+0x88288) [0x7ff2708fa288]
/usr/lib/asterisk/modules/chan_sip.so(+0x906d6) [0x7ff2709026d6]
/usr/lib/asterisk/modules/chan_sip.so(+0x91c72) [0x7ff270903c72]
/usr/lib/asterisk/modules/chan_sip.so(+0x93b86) [0x7ff270905b86]
/usr/sbin/asterisk(ast_io_wait+0x82) [0x4eb582]
/usr/lib/asterisk/modules/chan_sip.so(+0x6c623) [0x7ff2708de623]
/usr/sbin/asterisk() [0x5761bb]
/lib64/libpthread.so.0(+0x7851) [0x7ff2b3e02851]
/lib64/libc.so.6(clone+0x6d) [0x7ff2b537694d]
=== —> Waiting for Lock #3 (pbx_lua.c): MUTEX 1165 lua_load_extensions &config_file_lock 0x7ff2b0c67b80 (1)
/usr/sbin/asterisk(ast_bt_get_addresses+0xe) [0x4f566e]
/usr/sbin/asterisk(__ast_pthread_mutex_lock+0xb4) [0x4f3be4]
/usr/lib/asterisk/modules/pbx_lua.so(+0x2afb) [0x7ff2b0a63afb]
/usr/lib/asterisk/modules/pbx_lua.so(+0x2ea3) [0x7ff2b0a63ea3]
/usr/lib/asterisk/modules/pbx_lua.so(+0x3fad) [0x7ff2b0a64fad]
/usr/sbin/asterisk(pbx_find_extension+0x9b3) [0x5298b3]
/usr/sbin/asterisk() [0x52bf9c]
/usr/sbin/asterisk(ast_exists_extension+0x34) [0x52c824]
/usr/lib/asterisk/modules/chan_sip.so(+0x3b95c) [0x7ff2708ad95c]
/usr/lib/asterisk/modules/chan_sip.so(+0x88288) [0x7ff2708fa288]
/usr/lib/asterisk/modules/chan_sip.so(+0x906d6) [0x7ff2709026d6]
/usr/lib/asterisk/modules/chan_sip.so(+0x91c72) [0x7ff270903c72]
/usr/lib/asterisk/modules/chan_sip.so(+0x93b86) [0x7ff270905b86]
/usr/sbin/asterisk(ast_io_wait+0x82) [0x4eb582]
/usr/lib/asterisk/modules/chan_sip.so(+0x6c623) [0x7ff2708de623]
/usr/sbin/asterisk() [0x5761bb]
/lib64/libpthread.so.0(+0x7851) [0x7ff2b3e02851]
/lib64/libc.so.6(clone+0x6d) [0x7ff2b537694d]
=== — ---> Locked Here: pbx_lua.c line 1209 (lua_reload_extensions)
=== -------------------------------------------------------------------

=== Thread ID: 0x7ff26a21d700 (netconsole started at [ 1442] asterisk.c listener())
=== —> Lock #0 (loader.c): MUTEX 703 ast_module_reload &reloadlock 0x837240 (1)
/usr/sbin/asterisk(ast_bt_get_addresses+0xe) [0x4f566e]
/usr/sbin/asterisk(__ast_pthread_mutex_trylock+0xb6) [0x4f0d76]
/usr/sbin/asterisk(ast_module_reload+0x3d) [0x4ed85d]
/usr/sbin/asterisk() [0x492587]
/usr/sbin/asterisk(ast_cli_command_full+0x157) [0x494d07]
/usr/lib/asterisk/modules/res_clialiases.so(+0x1556) [0x7ff27d766556]
/usr/sbin/asterisk(ast_cli_command_full+0x157) [0x494d07]
/usr/sbin/asterisk(ast_cli_command_multiple_full+0x7c) [0x494f3c]
/usr/sbin/asterisk() [0x4403ca]
/usr/sbin/asterisk() [0x5761bb]
/lib64/libpthread.so.0(+0x7851) [0x7ff2b3e02851]
/lib64/libc.so.6(clone+0x6d) [0x7ff2b537694d]
=== —> Lock #1 (loader.c): MUTEX 743 ast_module_reload &(&module_list)->lock 0x8371b0 (1)
/usr/sbin/asterisk(ast_bt_get_addresses+0xe) [0x4f566e]
/usr/sbin/asterisk(__ast_pthread_mutex_lock+0xb4) [0x4f3be4]
/usr/sbin/asterisk(ast_module_reload+0x2ef) [0x4edb0f]
/usr/sbin/asterisk() [0x492587]
/usr/sbin/asterisk(ast_cli_command_full+0x157) [0x494d07]
/usr/lib/asterisk/modules/res_clialiases.so(+0x1556) [0x7ff27d766556]
/usr/sbin/asterisk(ast_cli_command_full+0x157) [0x494d07]
/usr/sbin/asterisk(ast_cli_command_multiple_full+0x7c) [0x494f3c]
/usr/sbin/asterisk() [0x4403ca]
/usr/sbin/asterisk() [0x5761bb]
/lib64/libpthread.so.0(+0x7851) [0x7ff2b3e02851]
/lib64/libc.so.6(clone+0x6d) [0x7ff2b537694d]
=== —> Lock #2 (pbx_lua.c): MUTEX 1209 lua_reload_extensions &config_file_lock 0x7ff2b0c67b80 (1)
/usr/sbin/asterisk(ast_bt_get_addresses+0xe) [0x4f566e]
/usr/sbin/asterisk(__ast_pthread_mutex_lock+0xb4) [0x4f3be4]
/usr/lib/asterisk/modules/pbx_lua.so(+0x4857) [0x7ff2b0a65857]
/usr/sbin/asterisk(ast_module_reload+0x668) [0x4ede88]
/usr/sbin/asterisk() [0x492587]
/usr/sbin/asterisk(ast_cli_command_full+0x157) [0x494d07]
/usr/lib/asterisk/modules/res_clialiases.so(+0x1556) [0x7ff27d766556]
/usr/sbin/asterisk(ast_cli_command_full+0x157) [0x494d07]
/usr/sbin/asterisk(ast_cli_command_multiple_full+0x7c) [0x494f3c]
/usr/sbin/asterisk() [0x4403ca]
/usr/sbin/asterisk() [0x5761bb]
/lib64/libpthread.so.0(+0x7851) [0x7ff2b3e02851]
/lib64/libc.so.6(clone+0x6d) [0x7ff2b537694d]
=== —> Lock #3 (pbx.c): MUTEX 8674 ast_merge_contexts_and_delete &context_merge_lock 0x83a680 (1)
/usr/sbin/asterisk(ast_bt_get_addresses+0xe) [0x4f566e]
/usr/sbin/asterisk(__ast_pthread_mutex_lock+0xb4) [0x4f3be4]
/usr/sbin/asterisk(ast_merge_contexts_and_delete+0x74) [0x52fed4]
/usr/lib/asterisk/modules/pbx_lua.so(+0x4897) [0x7ff2b0a65897]
/usr/sbin/asterisk(ast_module_reload+0x668) [0x4ede88]
/usr/sbin/asterisk() [0x492587]
/usr/sbin/asterisk(ast_cli_command_full+0x157) [0x494d07]
/usr/lib/asterisk/modules/res_clialiases.so(+0x1556) [0x7ff27d766556]
/usr/sbin/asterisk(ast_cli_command_full+0x157) [0x494d07]
/usr/sbin/asterisk(ast_cli_command_multiple_full+0x7c) [0x494f3c]
/usr/sbin/asterisk() [0x4403ca]
/usr/sbin/asterisk() [0x5761bb]
/lib64/libpthread.so.0(+0x7851) [0x7ff2b3e02851]
/lib64/libc.so.6(clone+0x6d) [0x7ff2b537694d]
=== —> Waiting for Lock #4 (pbx.c): MUTEX 11804 ast_wrlock_contexts &conlock 0x8392a0 (1)
/usr/sbin/asterisk(ast_bt_get_addresses+0xe) [0x4f566e]
/usr/sbin/asterisk(__ast_pthread_mutex_lock+0xb4) [0x4f3be4]
/usr/sbin/asterisk(ast_merge_contexts_and_delete+0x93) [0x52fef3]
/usr/lib/asterisk/modules/pbx_lua.so(+0x4897) [0x7ff2b0a65897]
/usr/sbin/asterisk(ast_module_reload+0x668) [0x4ede88]
/usr/sbin/asterisk() [0x492587]
/usr/sbin/asterisk(ast_cli_command_full+0x157) [0x494d07]
/usr/lib/asterisk/modules/res_clialiases.so(+0x1556) [0x7ff27d766556]
/usr/sbin/asterisk(ast_cli_command_full+0x157) [0x494d07]
/usr/sbin/asterisk(ast_cli_command_multiple_full+0x7c) [0x494f3c]
/usr/sbin/asterisk() [0x4403ca]
/usr/sbin/asterisk() [0x5761bb]
/lib64/libpthread.so.0(+0x7851) [0x7ff2b3e02851]
/lib64/libc.so.6(clone+0x6d) [0x7ff2b537694d]
=== — ---> Locked Here: pbx.c line 11809 (ast_rdlock_contexts)
=== -------------------------------------------------------------------

=======================================================================

0x7ff2b0f97700 62284 netconsole started at [ 1442] asterisk.c listener()
0x7ff2b0efd700 59759 handle_tcptls_connection started at [ 322] tcptls.c ast_tcptls_server_root()
0x7ff26a21d700 6184 netconsole started at [ 1442] asterisk.c listener()
0x7ff269f21700 5481 inotify_daemon started at [ 349] stdtime/localtime.c add_notify()
0x7ff26a1a1700 5479 autoservice_run started at [ 229] autoservice.c ast_autoservice_start()
0x7ff2693a7700 3366 monitor_sig_flags started at [ 4231] asterisk.c main()
0x7ff2696b6700 3364 tps_processing_function started at [ 466] taskprocessor.c ast_taskprocessor_get()
0x7ff269732700 3363 tps_processing_function started at [ 466] taskprocessor.c ast_taskprocessor_get()
0x7ff2697ae700 3361 cleanup started at [ 414] pbx_realtime.c load_module()
0x7ff26982a700 3343 lock_broker started at [ 509] func_lock.c load_module()
0x7ff26a391700 3340 network_thread started at [ 2265] pbx_dundi.c start_network_thread()
0x7ff26a299700 3342 process_clearcache started at [ 2267] pbx_dundi.c start_network_thread()
0x7ff26a315700 3341 process_precache started at [ 2266] pbx_dundi.c start_network_thread()
0x7ff26a40d700 3325 scan_thread started at [ 891] pbx_spool.c load_module()
0x7ff269627700 3324 do_monitor started at [ 5739] chan_unistim.c restart_monitor()
0x7ff2698a6700 3323 do_monitor started at [ 1166] chan_phone.c restart_monitor()
0x7ff269dfa700 3308 iax2_process_thread started at [12403] chan_iax2.c start_network_thread()
0x7ff269d7e700 3309 iax2_process_thread started at [12403] chan_iax2.c start_network_thread()
0x7ff269922700 3318 network_thread started at [12418] chan_iax2.c start_network_thread()
0x7ff26999e700 3317 iax2_process_thread started at [12403] chan_iax2.c start_network_thread()
0x7ff269a1a700 3316 iax2_process_thread started at [12403] chan_iax2.c start_network_thread()
0x7ff269a96700 3315 iax2_process_thread started at [12403] chan_iax2.c start_network_thread()
0x7ff269b12700 3314 iax2_process_thread started at [12403] chan_iax2.c start_network_thread()
0x7ff269b8e700 3313 iax2_process_thread started at [12403] chan_iax2.c start_network_thread()
0x7ff269c0a700 3312 iax2_process_thread started at [12403] chan_iax2.c start_network_thread()
0x7ff269c86700 3311 iax2_process_thread started at [12403] chan_iax2.c start_network_thread()
0x7ff269d02700 3310 iax2_process_thread started at [12403] chan_iax2.c start_network_thread()
0x7ff26a489700 3305 sched_run started at [ 182] sched.c ast_sched_start_thread()
0x7ff26a505700 3302 tps_processing_function started at [ 466] taskprocessor.c ast_taskprocessor_get()
0x7ff26a581700 3293 do_monitor started at [ 3915] chan_mgcp.c restart_monitor()
0x7ff2900cf700 3292 do_monitor started at [28728] chan_sip.c restart_monitor()
0x7ff29014b700 3291 desc->accept_fn started at [ 589] tcptls.c ast_tcptls_server_start()
0x7ff2901c7700 3290 desc->accept_fn started at [ 589] tcptls.c ast_tcptls_server_start()
0x7ff292c4f700 3288 do_timing started at [ 495] res_timing_pthread.c init_timing_thread()
0x7ff2b1013700 3283 do_refresh started at [ 1854] res_calendar.c load_module()
0x7ff2b21b8700 3280 tps_processing_function started at [ 466] taskprocessor.c ast_taskprocessor_get()
0x7ff2b213c700 3281 sched_run started at [ 182] sched.c ast_sched_start_thread()
0x7ff2b2234700 3279 do_parking_thread started at [ 9024] features.c ast_features_init()
0x7ff2b22b0700 3278 tps_processing_function started at [ 466] taskprocessor.c ast_taskprocessor_get()
0x7ff2b23a8700 3276 do_devstate_changes started at [ 741] devicestate.c ast_device_state_engine_init()
0x7ff2b232c700 3277 do_presence_changes started at [ 315] presencestate.c ast_presence_state_engine_init()
0x7ff2b2424700 3275 desc->accept_fn started at [ 589] tcptls.c ast_tcptls_server_start()
0x7ff2b24a0700 3274 desc->accept_fn started at [ 589] tcptls.c ast_tcptls_server_start()
0x7ff2b251c700 3273 desc->accept_fn started at [ 589] tcptls.c ast_tcptls_server_start()
0x7ff2b2598700 3272 tps_processing_function started at [ 466] taskprocessor.c ast_taskprocessor_get()
0x7ff2b2614700 3271 db_sync_thread started at [ 1017] db.c astdb_init()
0x7ff2b5a29700 3268 tps_processing_function started at [ 466] taskprocessor.c ast_taskprocessor_get()
0x7ff2b59ad700 3269 listener started at [ 1502] asterisk.c ast_makesocket()
0x7ff2b5931700 3270 logger_thread started at [ 1225] logger.c init_logger()
49 threads listed.