I have been struggling getting Asterisk realtime up and running for almost 2-weeks now, after 2 weeks I have almost all errors resolved. This is my first realtime setup that has been tough due to lack of documentation on the database aspects.
Each time a call starts, it rings but as soon as the receiving party picks up Asterisk crashes and it exits me out of the terminal. As I mentioned I have almost all errors resolved, I don’t believe it is connected to the issue but I will mention it in the interest of full disclosure. Most issues were related to database information rather, the sole remaining error is on 1 user/extension only, a warning appears on registration that the user does not have a mailbox however it does. No other users have this error and the config is identical in the voicemail database.
I am running asterisk 1.8.21.0 running with mySQL on a VPS running Ubuntu 12.02.2 LTS that has 1GB Ram and an Intel Xeon E5-2670 2.6GHz processor. Not an ideal environment however I have run Asterisk successfully (without realtime) on worse environments. I was advised on this environment to run a 32-bit version regardless of the processor being 64-bit, I have started from scratch on a 64-bit environment and the same issue occurs.
I have run a backtrace on the core dump according to the wiki article “Getting a Backtrace” but I don’t fully understand it and am hoping someone can help me to diagnose the issue.
Thanks in advance - the output from each step in the instructions is below.
STEP 1
# gdb -se "asterisk" -c core | tee /tmp/backtrace.txt
GNU gdb (Ubuntu/Linaro 7.4-2012.04-0ubuntu2.1) 7.4-2012.04
Copyright (C) 2012 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "i686-linux-gnu".
For bug reporting instructions, please see:
<http://bugs.launchpad.net/gdb-linaro/>...
Reading symbols from /usr/sbin/asterisk...done.
warning: [New LWP 4067]
[New LWP 4884]
[New LWP 4045]
[New LWP 4046]
[New LWP 4058]
[New LWP 4037]
[New LWP 4038]
[New LWP 4042]
[New LWP 4062]
[New LWP 4053]
[New LWP 4066]
[New LWP 4049]
[New LWP 4039]
[New LWP 4044]
[New LWP 4064]
[New LWP 4048]
[New LWP 4050]
[New LWP 4051]
[New LWP 4047]
[New LWP 4055]
[New LWP 4041]
[New LWP 4056]
[New LWP 4052]
[New LWP 4043]
[New LWP 4028]
[New LWP 4060]
[New LWP 4063]
[New LWP 4031]
[New LWP 4057]
[New LWP 4061]
[New LWP 4065]
[New LWP 4054]
[New LWP 4059]
Can't read pathname for load map: Input/output error.
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/i386-linux-gnu/libthread_db.so.1".
Core was generated by `asterisk -vvvvvcg'.
Program terminated with signal 6, Aborted.
#0 0xb77c6424 in __kernel_vsyscall ()
(gdb)
STEP 2 with “set logging on”
(gdb) bt
#0 0xb77c6424 in __kernel_vsyscall ()
#1 0xb743f1df in raise () from /lib/i386-linux-gnu/libc.so.6
#2 0xb7442825 in abort () from /lib/i386-linux-gnu/libc.so.6
#3 0xb747c39a in ?? () from /lib/i386-linux-gnu/libc.so.6
#4 0xb7486ee2 in ?? () from /lib/i386-linux-gnu/libc.so.6
#5 0x08085321 in internal_ao2_ref (user_data=0x9ab24c0, delta=-1) at astobj2.c:286
#6 0x08085223 in __ao2_ref (user_data=0x9ab24c0, delta=-1) at astobj2.c:244
#7 0x08086165 in ao2_iterator_destroy (i=0xb43c7180) at astobj2.c:830
#8 0x081012db in build_dialplan_useage_map (usage_map=0xb43c7200, complain=0) at features.c:6360
#9 0x08101b24 in load_config (reload=1) at features.c:6699
#10 0x08102277 in ast_features_reload () at features.c:6847
#11 0x0811d1ea in ast_module_reload (name=0x0) at loader.c:743
#12 0x08082245 in monitor_sig_flags (unused=0x0) at asterisk.c:3243
#13 0x081a2cc2 in dummy_start (data=0xa0d2b90) at utils.c:1073
#14 0xb72a9d4c in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#15 0xb74ffdde in clone () from /lib/i386-linux-gnu/libc.so.6
(gdb)
STEP 3
(gdb) bt full
#0 0xb77c6424 in __kernel_vsyscall ()
No symbol table info available.
#1 0xb743f1df in raise () from /lib/i386-linux-gnu/libc.so.6
No symbol table info available.
#2 0xb7442825 in abort () from /lib/i386-linux-gnu/libc.so.6
No symbol table info available.
#3 0xb747c39a in ?? () from /lib/i386-linux-gnu/libc.so.6
No symbol table info available.
#4 0xb7486ee2 in ?? () from /lib/i386-linux-gnu/libc.so.6
No symbol table info available.
#5 0x08085321 in internal_ao2_ref (user_data=0x9ab24c0, delta=-1) at astobj2.c:286
obj = 0x9ab2490
current_value = 0
ret = 1
__PRETTY_FUNCTION__ = "internal_ao2_ref"
#6 0x08085223 in __ao2_ref (user_data=0x9ab24c0, delta=-1) at astobj2.c:244
No locals.
#7 0x08086165 in ao2_iterator_destroy (i=0xb43c7180) at astobj2.c:830
No locals.
#8 0x081012db in build_dialplan_useage_map (usage_map=0xb43c7200, complain=0) at features.c:6360
status = 0
iter = {c = 0x9ab24c0, flags = 0, bucket = 7, c_version = 0, obj = 0x0, version = 0}
curlot = 0x0
#9 0x08101b24 in load_config (reload=1) at features.c:6699
config_flags = {flags = 2}
cfg = 0xb66007a0
old_usage_map = {first = 0x0, last = 0x0}
new_usage_map = {first = 0x0, last = 0x0}
__PRETTY_FUNCTION__ = "load_config"
#10 0x08102277 in ast_features_reload () at features.c:6847
con = 0x0
res = -44088064
__PRETTY_FUNCTION__ = "ast_features_reload"
#11 0x0811d1ea in ast_module_reload (name=0x0) at loader.c:743
cur = 0xb43c72f8
res = 2
i = 7
__PRETTY_FUNCTION__ = "ast_module_reload"
#12 0x08082245 in monitor_sig_flags (unused=0x0) at asterisk.c:3243
p = {fd = 15, events = 1, revents = 1}
a = -1219438211
#13 0x081a2cc2 in dummy_start (data=0xa0d2b90) at utils.c:1073
__cancel_buf = {__cancel_jmp_buf = {{__cancel_jmp_buf = {-1221873676, 0, 4001536, -1271106712, 181999771, 1181105635},
__mask_was_saved = 0}}, __pad = {0xb43c7424, 0x0, 0x0, 0x0}}
__cancel_routine = 0x807993d <ast_unregister_thread>
__cancel_arg = 0xb43c7b40
__not_first_call = 0
ret = 0x0
a = {start_routine = 0x80821e7 <monitor_sig_flags>, data = 0x0, name = 0x9d43b78 "monitor_sig_flags started at [ 4071] asterisk.c main()"}
#14 0xb72a9d4c in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
No symbol table info available.
#15 0xb74ffdde in clone () from /lib/i386-linux-gnu/libc.so.6
No symbol table info available.
(gdb)
STEP 4
(gdb) thread apply all bt
Thread 33 (Thread 0xb45a7b40 (LWP 4059)):
#0 0xb77c6424 in __kernel_vsyscall ()
#1 0xb74f1690 in poll () from /lib/i386-linux-gnu/libc.so.6
#2 0xb61767e3 in scan_thread (unused=0x0) at pbx_spool.c:693
#3 0x081a2cc2 in dummy_start (data=0xa02fab0) at utils.c:1073
#4 0xb72a9d4c in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#5 0xb74ffdde in clone () from /lib/i386-linux-gnu/libc.so.6
Thread 32 (Thread 0xb46d3b40 (LWP 4054)):
#0 0xb77c6424 in __kernel_vsyscall ()
#1 0xb74f1690 in poll () from /lib/i386-linux-gnu/libc.so.6
#2 0x081197be in ast_io_wait (ioc=0xa016530, howlong=1000) at io.c:279
#3 0xb6339392 in network_thread (ignore=0x0) at chan_iax2.c:12210
#4 0x081a2cc2 in dummy_start (data=0xa01b290) at utils.c:1073
#5 0xb72a9d4c in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#6 0xb74ffdde in clone () from /lib/i386-linux-gnu/libc.so.6
Thread 31 (Thread 0xb443fb40 (LWP 4065)):
#0 0xb77c6424 in __kernel_vsyscall ()
#1 0xb72ad96b in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
#2 0xb750d6ec in pthread_cond_wait () from /lib/i386-linux-gnu/libc.so.6
#3 0x0811ebec in __ast_cond_wait (filename=0x8200c98 "taskprocessor.c", lineno=310, func=0x8201336 "tps_processing_function",
cond_name=0x8200fe9 "&i->poll_cond", mutex_name=0x8200fd2 "&i->taskprocessor_lock", cond=0xa0c9a8c, t=0xa0c9ac0) at lock.c:547
#4 0x08192947 in tps_processing_function (data=0xa0c9a88) at taskprocessor.c:310
#5 0x081a2cc2 in dummy_start (data=0xa094a68) at utils.c:1073
#6 0xb72a9d4c in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#7 0xb74ffdde in clone () from /lib/i386-linux-gnu/libc.so.6
Thread 30 (Thread 0xb452fb40 (LWP 4061)):
#0 0xb77c6424 in __kernel_vsyscall ()
#1 0xb74c8f26 in nanosleep () from /lib/i386-linux-gnu/libc.so.6
#2 0xb74c8d1f in sleep () from /lib/i386-linux-gnu/libc.so.6
#3 0xb618ff79 in process_precache (ign=0x0) at pbx_dundi.c:2248
#4 0x081a2cc2 in dummy_start (data=0xa0c93a8) at utils.c:1073
#5 0xb72a9d4c in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#6 0xb74ffdde in clone () from /lib/i386-linux-gnu/libc.so.6
Thread 29 (Thread 0xb461fb40 (LWP 4057)):
#0 0xb77c6424 in __kernel_vsyscall ()
#1 0xb74c8f26 in nanosleep () from /lib/i386-linux-gnu/libc.so.6
#2 0xb5d822c0 in lock_broker (unused=0x0) at func_lock.c:196
#3 0x081a2cc2 in dummy_start (data=0xa0391d0) at utils.c:1073
#4 0xb72a9d4c in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#5 0xb74ffdde in clone () from /lib/i386-linux-gnu/libc.so.6
Thread 28 (Thread 0xb70b2b40 (LWP 4031)):
#0 0xb77c6424 in __kernel_vsyscall ()
#1 0xb72ad96b in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
#2 0xb750d6ec in pthread_cond_wait () from /lib/i386-linux-gnu/libc.so.6
#3 0x0811ebec in __ast_cond_wait (filename=0x81e1b94 "devicestate.c", lineno=549, func=0x81e222a "do_devstate_changes",
cond_name=0x81e1e07 "&change_pending", mutex_name=0x81e1e17 "&state_changes.lock", cond=0x824d1a0, t=0x8234c28) at lock.c:547
#4 0x080dc3bf in do_devstate_changes (data=0x0) at devicestate.c:549
#5 0x081a2cc2 in dummy_start (data=0x9aa9d50) at utils.c:1073
#6 0xb72a9d4c in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#7 0xb74ffdde in clone () from /lib/i386-linux-gnu/libc.so.6
Thread 27 (Thread 0xb44b7b40 (LWP 4063)):
#0 0xb77c6424 in __kernel_vsyscall ()
#1 0xb74f1690 in poll () from /lib/i386-linux-gnu/libc.so.6
#2 0xb62e8ea3 in do_monitor (data=0x0) at chan_phone.c:1086
#3 0x081a2cc2 in dummy_start (data=0xa0ac920) at utils.c:1073
#4 0xb72a9d4c in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#5 0xb74ffdde in clone () from /lib/i386-linux-gnu/libc.so.6
Thread 26 (Thread 0xb456bb40 (LWP 4060)):
#0 0xb77c6424 in __kernel_vsyscall ()
#1 0xb74f1690 in poll () from /lib/i386-linux-gnu/libc.so.6
#2 0x081197be in ast_io_wait (ioc=0xa055e50, howlong=1000) at io.c:279
#3 0xb618fb68 in network_thread (ignore=0x0) at pbx_dundi.c:2171
#4 0x081a2cc2 in dummy_start (data=0xa0912f8) at utils.c:1073
#5 0xb72a9d4c in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#6 0xb74ffdde in clone () from /lib/i386-linux-gnu/libc.so.6
Thread 25 (Thread 0xb712b6c0 (LWP 4028)):
#0 0x0811b6c5 in ast_module_unregister (info=0xb67873a0) at loader.c:194
#1 0xb6783ee6 in __unreg_module () at res_musiconhold.c:1986
#2 0xb77d644c in ?? () from /lib/ld-linux.so.2
#3 0xb7443f51 in ?? () from /lib/i386-linux-gnu/libc.so.6
#4 0xb7443fdd in exit () from /lib/i386-linux-gnu/libc.so.6
#5 0x0807db0c in really_quit (num=0, niceness=SHUTDOWN_FAST, restart=0) at asterisk.c:1840
#6 0x0807d2e0 in quit_handler (num=0, niceness=SHUTDOWN_FAST, restart=0) at asterisk.c:1676
#7 0x08084bfd in main (argc=2, argv=0xbfe45704) at asterisk.c:4081
Thread 24 (Thread 0xb4c7cb40 (LWP 4043)):
#0 0xb77c6424 in __kernel_vsyscall ()
#1 0xb72ad96b in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
#2 0xb750d6ec in pthread_cond_wait () from /lib/i386-linux-gnu/libc.so.6
#3 0x0811ebec in __ast_cond_wait (filename=0x81fef0c "sched.c", lineno=110, func=0x81ff39a "sched_run", cond_name=0x81fef1e "&st->cond",
mutex_name=0x81fef14 "&st->lock", cond=0xa015ebc, t=0xa015e9c) at lock.c:547
#4 0x081871b2 in sched_run (data=0xa015e98) at sched.c:110
#5 0x081a2cc2 in dummy_start (data=0xa015d90) at utils.c:1073
#6 0xb72a9d4c in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#7 0xb74ffdde in clone () from /lib/i386-linux-gnu/libc.so.6
Thread 23 (Thread 0xb474bb40 (LWP 4052)):
#0 0xb77c6424 in __kernel_vsyscall ()
#1 0xb72ad96b in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
#2 0xb750d6ec in pthread_cond_wait () from /lib/i386-linux-gnu/libc.so.6
#3 0x0811ebec in __ast_cond_wait (filename=0xb6351d54 "chan_iax2.c", lineno=11741, func=0xb6359184 "iax2_process_thread",
cond_name=0xb6351fbd "&thread->cond", mutex_name=0xb6351faf "&thread->lock", cond=0xa025b04, t=0xa025ae4) at lock.c:547
#4 0xb6336bde in iax2_process_thread (data=0xa024a50) at chan_iax2.c:11741
#5 0x081a2cc2 in dummy_start (data=0xa01b9c0) at utils.c:1073
#6 0xb72a9d4c in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#7 0xb74ffdde in clone () from /lib/i386-linux-gnu/libc.so.6
Thread 22 (Thread 0xb465bb40 (LWP 4056)):
#0 0xb77c6424 in __kernel_vsyscall ()
#1 0xb74f1690 in poll () from /lib/i386-linux-gnu/libc.so.6
#2 0x081197be in ast_io_wait (ioc=0xa0273b8, howlong=1000) at io.c:279
#3 0xb61f1e92 in do_monitor (data=0x0) at chan_skinny.c:6618
#4 0x081a2cc2 in dummy_start (data=0xa01b290) at utils.c:1073
#5 0xb72a9d4c in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#6 0xb74ffdde in clone () from /lib/i386-linux-gnu/libc.so.6
Thread 21 (Thread 0xb5878b40 (LWP 4041)):
#0 0xb77c6424 in __kernel_vsyscall ()
#1 0xb74f1690 in poll () from /lib/i386-linux-gnu/libc.so.6
#2 0xb6729043 in do_monitor (data=0x0) at chan_dahdi.c:11784
#3 0x081a2cc2 in dummy_start (data=0x9da1ee8) at utils.c:1073
#4 0xb72a9d4c in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#5 0xb74ffdde in clone () from /lib/i386-linux-gnu/libc.so.6
Thread 20 (Thread 0xb4697b40 (LWP 4055)):
#0 0xb77c6424 in __kernel_vsyscall ()
#1 0xb7501498 in accept () from /lib/i386-linux-gnu/libc.so.6
#2 0xb61f1aed in accept_thread (ignore=0x0) at chan_skinny.c:6556
#3 0x081a2cc2 in dummy_start (data=0xa019818) at utils.c:1073
#4 0xb72a9d4c in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#5 0xb74ffdde in clone () from /lib/i386-linux-gnu/libc.so.6
Thread 19 (Thread 0xb4a87b40 (LWP 4047)):
#0 0xb77c6424 in __kernel_vsyscall ()
#1 0xb72ad96b in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
#2 0xb750d6ec in pthread_cond_wait () from /lib/i386-linux-gnu/libc.so.6
#3 0x0811ebec in __ast_cond_wait (filename=0xb6351d54 "chan_iax2.c", lineno=11741, func=0xb6359184 "iax2_process_thread",
cond_name=0xb6351fbd "&thread->cond", mutex_name=0xb6351faf "&thread->lock", cond=0xa020424, t=0xa020404) at lock.c:547
#4 0xb6336bde in iax2_process_thread (data=0xa01f370) at chan_iax2.c:11741
#5 0x081a2cc2 in dummy_start (data=0xa018700) at utils.c:1073
#6 0xb72a9d4c in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#7 0xb74ffdde in clone () from /lib/i386-linux-gnu/libc.so.6
Thread 18 (Thread 0xb4787b40 (LWP 4051)):
#0 0xb77c6424 in __kernel_vsyscall ()
#1 0xb72ad96b in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
#2 0xb750d6ec in pthread_cond_wait () from /lib/i386-linux-gnu/libc.so.6
#3 0x0811ebec in __ast_cond_wait (filename=0xb6351d54 "chan_iax2.c", lineno=11741, func=0xb6359184 "iax2_process_thread",
cond_name=0xb6351fbd "&thread->cond", mutex_name=0xb6351faf "&thread->lock", cond=0xa0249a4, t=0xa024984) at lock.c:547
#4 0xb6336bde in iax2_process_thread (data=0xa0238f0) at chan_iax2.c:11741
#5 0x081a2cc2 in dummy_start (data=0xa019720) at utils.c:1073
#6 0xb72a9d4c in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#7 0xb74ffdde in clone () from /lib/i386-linux-gnu/libc.so.6
Thread 17 (Thread 0xb47c3b40 (LWP 4050)):
#0 0xb77c6424 in __kernel_vsyscall ()
#1 0xb72ad96b in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
#2 0xb750d6ec in pthread_cond_wait () from /lib/i386-linux-gnu/libc.so.6
#3 0x0811ebec in __ast_cond_wait (filename=0xb6351d54 "chan_iax2.c", lineno=11741, func=0xb6359184 "iax2_process_thread",
cond_name=0xb6351fbd "&thread->cond", mutex_name=0xb6351faf "&thread->lock", cond=0xa023844, t=0xa023824) at lock.c:547
#4 0xb6336bde in iax2_process_thread (data=0xa022790) at chan_iax2.c:11741
#5 0x081a2cc2 in dummy_start (data=0xa01bbb0) at utils.c:1073
#6 0xb72a9d4c in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#7 0xb74ffdde in clone () from /lib/i386-linux-gnu/libc.so.6
Thread 16 (Thread 0xb4a4bb40 (LWP 4048)):
#0 0xb77c6424 in __kernel_vsyscall ()
#1 0xb72ad96b in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
#2 0xb750d6ec in pthread_cond_wait () from /lib/i386-linux-gnu/libc.so.6
#3 0x0811ebec in __ast_cond_wait (filename=0xb6351d54 "chan_iax2.c", lineno=11741, func=0xb6359184 "iax2_process_thread",
cond_name=0xb6351fbd "&thread->cond", mutex_name=0xb6351faf "&thread->lock", cond=0xa021584, t=0xa021564) at lock.c:547
#4 0xb6336bde in iax2_process_thread (data=0xa0204d0) at chan_iax2.c:11741
#5 0x081a2cc2 in dummy_start (data=0xa01b9c0) at utils.c:1073
#6 0xb72a9d4c in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#7 0xb74ffdde in clone () from /lib/i386-linux-gnu/libc.so.6
Thread 15 (Thread 0xb447bb40 (LWP 4064)):
#0 0xb77c6424 in __kernel_vsyscall ()
#1 0xb74c8f26 in nanosleep () from /lib/i386-linux-gnu/libc.so.6
#2 0xb617b2ab in cleanup (unused=0x0) at pbx_realtime.c:133
#3 0x081a2cc2 in dummy_start (data=0xa0bb4e0) at utils.c:1073
#4 0xb72a9d4c in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#5 0xb74ffdde in clone () from /lib/i386-linux-gnu/libc.so.6
Thread 14 (Thread 0xb4affb40 (LWP 4044)):
#0 0xb77c6424 in __kernel_vsyscall ()
#1 0xb72ad96b in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
#2 0xb750d6ec in pthread_cond_wait () from /lib/i386-linux-gnu/libc.so.6
#3 0x0811ebec in __ast_cond_wait (filename=0xb6351d54 "chan_iax2.c", lineno=11741, func=0xb6359184 "iax2_process_thread",
cond_name=0xb6351fbd "&thread->cond", mutex_name=0xb6351faf "&thread->lock", cond=0xa01cfb4, t=0xa01cf94) at lock.c:547
#4 0xb6336bde in iax2_process_thread (data=0xa01bf00) at chan_iax2.c:11741
#5 0x081a2cc2 in dummy_start (data=0xa018700) at utils.c:1073
#6 0xb72a9d4c in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#7 0xb74ffdde in clone () from /lib/i386-linux-gnu/libc.so.6
Thread 13 (Thread 0xb58b4b40 (LWP 4039)):
#0 0xb77c6424 in __kernel_vsyscall ()
#1 0xb74f1690 in poll () from /lib/i386-linux-gnu/libc.so.6
#2 0x081197be in ast_io_wait (ioc=0x9d93b68, howlong=1000) at io.c:279
#3 0xb6283753 in do_monitor (data=0x0) at chan_sip.c:26547
#4 0x081a2cc2 in dummy_start (data=0x9da3d38) at utils.c:1073
#5 0xb72a9d4c in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#6 0xb74ffdde in clone () from /lib/i386-linux-gnu/libc.so.6
Thread 12 (Thread 0xb47ffb40 (LWP 4049)):
#0 0xb77c6424 in __kernel_vsyscall ()
#1 0xb72ad96b in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
#2 0xb750d6ec in pthread_cond_wait () from /lib/i386-linux-gnu/libc.so.6
#3 0x0811ebec in __ast_cond_wait (filename=0xb6351d54 "chan_iax2.c", lineno=11741, func=0xb6359184 "iax2_process_thread",
cond_name=0xb6351fbd "&thread->cond", mutex_name=0xb6351faf "&thread->lock", cond=0xa0226e4, t=0xa0226c4) at lock.c:547
#4 0xb6336bde in iax2_process_thread (data=0xa021630) at chan_iax2.c:11741
#5 0x081a2cc2 in dummy_start (data=0xa01bab8) at utils.c:1073
#6 0xb72a9d4c in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#7 0xb74ffdde in clone () from /lib/i386-linux-gnu/libc.so.6
Thread 11 (Thread 0xb4403b40 (LWP 4066)):
#0 0xb77c6424 in __kernel_vsyscall ()
#1 0xb72ad96b in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
#2 0xb750d6ec in pthread_cond_wait () from /lib/i386-linux-gnu/libc.so.6
#3 0x0811ebec in __ast_cond_wait (filename=0x8200c98 "taskprocessor.c", lineno=310, func=0x8201336 "tps_processing_function",
cond_name=0x8200fe9 "&i->poll_cond", mutex_name=0x8200fd2 "&i->taskprocessor_lock", cond=0xa0c4f9c, t=0xa0c4fd0) at lock.c:547
#4 0x08192947 in tps_processing_function (data=0xa0c4f98) at taskprocessor.c:310
#5 0x081a2cc2 in dummy_start (data=0xa0cd008) at utils.c:1073
#6 0xb72a9d4c in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#7 0xb74ffdde in clone () from /lib/i386-linux-gnu/libc.so.6
Thread 10 (Thread 0xb470fb40 (LWP 4053)):
#0 0xb77c6424 in __kernel_vsyscall ()
#1 0xb72ad96b in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
#2 0xb750d6ec in pthread_cond_wait () from /lib/i386-linux-gnu/libc.so.6
#3 0x0811ebec in __ast_cond_wait (filename=0xb6351d54 "chan_iax2.c", lineno=11741, func=0xb6359184 "iax2_process_thread",
cond_name=0xb6351fbd "&thread->cond", mutex_name=0xb6351faf "&thread->lock", cond=0xa026c64, t=0xa026c44) at lock.c:547
#4 0xb6336bde in iax2_process_thread (data=0xa025bb0) at chan_iax2.c:11741
#5 0x081a2cc2 in dummy_start (data=0xa019818) at utils.c:1073
#6 0xb72a9d4c in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#7 0xb74ffdde in clone () from /lib/i386-linux-gnu/libc.so.6
Thread 9 (Thread 0xb44f3b40 (LWP 4062)):
#0 0xb77c6424 in __kernel_vsyscall ()
#1 0xb74c8f26 in nanosleep () from /lib/i386-linux-gnu/libc.so.6
#2 0xb74c8d1f in sleep () from /lib/i386-linux-gnu/libc.so.6
#3 0xb618fd87 in process_clearcache (ignore=0x0) at pbx_dundi.c:2211
#4 0x081a2cc2 in dummy_start (data=0xa0912f8) at utils.c:1073
#5 0xb72a9d4c in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#6 0xb74ffdde in clone () from /lib/i386-linux-gnu/libc.so.6
Thread 8 (Thread 0xb583cb40 (LWP 4042)):
#0 0xb77c6424 in __kernel_vsyscall ()
#1 0xb72ad96b in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
#2 0xb750d6ec in pthread_cond_wait () from /lib/i386-linux-gnu/libc.so.6
#3 0x0811ebec in __ast_cond_wait (filename=0x8200c98 "taskprocessor.c", lineno=310, func=0x8201336 "tps_processing_function",
cond_name=0x8200fe9 "&i->poll_cond", mutex_name=0x8200fd2 "&i->taskprocessor_lock", cond=0xa015cf4, t=0xa015d28) at lock.c:547
#4 0x08192947 in tps_processing_function (data=0xa015cf0) at taskprocessor.c:310
#5 0x081a2cc2 in dummy_start (data=0xa015d90) at utils.c:1073
#6 0xb72a9d4c in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#7 0xb74ffdde in clone () from /lib/i386-linux-gnu/libc.so.6
Thread 7 (Thread 0xb5903b40 (LWP 4038)):
#0 0xb77c6424 in __kernel_vsyscall ()
#1 0xb72ad96b in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
#2 0xb750d6ec in pthread_cond_wait () from /lib/i386-linux-gnu/libc.so.6
#3 0x0811ebec in __ast_cond_wait (filename=0xb5cf2274 "res_timing_pthread.c", lineno=481, func=0xb5cf265a "do_timing",
cond_name=0xb5cf22b4 "&timing_thread.cond", mutex_name=0xb5cf22a0 "&timing_thread.lock", cond=0xb5cf4204, t=0xb5cf41e4) at lock.c:547
#4 0xb5cf1cd0 in do_timing (arg=0x0) at res_timing_pthread.c:481
#5 0x081a2cc2 in dummy_start (data=0x9d57d30) at utils.c:1073
#6 0xb72a9d4c in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#7 0xb74ffdde in clone () from /lib/i386-linux-gnu/libc.so.6
Thread 6 (Thread 0xb67c3b40 (LWP 4037)):
#0 0xb77c6424 in __kernel_vsyscall ()
#1 0xb72add13 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
#2 0xb750d744 in pthread_cond_timedwait () from /lib/i386-linux-gnu/libc.so.6
#3 0x0811ec13 in __ast_cond_timedwait (filename=0xb700e0a8 "res_calendar.c", lineno=1695, func=0xb700f260 "do_refresh",
cond_name=0xb700e71a "&refresh_condition", mutex_name=0xb700e5f1 "&refreshlock", cond=0xb70125c0, t=0xb70125a0, abstime=0xb67c32e4) at lock.c:654
#4 0xb700da0e in do_refresh (data=0x0) at res_calendar.c:1695
#5 0x081a2cc2 in dummy_start (data=0x9d57d30) at utils.c:1073
#6 0xb72a9d4c in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#7 0xb74ffdde in clone () from /lib/i386-linux-gnu/libc.so.6
Thread 5 (Thread 0xb45e3b40 (LWP 4058)):
#0 0xb77c6424 in __kernel_vsyscall ()
#1 0xb74f1690 in poll () from /lib/i386-linux-gnu/libc.so.6
#2 0x081197be in ast_io_wait (ioc=0xa06c9d0, howlong=1000) at io.c:279
#3 0xb61cb4db in do_monitor (data=0x0) at chan_unistim.c:4644
#4 0x081a2cc2 in dummy_start (data=0xa061970) at utils.c:1073
#5 0xb72a9d4c in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#6 0xb74ffdde in clone () from /lib/i386-linux-gnu/libc.so.6
Thread 4 (Thread 0xb4c40b40 (LWP 4046)):
#0 0xb77c6424 in __kernel_vsyscall ()
#1 0xb72ad96b in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
#2 0xb750d6ec in pthread_cond_wait () from /lib/i386-linux-gnu/libc.so.6
#3 0x0811ebec in __ast_cond_wait (filename=0xb6351d54 "chan_iax2.c", lineno=11741, func=0xb6359184 "iax2_process_thread",
cond_name=0xb6351fbd "&thread->cond", mutex_name=0xb6351faf "&thread->lock", cond=0xa01f2c4, t=0xa01f2a4) at lock.c:547
#4 0xb6336bde in iax2_process_thread (data=0xa01e210) at chan_iax2.c:11741
#5 0x081a2cc2 in dummy_start (data=0xa018710) at utils.c:1073
#6 0xb72a9d4c in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#7 0xb74ffdde in clone () from /lib/i386-linux-gnu/libc.so.6
Thread 3 (Thread 0xb4ac3b40 (LWP 4045)):
#0 0xb77c6424 in __kernel_vsyscall ()
#1 0xb72ad96b in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
#2 0xb750d6ec in pthread_cond_wait () from /lib/i386-linux-gnu/libc.so.6
#3 0x0811ebec in __ast_cond_wait (filename=0xb6351d54 "chan_iax2.c", lineno=11741, func=0xb6359184 "iax2_process_thread",
cond_name=0xb6351fbd "&thread->cond", mutex_name=0xb6351faf "&thread->lock", cond=0xa01e164, t=0xa01e144) at lock.c:547
#4 0xb6336bde in iax2_process_thread (data=0xa01d0b0) at chan_iax2.c:11741
#5 0x081a2cc2 in dummy_start (data=0xa018710) at utils.c:1073
#6 0xb72a9d4c in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#7 0xb74ffdde in clone () from /lib/i386-linux-gnu/libc.so.6
Thread 2 (Thread 0xb70eeb40 (LWP 4884)):
#0 0xb77c6424 in __kernel_vsyscall ()
#1 0xb74ef48b in read () from /lib/i386-linux-gnu/libc.so.6
#2 0x0818b452 in inotify_daemon (data=0x0) at stdtime/localtime.c:295
#3 0x081a2cc2 in dummy_start (data=0xb66010b8) at utils.c:1073
#4 0xb72a9d4c in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#5 0xb74ffdde in clone () from /lib/i386-linux-gnu/libc.so.6
Thread 1 (Thread 0xb43c7b40 (LWP 4067)):
#0 0xb77c6424 in __kernel_vsyscall ()
#1 0xb743f1df in raise () from /lib/i386-linux-gnu/libc.so.6
#2 0xb7442825 in abort () from /lib/i386-linux-gnu/libc.so.6
#3 0xb747c39a in ?? () from /lib/i386-linux-gnu/libc.so.6
#4 0xb7486ee2 in ?? () from /lib/i386-linux-gnu/libc.so.6
#5 0x08085321 in internal_ao2_ref (user_data=0x9ab24c0, delta=-1) at astobj2.c:286
#6 0x08085223 in __ao2_ref (user_data=0x9ab24c0, delta=-1) at astobj2.c:244
#7 0x08086165 in ao2_iterator_destroy (i=0xb43c7180) at astobj2.c:830
#8 0x081012db in build_dialplan_useage_map (usage_map=0xb43c7200, complain=0) at features.c:6360
#9 0x08101b24 in load_config (reload=1) at features.c:6699
#10 0x08102277 in ast_features_reload () at features.c:6847
#11 0x0811d1ea in ast_module_reload (name=0x0) at loader.c:743
#12 0x08082245 in monitor_sig_flags (unused=0x0) at asterisk.c:3243
#13 0x081a2cc2 in dummy_start (data=0xa0d2b90) at utils.c:1073
#14 0xb72a9d4c in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#15 0xb74ffdde in clone () from /lib/i386-linux-gnu/libc.so.6
(gdb)