AsteriskGUI crashes Asterisk

Asterisk Asterisk Support
1

I recently upgraded from the Asterisk version in the Ubuntu repos to the one in Digium’s repos (Asterisk 1.8.5~rc1-1digium0+1~natty), setup AsteriskGUI again but it now consistently crashes Asterisk on login (it gets stuck on “Updating Extensions.conf”, though as far as I can tell it returns from where it puts up that message and then fails somewhere later on).

Running Asterisk via safe_asterisk produces the following output:

*** glibc detected *** /usr/sbin/asterisk: double free or corruption (!prev): 0x00000000021b7000 *** ======= Backtrace: ========= /lib/x86_64-linux-gnu/libc.so.6(+0x78a8f)[0x7ffeea709a8f] /lib/x86_64-linux-gnu/libc.so.6(cfree+0x73)[0x7ffeea70d8e3] /usr/sbin/asterisk(ast_http_send+0x1db)[0x4a954b] /usr/sbin/asterisk[0x4bd334] /usr/sbin/asterisk[0x4bda11] /usr/sbin/asterisk[0x4aa184] /usr/sbin/asterisk[0x50995d] /usr/sbin/asterisk[0x514297] /lib/x86_64-linux-gnu/libpthread.so.0(+0x6d8c)[0x7ffee9d1cd8c] /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)[0x7ffeea77704d] ======= Memory map: ======== 00400000-005a4000 r-xp 00000000 fb:00 5651582 /usr/sbin/asterisk 007a4000-007a5000 r--p 001a4000 fb:00 5651582 /usr/sbin/asterisk 007a5000-007b1000 rw-p 001a5000 fb:00 5651582 /usr/sbin/asterisk 007b1000-007e8000 rw-p 00000000 00:00 0 01725000-02498000 rw-p 00000000 00:00 0 [heap] 7ffebd66a000-7ffebd66b000 ---p 00000000 00:00 0 7ffebd66b000-7ffebd6e6000 rw-p 00000000 00:00 0 7ffebd6e6000-7ffebd6e7000 ---p 00000000 00:00 0 7ffebd6e7000-7ffebd762000 rw-p 00000000 00:00 0 7ffebd762000-7ffebd763000 ---p 00000000 00:00 0 7ffebd763000-7ffebd7de000 rw-p 00000000 00:00 0 7ffebd7de000-7ffebd7df000 ---p 00000000 00:00 0 7ffebd7df000-7ffebd85a000 rw-p 00000000 00:00 0 7ffebd85a000-7ffebd85b000 ---p 00000000 00:00 0 7ffebd85b000-7ffebd8d6000 rw-p 00000000 00:00 0 7ffebd8d6000-7ffebd8d7000 ---p 00000000 00:00 0 7ffebd8d7000-7ffebd952000 rw-p 00000000 00:00 0 7ffebd952000-7ffebd953000 ---p 00000000 00:00 0 7ffebd953000-7ffebd9ce000 rw-p 00000000 00:00 0 7ffebd9ce000-7ffebd9cf000 ---p 00000000 00:00 0 7ffebd9cf000-7ffebda4a000 rw-p 00000000 00:00 0 7ffebda4a000-7ffebda4b000 ---p 00000000 00:00 0 7ffebda4b000-7ffebdac6000 rw-p 00000000 00:00 0 7ffebdac6000-7ffebdac7000 ---p 00000000 00:00 0 7ffebdac7000-7ffebdb42000 rw-p 00000000 00:00 0 7ffebdb42000-7ffebdb43000 ---p 00000000 00:00 0 7ffebdb43000-7ffebdbbe000 rw-p 00000000 00:00 0 7ffebdbbe000-7ffebdbbf000 ---p 00000000 00:00 0 7ffebdbbf000-7ffebdc3a000 rw-p 00000000 00:00 0 7ffebdc3a000-7ffebdc3b000 ---p 00000000 00:00 0 7ffebdc3b000-7ffebdcb6000 rw-p 00000000 00:00 0 7ffebdcb6000-7ffebdcb7000 ---p 00000000 00:00 0 7ffebdcb7000-7ffebdd32000 rw-p 00000000 00:00 0 7ffebdd32000-7ffebdd33000 ---p 00000000 00:00 0 7ffebdd33000-7ffebddae000 rw-p 00000000 00:00 0 7ffebddae000-7ffebddaf000 ---p 00000000 00:00 0 7ffebddaf000-7ffebde2a000 rw-p 00000000 00:00 0 7ffebde2a000-7ffebde2b000 ---p 00000000 00:00 0 7ffebde2b000-7ffebdea6000 rw-p 00000000 00:00 0 7ffebdea6000-7ffebdea7000 ---p 00000000 00:00 0 7ffebdea7000-7ffebdf22000 rw-p 00000000 00:00 0 7ffebdf22000-7ffebdf23000 ---p 00000000 00:00 0 7ffebdf23000-7ffebdf9e000 rw-p 00000000 00:00 0 7ffebdf9e000-7ffebdf9f000 ---p 00000000 00:00 0 7ffebdf9f000-7ffebe01a000 rw-p 00000000 00:00 0 7ffebe01a000-7ffebe01b000 ---p 00000000 00:00 0 7ffebe01b000-7ffebe096000 rw-p 00000000 00:00 0 7ffebe096000-7ffebe097000 ---p 00000000 00:00 0 7ffebe097000-7ffebe112000 rw-p 00000000 00:00 0 7ffebe112000-7ffebe113000 ---p 00000000 00:00 0 7ffebe113000-7ffebe18e000 rw-p 00000000 00:00 0 7ffebe18e000-7ffebe18f000 ---p 00000000 00:00 0 7ffebe18f000-7ffebe20a000 rw-p 00000000 00:00 0 7ffebe20a000-7ffebe20b000 ---p 00000000 00:00 0 7ffebe20b000-7ffebe286000 rw-p 00000000 00:00 0 7ffebe286000-7ffebe287000 ---p 00000000 00:00 0 7ffebe287000-7ffebe302000 rw-p 00000000 00:00 0 7ffebe302000-7ffebe303000 ---p 00000000 00:00 0 7ffebe303000-7ffebe37e000 rw-p 00000000 00:00 0 7ffebe37e000-7ffebe37f000 ---p 00000000 00:00 0 7ffebe37f000-7ffebe47b000 rw-p 00000000 00:00 0 7ffebe47b000-7ffebe487000 r-xp 00000000 fb:00 5769814 /lib/x86_64-linux-gnu/libnss_files-2.13.so 7ffebe487000-7ffebe686000 ---p 0000c000 fb:00 5769814 /lib/x86_64-linux-gnu/libnss_files-2.13.so 7ffebe686000-7ffebe687000 r--p 0000b000 fb:00 5769814 /lib/x86_64-linux-gnu/libnss_files-2.13.so 7ffebe687000-7ffebe688000 rw-p 0000c000 fb:00 5769814 /lib/x86_64-linux-gnu/libnss_files-2.13.so 7ffebe688000-7ffebe68a000 r-xp 00000000 fb:00 5903454 /usr/lib/asterisk/modules/res_clialiases.so 7ffebe68a000-7ffebe889000 ---p 00002000 fb:00 5903454 /usr/lib/asterisk/modules/res_clialiases.so 7ffebe889000-7ffebe88a000 r--p 00001000 fb:00 5903454 /usr/lib/asterisk/modules/res_clialiases.so 7ffebe88a000-7ffebe88b000 rw-p 00002000 fb:00 5903454 /usr/lib/asterisk/modules/res_clialiases.so 7ffebe88b000-7ffebe892000 r-xp 00000000 fb:00 5903093 /usr/lib/asterisk/modules/res_calendar_ews.so 7ffebe892000-7ffebea91000 ---p 00007000 fb:00 5903093 /usr/lib/asterisk/modules/res_calendar_ews.so 7ffebea91000-7ffebea92000 r--p 00006000 fb:00 5903093 /usr/lib/asterisk/modules/res_calendar_ews.so 7ffebea92000-7ffebea93000 rw-p 00007000 fb:00 5903093 /usr/lib/asterisk/modules/res_calendar_ews.so 7ffebea93000-7ffebea95000 r-xp 00000000 fb:00 5903474 /usr/lib/asterisk/modules/format_vox.so 7ffebea95000-7ffebec94000 ---p 00002000 fb:00 5903474 /usr/lib/asterisk/modules/format_vox.so 7ffebec94000-7ffebec95000 r--p 00001000 fb:00 5903474 /usr/lib/asterisk/modules/format_vox.so 7ffebec95000-7ffebec96000 rw-p 00002000 fb:00 5903474 /usr/lib/asterisk/modules/format_vox.so 7ffebec96000-7ffebec9c000 r-xp 00000000 fb:00 5903223 /usr/lib/asterisk/modules/format_wav_gsm.so 7ffebec9c000-7ffebee9b000 ---p 00006000 fb:00 5903223 /usr/lib/asterisk/modules/format_wav_gsm.so 7ffebee9b000-7ffebee9c000 r--p 00005000 fb:00 5903223 /usr/lib/asterisk/modules/format_wav_gsm.so 7ffebee9c000-7ffebee9d000 rw-p 00006000 fb:00 5903223 /usr/lib/asterisk/modules/format_wav_gsm.so 7ffebee9d000-7ffebee9f000 r-xp 00000000 fb:00 5903227 /usr/lib/asterisk/modules/format_siren7.so 7ffebee9f000-7ffebf09e000 ---p 00002000 fb:00 5903227 /usr/lib/asterisk/modules/format_siren7.so 7ffebf09e000-7ffebf09f000 r--p 00001000 fb:00 5903227 /usr/lib/asterisk/modules/format_siren7.so 7ffebf09f000-7ffebf0a0000 rw-p 00002000 fb:00 5903227 /usr/lib/asterisk/modules/format_siren7.so 7ffebf0a0000-7ffebf0a3000 r-xp 00000000 fb:00 5903120 /usr/lib/asterisk/modules/app_talkdetect.so 7ffebf0a3000-7ffebf2a2000 ---p 00003000 fb:00 5903120 /usr/lib/asterisk/modules/app_talkdetect.so 7ffebf2a2000-7ffebf2a3000 r--p 00002000 fb:00 5903120 /usr/lib/asterisk/modules/app_talkdetect.so 7ffebf2a3000-7ffebf2a4000 rw-p 00003000 fb:00 5903120 /usr/lib/asterisk/modules/app_talkdetect.so 7ffebf2a4000-7ffebf2a6000 r-xp 00000000 fb:00 5903094 /usr/lib/asterisk/modules/format_h263.so 7ffebf2a6000-7ffebf4a5000 ---p 00002000 fb:00 5903094 /usr/lib/asterisk/modules/format_h263.so 7ffebf4a5000-7ffebf4a6000 r--p 00001000 fb:00 5903094 /usr/lib/asterisk/modules/format_h263.so 7ffebf4a6000-7ffebf4a7000 rw-p 00002000 fb:00 5903094 /usr/lib/asterisk/modules/format_h263.soAsterisk ended with exit status 134 Asterisk exited on signal 6.

The machine it runs on is this:

Has someone come across a similar problem?

2

Please provide this report to the issue tracker:

issues.asterisk.org/jira

Guidelines for issue reporting are available here:

asterisk.org/developers/bug-guidelines

3

In particular, this is a memory management problem, and one that happens before significant CPU resources are used, so you should follow the procedures for using valgrind.

4

Done, issue reported here: issues.asterisk.org/jira/browse/ASTERISK-18133 .

5

Many thanks. Please bear with the community, as a large workload makes response times to issue reports less than ideal.

6

Same on the below environment.

CentOS release 5.5 (Final) / Kernel - 2.6.18-238.12.1.el5 / Asterisk Version - 1.8.5.0 / Asterisk GUI version - asterisk-gui-2.1.0-0.rc1.1_centos5. Even tried with SVN code for GUI but no luck. This issue was not present in asterisk 1.8.3 and asterisk-gui-2.1.0-0.rc1.1_centos5 on the same environment. Asterisk and related components were installed through YUM repository. I hope we have a solution sooner on this.

Thanks in advance.

7

You need to provide the additional information on the issue report, not here. You should also attach the crash dump to that, not refer to it here.

I’d also strongly advise running the memory management bugs procedure and attaching the valgrind output to the bug report.

Even then, unless there start to be a lot of "me too"s, don’t expect it to get fixed for some time.

8

Problem is resolved by r327950. Install that fix and the problem goes away.

9

Running latest version from repository and get this? Also running latest version of AsteriskGUI. Same fix applies?

# *** glibc detected *** /usr/sbin/asterisk: double free or corruption (!prev): 0x08fa1328 ***
======= Backtrace: =========
/lib/libc.so.6[0x9bf6c5]
/lib/libc.so.6(cfree+0x59)[0x9bfb09]
/usr/sbin/asterisk(ast_http_send+0x199)[0x810bdc9]
/usr/sbin/asterisk[0x8125d4f]
/usr/sbin/asterisk[0x8126480]
/usr/sbin/asterisk[0x810e629]
/usr/sbin/asterisk[0x8183732]
/usr/sbin/asterisk[0x818fe8b]
/lib/libpthread.so.0[0xabd832]
/lib/libc.so.6(clone+0x5e)[0xa2845e]
======= Memory map: ========
00101000-00195000 r-xp 00000000 08:05 44417733   /usr/lib/libkrb5.so.3.3
00195000-00198000 rwxp 00093000 08:05 44417733   /usr/lib/libkrb5.so.3.3
00198000-00199000 r-xp 00000000 08:05 47480870   /usr/lib/asterisk/modules/app_echo.so
00199000-0019a000 rwxp 00000000 08:05 47480870   /usr/lib/asterisk/modules/app_echo.so
0019a000-001c7000 r-xp 00000000 08:05 44417734   /usr/lib/libgssapi_krb5.so.2.2
001c7000-001c8000 rwxp 0002d000 08:05 44417734   /usr/lib/libgssapi_krb5.so.2.2
001c8000-001c9000 r-xp 00000000 08:05 47481542   /usr/lib/asterisk/modules/func_sysinfo.so
001c9000-001ca000 rwxp 00001000 08:05 47481542   /usr/lib/asterisk/modules/func_sysinfo.so
001ca000-001f0000 r-xp 00000000 08:05 44417732   /usr/lib/libk5crypto.so.3.1
001f0000-001f1000 rwxp 00025000 08:05 44417732   /usr/lib/libk5crypto.so.3.1
001f1000-001f2000 r-xp 00000000 08:05 47481458   /usr/lib/asterisk/modules/app_waitforring.so
001f2000-001f3000 rwxp 00001000 08:05 47481458   /usr/lib/asterisk/modules/app_waitforring.so
001f3000-00237000 r-xp 00000000 08:05 138806530  /lib/libssl.so.0.9.8e
00237000-0023b000 rwxp 00043000 08:05 138806530  /lib/libssl.so.0.9.8e
0023b000-00276000 r-xp 00000000 08:05 138806525  /lib/libsepol.so.1
00276000-00277000 rwxp 0003b000 08:05 138806525  /lib/libsepol.so.1
00277000-00281000 rwxp 00277000 00:00 0
00281000-00283000 r-xp 00000000 08:05 47481573   /usr/lib/asterisk/modules/res_stun_monitor.so
00283000-00284000 rwxp 00002000 08:05 47481573   /usr/lib/asterisk/modules/res_stun_monitor.so
00284000-00294000 r-xp 00000000 08:05 47481555   /usr/lib/asterisk/modules/res_agi.so
00294000-00296000 rwxp 0000f000 08:05 47481555   /usr/lib/asterisk/modules/res_agi.so
00296000-002b5000 r-xp 00000000 08:05 47481390   /usr/lib/asterisk/modules/res_ael_share.so
002b5000-002b6000 rwxp 0001f000 08:05 47481390   /usr/lib/asterisk/modules/res_ael_share.so
002b6000-002b7000 rwxp 002b6000 00:00 0
002b7000-002bc000 r-xp 00000000 08:05 47481563   /usr/lib/asterisk/modules/res_monitor.so
002bc000-002bd000 rwxp 00005000 08:05 47481563   /usr/lib/asterisk/modules/res_monitor.so
002bd000-002c2000 r-xp 00000000 08:05 47481577   /usr/lib/asterisk/modules/app_mysql.so
002c2000-002c3000 rwxp 00004000 08:05 47481577   /usr/lib/asterisk/modules/app_mysql.so
002c3000-002c6000 r-xp 00000000 08:05 47481519   /usr/lib/asterisk/modules/func_devstate.so
002c6000-002c7000 rwxp 00002000 08:05 47481519   /usr/lib/asterisk/modules/func_devstate.so
002c7000-002c9000 r-xp 00000000 08:05 47481534   /usr/lib/asterisk/modules/func_pitchshift.so
002c9000-002ca000 rwxp 00002000 08:05 47481534   /usr/lib/asterisk/modules/func_pitchshift.so
002ca000-002cd000 r-xp 00000000 08:05 47481574   /usr/lib/asterisk/modules/res_timing_pthread.so
002cd000-002ce000 rwxp 00002000 08:05 47481574   /usr/lib/asterisk/modules/res_timing_pthread.so
002ce000-002d1000 r-xp 00000000 08:05 47480868   /usr/lib/asterisk/modules/app_disa.so
002d1000-002d2000 rwxp 00002000 08:05 47480868   /usr/lib/asterisk/modules/app_disa.so
002d2000-002d9000 r-xp 00000000 08:05 47481566   /usr/lib/asterisk/modules/res_phoneprov.so
002d9000-002da000 rwxp 00006000 08:05 47481566   /usr/lib/asterisk/modules/res_phoneprov.so
002da000-002dc000 r-xp 00000000 08:05 47481497   /usr/lib/asterisk/modules/format_h264.so
002dc000-002dd000 rwxp 00001000 08:05 47481497   /usr/lib/asterisk/modules/format_h264.so
002dd000-002e3000 r-xp 00000000 08:05 47481578   /usr/lib/asterisk/modules/cdr_mysql.so
002e3000-002e4000 rwxp 00006000 08:05 47481578   /usr/lib/asterisk/modules/cdr_mysql.so
002e4000-00302000 r-xp 00000000 08:/usr/sbin/safe_asterisk: line 145: 22952 Aborted                 (core dumped) nice -n $PRIORITY ${ASTSBINDIR}/asterisk -f ${CLIARGS} ${ASTARGS} > /dev/${TTY} 2>&1 < /dev/${TTY}
Asterisk ended with exit status 134
Asterisk exited on signal 6.
Automatically restarting Asterisk.
10

You’ve got memory corruption.

You definitely need to analyze the core file properly wiki.asterisk.org/wiki/display/ … +Backtrace

If you can reproduce it, you will get encouraged to use valgrind wiki.asterisk.org/wiki/display/AST/Valgrind This may be difficult for performance reasons, or because it is difficult to reproduce the crash, but you may not have much success in gettin the bug progressed without it.

You should look in the log files, to see if there are any locking problems reported in the lead up to the crash. These are often more useful than the above, but you may have trouble getting the bug marshalls to accept that.

11

[quote=“david55”]You’ve got memory corruption.

You definitely need to analyze the core file properly wiki.asterisk.org/wiki/display/ … +Backtrace

If you can reproduce it, you will get encouraged to use valgrind wiki.asterisk.org/wiki/display/AST/Valgrind This may be difficult for performance reasons, or because it is difficult to reproduce the crash, but you may not have much success in gettin the bug progressed without it.

You should look in the log files, to see if there are any locking problems reported in the lead up to the crash. These are often more useful than the above, but you may have trouble getting the bug marshalls to accept that.[/quote]

I can reproduce it every time, just have to logon to AsteriskGUI.

12

Followed info for back trace.


[root@eros ~]# gdb -se "asterisk" -c core.25377 | tee /tmp/backtrace.txt
GNU gdb (GDB) Red Hat Enterprise Linux (7.0.1-32.el5_6.2)
Copyright (C) 2009 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "i386-redhat-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /usr/sbin/asterisk...(no debugging symbols found)...done.

warning: .dynamic section for "/usr/lib/libtonezone.so.2.0" is not at the expected address

warning: difference appears to be caused by prelink, adjusting expectations

warning: .dynamic section for "/usr/lib/libopenr2.so.1" is not at the expected address

warning: difference appears to be caused by prelink, adjusting expectations
[New Thread 25419]
[New Thread 25418]
[New Thread 25417]
[New Thread 25416]
[New Thread 25415]
[New Thread 25414]
[New Thread 25413]
[New Thread 25412]
[New Thread 25411]
[New Thread 25410]
[New Thread 25409]
[New Thread 25408]
[New Thread 25407]
[New Thread 25406]
[New Thread 25405]
[New Thread 25404]
[New Thread 25403]
[New Thread 25402]
[New Thread 25401]
[New Thread 25400]
[New Thread 25399]
[New Thread 25398]
[New Thread 25397]
[New Thread 25396]
[New Thread 25395]
[New Thread 25394]
[New Thread 25393]
[New Thread 25392]
[New Thread 25391]
[New Thread 25390]
[New Thread 25389]
[New Thread 25388]
[New Thread 25387]
[New Thread 25386]
[New Thread 25385]
[New Thread 25384]
[New Thread 25383]
[New Thread 25382]
[New Thread 25381]
[New Thread 25380]
[New Thread 25379]
[New Thread 25378]
[New Thread 25377]
Reading symbols from /lib/libssl.so.6...(no debugging symbols found)...done.
Loaded symbols for /lib/libssl.so.6
Reading symbols from /lib/libcrypto.so.6...(no debugging symbols found)...done.
Loaded symbols for /lib/libcrypto.so.6
Reading symbols from /lib/libc.so.6...(no debugging symbols found)...done.
Loaded symbols for /lib/libc.so.6
Reading symbols from /usr/lib/libxml2.so.2...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libxml2.so.2
Reading symbols from /usr/lib/libz.so.1...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libz.so.1
Reading symbols from /lib/libm.so.6...(no debugging symbols found)...done.
Loaded symbols for /lib/libm.so.6
Reading symbols from /lib/libdl.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib/libdl.so.2
Reading symbols from /lib/libpthread.so.0...(no debugging symbols found)...done.
Loaded symbols for /lib/libpthread.so.0
Reading symbols from /usr/lib/libncurses.so.5...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libncurses.so.5
Reading symbols from /lib/libresolv.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib/libresolv.so.2
Reading symbols from /usr/lib/libgssapi_krb5.so.2...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libgssapi_krb5.so.2
Reading symbols from /usr/lib/libkrb5.so.3...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libkrb5.so.3
Reading symbols from /lib/libcom_err.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib/libcom_err.so.2
Reading symbols from /usr/lib/libk5crypto.so.3...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libk5crypto.so.3
Reading symbols from /lib/ld-linux.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib/ld-linux.so.2
Reading symbols from /usr/lib/libkrb5support.so.0...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libkrb5support.so.0
Reading symbols from /lib/libkeyutils.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib/libkeyutils.so.1
Reading symbols from /lib/libselinux.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib/libselinux.so.1
Reading symbols from /lib/libsepol.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib/libsepol.so.1
Reading symbols from /usr/lib/asterisk/modules/res_stun_monitor.so...done.
Loaded symbols for /usr/lib/asterisk/modules/res_stun_monitor.so
Reading symbols from /usr/lib/asterisk/modules/res_calendar.so...done.
Loaded symbols for /usr/lib/asterisk/modules/res_calendar.so
Reading symbols from /usr/lib/asterisk/modules/res_agi.so...done.
Loaded symbols for /usr/lib/asterisk/modules/res_agi.so
Reading symbols from /usr/lib/asterisk/modules/res_fax.so...done.
Loaded symbols for /usr/lib/asterisk/modules/res_fax.so
Reading symbols from /usr/lib/asterisk/modules/res_smdi.so...done.
Loaded symbols for /usr/lib/asterisk/modules/res_smdi.so
Reading symbols from /usr/lib/asterisk/modules/res_speech.so...done.
Loaded symbols for /usr/lib/asterisk/modules/res_speech.so
Reading symbols from /usr/lib/asterisk/modules/res_ael_share.so...done.
Loaded symbols for /usr/lib/asterisk/modules/res_ael_share.so
Reading symbols from /usr/lib/asterisk/modules/res_crypto.so...done.
Loaded symbols for /usr/lib/asterisk/modules/res_crypto.so
Reading symbols from /usr/lib/asterisk/modules/res_monitor.so...done.
Loaded symbols for /usr/lib/asterisk/modules/res_monitor.so
Reading symbols from /usr/lib/asterisk/modules/res_musiconhold.so...done.
Loaded symbols for /usr/lib/asterisk/modules/res_musiconhold.so
Reading symbols from /usr/lib/asterisk/modules/func_global.so...done.
Loaded symbols for /usr/lib/asterisk/modules/func_global.so
Reading symbols from /usr/lib/asterisk/modules/codec_g722.so...done.
Loaded symbols for /usr/lib/asterisk/modules/codec_g722.so
Reading symbols from /usr/lib/asterisk/modules/res_realtime.so...done.
Loaded symbols for /usr/lib/asterisk/modules/res_realtime.so
Reading symbols from /usr/lib/asterisk/modules/app_stack.so...done.
Loaded symbols for /usr/lib/asterisk/modules/app_stack.so
Reading symbols from /usr/lib/asterisk/modules/format_g719.so...done.
Loaded symbols for /usr/lib/asterisk/modules/format_g719.so
Reading symbols from /usr/lib/asterisk/modules/codec_a_mu.so...done.
Loaded symbols for /usr/lib/asterisk/modules/codec_a_mu.so
Reading symbols from /usr/lib/asterisk/modules/app_mysql.so...done.
Loaded symbols for /usr/lib/asterisk/modules/app_mysql.so
Reading symbols from /usr/lib/mysql/libmysqlclient.so.15...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/mysql/libmysqlclient.so.15
Reading symbols from /lib/libcrypt.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib/libcrypt.so.1
Reading symbols from /lib/libnsl.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib/libnsl.so.1
Reading symbols from /usr/lib/asterisk/modules/chan_bridge.so...done.
Loaded symbols for /usr/lib/asterisk/modules/chan_bridge.so
Reading symbols from /usr/lib/asterisk/modules/func_devstate.so...done.
Loaded symbols for /usr/lib/asterisk/modules/func_devstate.so
Reading symbols from /usr/lib/asterisk/modules/res_timing_dahdi.so...done.
Loaded symbols for /usr/lib/asterisk/modules/res_timing_dahdi.so
Reading symbols from /usr/lib/asterisk/modules/app_url.so...done.
Loaded symbols for /usr/lib/asterisk/modules/app_url.so
Reading symbols from /usr/lib/asterisk/modules/app_festival.so...done.
Loaded symbols for /usr/lib/asterisk/modules/app_festival.so
Reading symbols from /usr/lib/asterisk/modules/format_pcm.so...done.
Loaded symbols for /usr/lib/asterisk/modules/format_pcm.so
Reading symbols from /usr/lib/asterisk/modules/func_pitchshift.so...done.
Loaded symbols for /usr/lib/asterisk/modules/func_pitchshift.so
Reading symbols from /usr/lib/asterisk/modules/app_system.so...done.
Loaded symbols for /usr/lib/asterisk/modules/app_system.so
Reading symbols from /usr/lib/asterisk/modules/res_timing_pthread.so...done.
Loaded symbols for /usr/lib/asterisk/modules/res_timing_pthread.so
Reading symbols from /usr/lib/asterisk/modules/app_disa.so...done.
Loaded symbols for /usr/lib/asterisk/modules/app_disa.so
Reading symbols from /usr/lib/asterisk/modules/res_config_mysql.so...done.
Loaded symbols for /usr/lib/asterisk/modules/res_config_mysql.so
Reading symbols from /usr/lib/asterisk/modules/res_phoneprov.so...done.
Loaded symbols for /usr/lib/asterisk/modules/res_phoneprov.so
Reading symbols from /usr/lib/asterisk/modules/app_echo.so...done.
Loaded symbols for /usr/lib/asterisk/modules/app_echo.so
Reading symbols from /usr/lib/asterisk/modules/format_h264.so...done.
Loaded symbols for /usr/lib/asterisk/modules/format_h264.so
Reading symbols from /usr/lib/asterisk/modules/cdr_mysql.so...done.
Loaded symbols for /usr/lib/asterisk/modules/cdr_mysql.so
Reading symbols from /usr/lib/asterisk/modules/app_followme.so...done.
Loaded symbols for /usr/lib/asterisk/modules/app_followme.so
Reading symbols from /usr/lib/asterisk/modules/res_limit.so...done.
Loaded symbols for /usr/lib/asterisk/modules/res_limit.so
Reading symbols from /usr/lib/asterisk/modules/pbx_dundi.so...done.
Loaded symbols for /usr/lib/asterisk/modules/pbx_dundi.so
Reading symbols from /usr/lib/asterisk/modules/func_cut.so...done.
Loaded symbols for /usr/lib/asterisk/modules/func_cut.so
Reading symbols from /usr/lib/asterisk/modules/res_rtp_multicast.so...done.
Loaded symbols for /usr/lib/asterisk/modules/res_rtp_multicast.so
Reading symbols from /usr/lib/asterisk/modules/cdr_custom.so...done.
Loaded symbols for /usr/lib/asterisk/modules/cdr_custom.so
Reading symbols from /usr/lib/asterisk/modules/codec_adpcm.so...done.
Loaded symbols for /usr/lib/asterisk/modules/codec_adpcm.so
Reading symbols from /usr/lib/asterisk/modules/app_milliwatt.so...done.
Loaded symbols for /usr/lib/asterisk/modules/app_milliwatt.so
Reading symbols from /usr/lib/asterisk/modules/codec_dahdi.so...done.
Loaded symbols for /usr/lib/asterisk/modules/codec_dahdi.so
Reading symbols from /usr/lib/asterisk/modules/func_shell.so...done.
Loaded symbols for /usr/lib/asterisk/modules/func_shell.so
Reading symbols from /usr/lib/asterisk/modules/app_softhangup.so...done.
Loaded symbols for /usr/lib/asterisk/modules/app_softhangup.so
Reading symbols from /usr/lib/asterisk/modules/func_sysinfo.so...done.
Loaded symbols for /usr/lib/asterisk/modules/func_sysinfo.so
Reading symbols from /usr/lib/asterisk/modules/func_srv.so...done.
Loaded symbols for /usr/lib/asterisk/modules/func_srv.so
Reading symbols from /usr/lib/asterisk/modules/cdr_syslog.so...done.
Loaded symbols for /usr/lib/asterisk/modules/cdr_syslog.so
Reading symbols from /usr/lib/asterisk/modules/app_waitforring.so...done.
Loaded symbols for /usr/lib/asterisk/modules/app_waitforring.so
Reading symbols from /usr/lib/asterisk/modules/format_wav_gsm.so...done.
Loaded symbols for /usr/lib/asterisk/modules/format_wav_gsm.so
Reading symbols from /usr/lib/asterisk/modules/codec_alaw.so...done.
Loaded symbols for /usr/lib/asterisk/modules/codec_alaw.so
Reading symbols from /usr/lib/asterisk/modules/func_sha1.so...done.
Loaded symbols for /usr/lib/asterisk/modules/func_sha1.so
Reading symbols from /usr/lib/asterisk/modules/chan_oss.so...done.
Loaded symbols for /usr/lib/asterisk/modules/chan_oss.so
Reading symbols from /usr/lib/asterisk/modules/app_amd.so...done.
Loaded symbols for /usr/lib/asterisk/modules/app_amd.so
Reading symbols from /usr/lib/asterisk/modules/pbx_realtime.so...done.
Loaded symbols for /usr/lib/asterisk/modules/pbx_realtime.so
Reading symbols from /usr/lib/asterisk/modules/func_module.so...done.
Loaded symbols for /usr/lib/asterisk/modules/func_module.so
Reading symbols from /usr/lib/asterisk/modules/app_confbridge.so...done.
Loaded symbols for /usr/lib/asterisk/modules/app_confbridge.so
Reading symbols from /usr/lib/asterisk/modules/func_extstate.so...done.
Loaded symbols for /usr/lib/asterisk/modules/func_extstate.so
Reading symbols from /usr/lib/asterisk/modules/app_playtones.so...done.
Loaded symbols for /usr/lib/asterisk/modules/app_playtones.so
Reading symbols from /usr/lib/asterisk/modules/app_privacy.so...done.
Loaded symbols for /usr/lib/asterisk/modules/app_privacy.so
Reading symbols from /usr/lib/asterisk/modules/format_vox.so...done.
Loaded symbols for /usr/lib/asterisk/modules/format_vox.so
Reading symbols from /usr/lib/asterisk/modules/app_senddtmf.so...done.
Loaded symbols for /usr/lib/asterisk/modules/app_senddtmf.so
Reading symbols from /usr/lib/asterisk/modules/app_dahdibarge.so...done.
Loaded symbols for /usr/lib/asterisk/modules/app_dahdibarge.so
Reading symbols from /usr/lib/asterisk/modules/cel_manager.so...done.
Loaded symbols for /usr/lib/asterisk/modules/cel_manager.so
Reading symbols from /usr/lib/asterisk/modules/func_realtime.so...done.
Loaded symbols for /usr/lib/asterisk/modules/func_realtime.so
Reading symbols from /usr/lib/asterisk/modules/chan_dahdi.so...done.
Loaded symbols for /usr/lib/asterisk/modules/chan_dahdi.so
Reading symbols from /usr/lib/libtonezone.so.2.0...done.
Loaded symbols for /usr/lib/libtonezone.so.2.0
Reading symbols from /usr/lib/libpri.so.1.4...done.
Loaded symbols for /usr/lib/libpri.so.1.4
Reading symbols from /usr/lib/libss7.so.1...done.
Loaded symbols for /usr/lib/libss7.so.1
Reading symbols from /usr/lib/libopenr2.so.1...done.
Loaded symbols for /usr/lib/libopenr2.so.1
Reading symbols from /usr/lib/asterisk/modules/pbx_loopback.so...done.
Loaded symbols for /usr/lib/asterisk/modules/pbx_loopback.so
Reading symbols from /usr/lib/asterisk/modules/format_g726.so...done.
Loaded symbols for /usr/lib/asterisk/modules/format_g726.so
Reading symbols from /usr/lib/asterisk/modules/app_alarmreceiver.so...done.
Loaded symbols for /usr/lib/asterisk/modules/app_alarmreceiver.so
Reading symbols from /usr/lib/asterisk/modules/app_test.so...done.
Loaded symbols for /usr/lib/asterisk/modules/app_test.so
Reading symbols from /usr/lib/asterisk/modules/app_originate.so...done.
Loaded symbols for /usr/lib/asterisk/modules/app_originate.so
Reading symbols from /usr/lib/asterisk/modules/format_siren14.so...done.
Loaded symbols for /usr/lib/asterisk/modules/format_siren14.so
Reading symbols from /usr/lib/asterisk/modules/func_channel.so...done.
Loaded symbols for /usr/lib/asterisk/modules/func_channel.so
Reading symbols from /usr/lib/asterisk/modules/app_readexten.so...done.
Loaded symbols for /usr/lib/asterisk/modules/app_readexten.so
Reading symbols from /usr/lib/asterisk/modules/func_dialgroup.so...done.
Loaded symbols for /usr/lib/asterisk/modules/func_dialgroup.so
Reading symbols from /usr/lib/asterisk/modules/func_config.so...done.
Loaded symbols for /usr/lib/asterisk/modules/func_config.so
Reading symbols from /usr/lib/asterisk/modules/chan_phone.so...done.
Loaded symbols for /usr/lib/asterisk/modules/chan_phone.so
Reading symbols from /usr/lib/asterisk/modules/res_mutestream.so...done.
Loaded symbols for /usr/lib/asterisk/modules/res_mutestream.so
Reading symbols from /usr/lib/asterisk/modules/app_authenticate.so...done.
Loaded symbols for /usr/lib/asterisk/modules/app_authenticate.so
Reading symbols from /usr/lib/asterisk/modules/app_page.so...done.
Loaded symbols for /usr/lib/asterisk/modules/app_page.so
Reading symbols from /usr/lib/asterisk/modules/func_env.so...done.
Loaded symbols for /usr/lib/asterisk/modules/func_env.so
Reading symbols from /usr/lib/asterisk/modules/cdr_manager.so...done.
Loaded symbols for /usr/lib/asterisk/modules/cdr_manager.so
Reading symbols from /usr/lib/asterisk/modules/format_mp3.so...done.
Loaded symbols for /usr/lib/asterisk/modules/format_mp3.so
Reading symbols from /usr/lib/asterisk/modules/app_playback.so...done.
Loaded symbols for /usr/lib/asterisk/modules/app_playback.so
Reading symbols from /usr/lib/asterisk/modules/format_g729.so...done.
Loaded symbols for /usr/lib/asterisk/modules/format_g729.so
Reading symbols from /usr/lib/asterisk/modules/func_callcompletion.so...done.
Loaded symbols for /usr/lib/asterisk/modules/func_callcompletion.so
Reading symbols from /usr/lib/asterisk/modules/func_logic.so...done.
Loaded symbols for /usr/lib/asterisk/modules/func_logic.so
Reading symbols from /usr/lib/asterisk/modules/res_security_log.so...done.
Loaded symbols for /usr/lib/asterisk/modules/res_security_log.so
Reading symbols from /usr/lib/asterisk/modules/app_talkdetect.so...done.
Loaded symbols for /usr/lib/asterisk/modules/app_talkdetect.so
Reading symbols from /usr/lib/asterisk/modules/bridge_builtin_features.so...done.
Loaded symbols for /usr/lib/asterisk/modules/bridge_builtin_features.so
Reading symbols from /usr/lib/asterisk/modules/func_iconv.so...done.
Loaded symbols for /usr/lib/asterisk/modules/func_iconv.so
Reading symbols from /usr/lib/asterisk/modules/chan_local.so...done.
Loaded symbols for /usr/lib/asterisk/modules/chan_local.so
Reading symbols from /usr/lib/asterisk/modules/chan_sip.so...done.
Loaded symbols for /usr/lib/asterisk/modules/chan_sip.so
Reading symbols from /usr/lib/asterisk/modules/cel_custom.so...done.
Loaded symbols for /usr/lib/asterisk/modules/cel_custom.so
Reading symbols from /usr/lib/asterisk/modules/app_celgenuserevent.so...done.
Loaded symbols for /usr/lib/asterisk/modules/app_celgenuserevent.so
Reading symbols from /usr/lib/asterisk/modules/func_blacklist.so...done.
Loaded symbols for /usr/lib/asterisk/modules/func_blacklist.so
Reading symbols from /usr/lib/asterisk/modules/app_waituntil.so...done.
Loaded symbols for /usr/lib/asterisk/modules/app_waituntil.so
Reading symbols from /usr/lib/asterisk/modules/func_audiohookinherit.so...done.
Loaded symbols for /usr/lib/asterisk/modules/func_audiohookinherit.so
Reading symbols from /usr/lib/asterisk/modules/app_mixmonitor.so...done.
Loaded symbols for /usr/lib/asterisk/modules/app_mixmonitor.so
Reading symbols from /usr/lib/asterisk/modules/bridge_multiplexed.so...done.
Loaded symbols for /usr/lib/asterisk/modules/bridge_multiplexed.so
Reading symbols from /usr/lib/asterisk/modules/func_aes.so...done.
Loaded symbols for /usr/lib/asterisk/modules/func_aes.so
Reading symbols from /usr/lib/asterisk/modules/app_mp3.so...done.
Loaded symbols for /usr/lib/asterisk/modules/app_mp3.so
Reading symbols from /usr/lib/asterisk/modules/codec_g726.so...done.
Loaded symbols for /usr/lib/asterisk/modules/codec_g726.so
Reading symbols from /usr/lib/asterisk/modules/app_macro.so...done.
Loaded symbols for /usr/lib/asterisk/modules/app_macro.so
Reading symbols from /usr/lib/asterisk/modules/app_record.so...done.
Loaded symbols for /usr/lib/asterisk/modules/app_record.so
Reading symbols from /usr/lib/asterisk/modules/func_frame_trace.so...done.
Loaded symbols for /usr/lib/asterisk/modules/func_frame_trace.so
Reading symbols from /usr/lib/asterisk/modules/app_saycountpl.so...done.
Loaded symbols for /usr/lib/asterisk/modules/app_saycountpl.so
Reading symbols from /usr/lib/asterisk/modules/app_voicemail.so...done.
Loaded symbols for /usr/lib/asterisk/modules/app_voicemail.so
Reading symbols from /usr/lib/asterisk/modules/func_cdr.so...done.
Loaded symbols for /usr/lib/asterisk/modules/func_cdr.so
Reading symbols from /usr/lib/asterisk/modules/app_userevent.so...done.
Loaded symbols for /usr/lib/asterisk/modules/app_userevent.so
Reading symbols from /usr/lib/asterisk/modules/func_base64.so...done.
Loaded symbols for /usr/lib/asterisk/modules/func_base64.so
Reading symbols from /usr/lib/asterisk/modules/format_siren7.so...done.
Loaded symbols for /usr/lib/asterisk/modules/format_siren7.so
Reading symbols from /usr/lib/asterisk/modules/codec_ulaw.so...done.
Loaded symbols for /usr/lib/asterisk/modules/codec_ulaw.so
Reading symbols from /usr/lib/asterisk/modules/format_sln.so...done.
Loaded symbols for /usr/lib/asterisk/modules/format_sln.so
Reading symbols from /usr/lib/asterisk/modules/format_gsm.so...done.
Loaded symbols for /usr/lib/asterisk/modules/format_gsm.so
Reading symbols from /usr/lib/asterisk/modules/func_callerid.so...done.
Loaded symbols for /usr/lib/asterisk/modules/func_callerid.so
Reading symbols from /usr/lib/asterisk/modules/app_parkandannounce.so...done.
Loaded symbols for /usr/lib/asterisk/modules/app_parkandannounce.so
Reading symbols from /usr/lib/asterisk/modules/app_dial.so...done.
Loaded symbols for /usr/lib/asterisk/modules/app_dial.so
Reading symbols from /usr/lib/asterisk/modules/app_db.so...done.
Loaded symbols for /usr/lib/asterisk/modules/app_db.so
Reading symbols from /usr/lib/asterisk/modules/chan_skinny.so...done.
Loaded symbols for /usr/lib/asterisk/modules/chan_skinny.so
Reading symbols from /usr/lib/asterisk/modules/bridge_softmix.so...done.
Loaded symbols for /usr/lib/asterisk/modules/bridge_softmix.so
Reading symbols from /usr/lib/asterisk/modules/func_dialplan.so...done.
Loaded symbols for /usr/lib/asterisk/modules/func_dialplan.so
Reading symbols from /usr/lib/asterisk/modules/app_cdr.so...done.
Loaded symbols for /usr/lib/asterisk/modules/app_cdr.so
Reading symbols from /usr/lib/asterisk/modules/func_groupcount.so...done.
Loaded symbols for /usr/lib/asterisk/modules/func_groupcount.so
Reading symbols from /usr/lib/asterisk/modules/app_forkcdr.so...done.
Loaded symbols for /usr/lib/asterisk/modules/app_forkcdr.so
Reading symbols from /usr/lib/asterisk/modules/chan_mobile.so...done.
Loaded symbols for /usr/lib/asterisk/modules/chan_mobile.so
Reading symbols from /usr/lib/libbluetooth.so.2...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libbluetooth.so.2
Reading symbols from /usr/lib/asterisk/modules/app_chanisavail.so...done.
Loaded symbols for /usr/lib/asterisk/modules/app_chanisavail.so
Reading symbols from /usr/lib/asterisk/modules/app_readfile.so...done.
Loaded symbols for /usr/lib/asterisk/modules/app_readfile.so
Reading symbols from /usr/lib/asterisk/modules/app_meetme.so...done.
Loaded symbols for /usr/lib/asterisk/modules/app_meetme.so
Reading symbols from /usr/lib/asterisk/modules/cdr_csv.so...done.
Loaded symbols for /usr/lib/asterisk/modules/cdr_csv.so
Reading symbols from /usr/lib/asterisk/modules/chan_iax2.so...done.
Loaded symbols for /usr/lib/asterisk/modules/chan_iax2.so
Reading symbols from /usr/lib/asterisk/modules/func_uri.so...done.
Loaded symbols for /usr/lib/asterisk/modules/func_uri.so
Reading symbols from /usr/lib/asterisk/modules/res_clioriginate.so...done.
Loaded symbols for /usr/lib/asterisk/modules/res_clioriginate.so
Reading symbols from /usr/lib/asterisk/modules/app_channelredirect.so...done.
Loaded symbols for /usr/lib/asterisk/modules/app_channelredirect.so
Reading symbols from /usr/lib/asterisk/modules/chan_unistim.so...done.
Loaded symbols for /usr/lib/asterisk/modules/chan_unistim.so
Reading symbols from /usr/lib/asterisk/modules/format_g723.so...done.
Loaded symbols for /usr/lib/asterisk/modules/format_g723.so
Reading symbols from /usr/lib/asterisk/modules/format_ilbc.so...done.
Loaded symbols for /usr/lib/asterisk/modules/format_ilbc.so
Reading symbols from /usr/lib/asterisk/modules/app_sendtext.so...done.
Loaded symbols for /usr/lib/asterisk/modules/app_sendtext.so
Reading symbols from /usr/lib/asterisk/modules/func_timeout.so...done.
Loaded symbols for /usr/lib/asterisk/modules/func_timeout.so
Reading symbols from /usr/lib/asterisk/modules/app_image.so...done.
Loaded symbols for /usr/lib/asterisk/modules/app_image.so
Reading symbols from /usr/lib/asterisk/modules/app_dictate.so...done.
Loaded symbols for /usr/lib/asterisk/modules/app_dictate.so
Reading symbols from /usr/lib/asterisk/modules/app_dahdiras.so...done.
Loaded symbols for /usr/lib/asterisk/modules/app_dahdiras.so
Reading symbols from /usr/lib/asterisk/modules/func_rand.so...done.
Loaded symbols for /usr/lib/asterisk/modules/func_rand.so
Reading symbols from /usr/lib/asterisk/modules/chan_multicast_rtp.so...done.
Loaded symbols for /usr/lib/asterisk/modules/chan_multicast_rtp.so
Reading symbols from /usr/lib/asterisk/modules/func_md5.so...done.
Loaded symbols for /usr/lib/asterisk/modules/func_md5.so
Reading symbols from /usr/lib/asterisk/modules/app_setcallerid.so...done.
Loaded symbols for /usr/lib/asterisk/modules/app_setcallerid.so
Reading symbols from /usr/lib/asterisk/modules/app_controlplayback.so...done.
Loaded symbols for /usr/lib/asterisk/modules/app_controlplayback.so
Reading symbols from /usr/lib/asterisk/modules/app_waitforsilence.so...done.
Loaded symbols for /usr/lib/asterisk/modules/app_waitforsilence.so
Reading symbols from /usr/lib/asterisk/modules/app_ices.so...done.
Loaded symbols for /usr/lib/asterisk/modules/app_ices.so
Reading symbols from /usr/lib/asterisk/modules/res_clialiases.so...done.
Loaded symbols for /usr/lib/asterisk/modules/res_clialiases.so
Reading symbols from /usr/lib/asterisk/modules/app_externalivr.so...done.
Loaded symbols for /usr/lib/asterisk/modules/app_externalivr.so
Reading symbols from /usr/lib/asterisk/modules/app_minivm.so...done.
Loaded symbols for /usr/lib/asterisk/modules/app_minivm.so
Reading symbols from /usr/lib/asterisk/modules/app_read.so...done.
Loaded symbols for /usr/lib/asterisk/modules/app_read.so
Reading symbols from /usr/lib/asterisk/modules/format_h263.so...done.
Loaded symbols for /usr/lib/asterisk/modules/format_h263.so
Reading symbols from /usr/lib/asterisk/modules/pbx_config.so...done.
Loaded symbols for /usr/lib/asterisk/modules/pbx_config.so
Reading symbols from /usr/lib/asterisk/modules/app_dumpchan.so...done.
Loaded symbols for /usr/lib/asterisk/modules/app_dumpchan.so
Reading symbols from /usr/lib/asterisk/modules/app_directed_pickup.so...done.
Loaded symbols for /usr/lib/asterisk/modules/app_directed_pickup.so
Reading symbols from /usr/lib/asterisk/modules/app_while.so...done.
Loaded symbols for /usr/lib/asterisk/modules/app_while.so
Reading symbols from /usr/lib/asterisk/modules/format_sln16.so...done.
Loaded symbols for /usr/lib/asterisk/modules/format_sln16.so
Reading symbols from /usr/lib/asterisk/modules/app_morsecode.so...done.
Loaded symbols for /usr/lib/asterisk/modules/app_morsecode.so
Reading symbols from /usr/lib/asterisk/modules/app_verbose.so...done.
Loaded symbols for /usr/lib/asterisk/modules/app_verbose.so
Reading symbols from /usr/lib/asterisk/modules/func_lock.so...done.
Loaded symbols for /usr/lib/asterisk/modules/func_lock.so
Reading symbols from /usr/lib/asterisk/modules/func_volume.so...done.
Loaded symbols for /usr/lib/asterisk/modules/func_volume.so
Reading symbols from /usr/lib/asterisk/modules/pbx_spool.so...done.
Loaded symbols for /usr/lib/asterisk/modules/pbx_spool.so
Reading symbols from /usr/lib/asterisk/modules/app_nbscat.so...done.
Loaded symbols for /usr/lib/asterisk/modules/app_nbscat.so
Reading symbols from /usr/lib/asterisk/modules/func_strings.so...done.
Loaded symbols for /usr/lib/asterisk/modules/func_strings.so
Reading symbols from /usr/lib/asterisk/modules/codec_gsm.so...done.
Loaded symbols for /usr/lib/asterisk/modules/codec_gsm.so
Reading symbols from /usr/lib/asterisk/modules/func_enum.so...done.
Loaded symbols for /usr/lib/asterisk/modules/func_enum.so
Reading symbols from /usr/lib/asterisk/modules/res_convert.so...done.
Loaded symbols for /usr/lib/asterisk/modules/res_convert.so
Reading symbols from /usr/lib/asterisk/modules/app_sms.so...done.
Loaded symbols for /usr/lib/asterisk/modules/app_sms.so
Reading symbols from /usr/lib/asterisk/modules/app_chanspy.so...done.
Loaded symbols for /usr/lib/asterisk/modules/app_chanspy.so
Reading symbols from /usr/lib/asterisk/modules/app_getcpeid.so...done.
Loaded symbols for /usr/lib/asterisk/modules/app_getcpeid.so
Reading symbols from /usr/lib/asterisk/modules/chan_agent.so...done.
Loaded symbols for /usr/lib/asterisk/modules/chan_agent.so
Reading symbols from /usr/lib/asterisk/modules/func_vmcount.so...done.
Loaded symbols for /usr/lib/asterisk/modules/func_vmcount.so
Reading symbols from /usr/lib/asterisk/modules/bridge_simple.so...done.
Loaded symbols for /usr/lib/asterisk/modules/bridge_simple.so
Reading symbols from /usr/lib/asterisk/modules/app_zapateller.so...done.
Loaded symbols for /usr/lib/asterisk/modules/app_zapateller.so
Reading symbols from /usr/lib/asterisk/modules/app_exec.so...done.
Loaded symbols for /usr/lib/asterisk/modules/app_exec.so
Reading symbols from /usr/lib/asterisk/modules/app_speech_utils.so...done.
Loaded symbols for /usr/lib/asterisk/modules/app_speech_utils.so
Reading symbols from /usr/lib/asterisk/modules/app_directory.so...done.
Loaded symbols for /usr/lib/asterisk/modules/app_directory.so
Reading symbols from /usr/lib/asterisk/modules/func_sprintf.so...done.
Loaded symbols for /usr/lib/asterisk/modules/func_sprintf.so
Reading symbols from /usr/lib/asterisk/modules/res_rtp_asterisk.so...done.
Loaded symbols for /usr/lib/asterisk/modules/res_rtp_asterisk.so
Reading symbols from /usr/lib/asterisk/modules/app_adsiprog.so...done.
Loaded symbols for /usr/lib/asterisk/modules/app_adsiprog.so
Reading symbols from /usr/lib/asterisk/modules/pbx_ael.so...done.
Loaded symbols for /usr/lib/asterisk/modules/pbx_ael.so
Reading symbols from /usr/lib/asterisk/modules/func_db.so...done.
Loaded symbols for /usr/lib/asterisk/modules/func_db.so
Reading symbols from /usr/lib/asterisk/modules/format_jpeg.so...done.
Loaded symbols for /usr/lib/asterisk/modules/format_jpeg.so
Reading symbols from /usr/lib/asterisk/modules/chan_ooh323.so...done.
Loaded symbols for /usr/lib/asterisk/modules/chan_ooh323.so
Reading symbols from /usr/lib/asterisk/modules/app_flash.so...done.
Loaded symbols for /usr/lib/asterisk/modules/app_flash.so
Reading symbols from /usr/lib/asterisk/modules/func_version.so...done.
Loaded symbols for /usr/lib/asterisk/modules/func_version.so
Reading symbols from /usr/lib/asterisk/modules/func_math.so...done.
Loaded symbols for /usr/lib/asterisk/modules/func_math.so
Reading symbols from /usr/lib/asterisk/modules/format_wav.so...done.
Loaded symbols for /usr/lib/asterisk/modules/format_wav.so
Reading symbols from /usr/lib/asterisk/modules/codec_lpc10.so...done.
Loaded symbols for /usr/lib/asterisk/modules/codec_lpc10.so
Reading symbols from /usr/lib/asterisk/modules/app_queue.so...done.
Loaded symbols for /usr/lib/asterisk/modules/app_queue.so
Reading symbols from /usr/lib/asterisk/modules/app_transfer.so...done.
Loaded symbols for /usr/lib/asterisk/modules/app_transfer.so
Reading symbols from /usr/lib/asterisk/modules/app_sayunixtime.so...done.
Loaded symbols for /usr/lib/asterisk/modules/app_sayunixtime.so
Reading symbols from /lib/libnss_files.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib/libnss_files.so.2
Reading symbols from /lib/libnss_dns.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib/libnss_dns.so.2
Reading symbols from /lib/libgcc_s.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib/libgcc_s.so.1
Core was generated by `asterisk -vvvvvvvvvvvg -c'.
Program terminated with signal 6, Aborted.
#0  0x00261402 in __kernel_vsyscall ()
(gdb)

EDIT Forgot last part of BT


(gdb) thread apply all bt

Thread 44 (Thread 25377):
#0  0x00261402 in __kernel_vsyscall ()
#1  0x00a1900b in read () from /lib/libc.so.6
#2  0x081a13b1 in read_char ()
#3  0x0819c415 in el_getc ()
#4  0x0819c828 in el_gets ()
#5  0x08083152 in main ()

Thread 43 (Thread 25378):
#0  0x00261402 in __kernel_vsyscall ()
#1  0x00a1900b in read () from /lib/libc.so.6
#2  0x0817c9a9 in inotify_daemon ()
#3  0x0818fe8b in dummy_start ()
#4  0x00abd832 in start_thread () from /lib/libpthread.so.0
#5  0x00a2845e in clone () from /lib/libc.so.6

Thread 42 (Thread 25379):
#0  0x00261402 in __kernel_vsyscall ()
#1  0x00ac1bc5 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
#2  0x00a34b2d in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libc.so.6
#3  0x08182d34 in tps_processing_function ()
#4  0x0818fe8b in dummy_start ()
#5  0x00abd832 in start_thread () from /lib/libpthread.so.0
#6  0x00a2845e in clone () from /lib/libc.so.6

Thread 41 (Thread 25380):
#0  0x00261402 in __kernel_vsyscall ()
#1  0x00a1e753 in poll () from /lib/libc.so.6
#2  0x0807d631 in listener ()
#3  0x0818fe8b in dummy_start ()
#4  0x00abd832 in start_thread () from /lib/libpthread.so.0
#5  0x00a2845e in clone () from /lib/libc.so.6

Thread 40 (Thread 25381):
#0  0x00261402 in __kernel_vsyscall ()
#1  0x00ac1bc5 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
#2  0x00a34b2d in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libc.so.6
#3  0x0811b263 in logger_thread ()
#4  0x0818fe8b in dummy_start ()
#5  0x00abd832 in start_thread () from /lib/libpthread.so.0
#6  0x00a2845e in clone () from /lib/libc.so.6

Thread 39 (Thread 25382):
#0  0x00261402 in __kernel_vsyscall ()
#1  0x00a1e753 in poll () from /lib/libc.so.6
#2  0x0818fe23 in ast_wait_for_input ()
#3  0x08183c77 in ast_tcptls_server_root ()
#4  0x0818fe8b in dummy_start ()
#5  0x00abd832 in start_thread () from /lib/libpthread.so.0
#6  0x00a2845e in clone () from /lib/libc.so.6

Thread 38 (Thread 25383):
#0  0x00261402 in __kernel_vsyscall ()
#1  0x00a1e753 in poll () from /lib/libc.so.6
#2  0x0818fe23 in ast_wait_for_input ()
#3  0x08183c77 in ast_tcptls_server_root ()
#4  0x0818fe8b in dummy_start ()
#5  0x00abd832 in start_thread () from /lib/libpthread.so.0
#6  0x00a2845e in clone () from /lib/libc.so.6

Thread 37 (Thread 25384):
#0  0x00261402 in __kernel_vsyscall ()
#1  0x00ac1bc5 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
#2  0x00a34b2d in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libc.so.6
#3  0x080d9978 in do_devstate_changes ()
#4  0x0818fe8b in dummy_start ()
#5  0x00abd832 in start_thread () from /lib/libpthread.so.0
#6  0x00a2845e in clone () from /lib/libc.so.6

Thread 36 (Thread 25385):
#0  0x00261402 in __kernel_vsyscall ()
#1  0x00ac1bc5 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
#2  0x00a34b2d in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libc.so.6
#3  0x08182d34 in tps_processing_function ()
#4  0x0818fe8b in dummy_start ()
#5  0x00abd832 in start_thread () from /lib/libpthread.so.0
#6  0x00a2845e in clone () from /lib/libc.so.6

Thread 35 (Thread 25386):
#0  0x00261402 in __kernel_vsyscall ()
#1  0x00a1e753 in poll () from /lib/libc.so.6
#2  0x080f8b27 in do_parking_thread ()
#3  0x0818fe8b in dummy_start ()
#4  0x00abd832 in start_thread () from /lib/libpthread.so.0
#5  0x00a2845e in clone () from /lib/libc.so.6

Thread 34 (Thread 25387):
#0  0x00261402 in __kernel_vsyscall ()
#1  0x00ac1bc5 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
#2  0x00a34b2d in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libc.so.6
#3  0x080d690e in db_sync_thread ()
#4  0x0818fe8b in dummy_start ()
#5  0x00abd832 in start_thread () from /lib/libpthread.so.0
#6  0x00a2845e in clone () from /lib/libc.so.6

Thread 33 (Thread 25388):
#0  0x00261402 in __kernel_vsyscall ()
#1  0x00ac1bc5 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
#2  0x00a34b2d in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libc.so.6
#3  0x08182d34 in tps_processing_function ()
#4  0x0818fe8b in dummy_start ()
#5  0x00abd832 in start_thread () from /lib/libpthread.so.0
#6  0x00a2845e in clone () from /lib/libc.so.6

Thread 32 (Thread 25389):
#0  0x00261402 in __kernel_vsyscall ()
#1  0x00ac1bc5 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
#2  0x00a34b2d in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libc.so.6
#3  0x0817a2ae in sched_run ()
#4  0x0818fe8b in dummy_start ()
#5  0x00abd832 in start_thread () from /lib/libpthread.so.0
#6  0x00a2845e in clone () from /lib/libc.so.6

Thread 31 (Thread 25390):
#0  0x00261402 in __kernel_vsyscall ()
#1  0x00ac1ef2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
#2  0x00a34b74 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/libc.so.6
#3  0x004b2dda in do_refresh (data=0x0) at res_calendar.c:1677
#4  0x0818fe8b in dummy_start ()
#5  0x00abd832 in start_thread () from /lib/libpthread.so.0
#6  0x00a2845e in clone () from /lib/libc.so.6

Thread 30 (Thread 25391):
#0  0x00261402 in __kernel_vsyscall ()
#1  0x00ac1bc5 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
#2  0x00a34b2d in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libc.so.6
#3  0x0028856d in do_timing (arg=0x0) at res_timing_pthread.c:465
#4  0x0818fe8b in dummy_start ()
#5  0x00abd832 in start_thread () from /lib/libpthread.so.0
#6  0x00a2845e in clone () from /lib/libc.so.6

Thread 29 (Thread 25392):
#0  0x00261402 in __kernel_vsyscall ()
#1  0x00a1e753 in poll () from /lib/libc.so.6
#2  0x08112fc4 in ast_io_wait ()
#3  0x029ecb4c in do_monitor (data=0x0) at chan_sip.c:25004
#4  0x0818fe8b in dummy_start ()
#5  0x00abd832 in start_thread () from /lib/libpthread.so.0
#6  0x00a2845e in clone () from /lib/libc.so.6

Thread 28 (Thread 25393):
#0  0x00261402 in __kernel_vsyscall ()
#1  0x00a1e753 in poll () from /lib/libc.so.6
#2  0x00ffdc2f in do_monitor (data=0x0) at chan_dahdi.c:11530
#3  0x0818fe8b in dummy_start ()
#4  0x00abd832 in start_thread () from /lib/libpthread.so.0
#5  0x00a2845e in clone () from /lib/libc.so.6

Thread 27 (Thread 25394):
#0  0x00261402 in __kernel_vsyscall ()
#1  0x00a290f8 in accept () from /lib/libc.so.6
#2  0x00827726 in accept_thread (ignore=0x0) at chan_skinny.c:6503
#3  0x0818fe8b in dummy_start ()
#4  0x00abd832 in start_thread () from /lib/libpthread.so.0
#5  0x00a2845e in clone () from /lib/libc.so.6

Thread 26 (Thread 25395):
#0  0x00261402 in __kernel_vsyscall ()
#1  0x00a1e753 in poll () from /lib/libc.so.6
#2  0x08112fc4 in ast_io_wait ()
#3  0x00824727 in do_monitor (data=0x0) at chan_skinny.c:6565
#4  0x0818fe8b in dummy_start ()
#5  0x00abd832 in start_thread () from /lib/libpthread.so.0
#6  0x00a2845e in clone () from /lib/libc.so.6

Thread 25 (Thread 25396):
#0  0x00261402 in __kernel_vsyscall ()
#1  0x00ac1bc5 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
#2  0x00a34b2d in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libc.so.6
#3  0x08182d34 in tps_processing_function ()
#4  0x0818fe8b in dummy_start ()
#5  0x00abd832 in start_thread () from /lib/libpthread.so.0
#6  0x00a2845e in clone () from /lib/libc.so.6

Thread 24 (Thread 25397):
#0  0x00261402 in __kernel_vsyscall ()
#1  0x00ac1bc5 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
#2  0x00a34b2d in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libc.so.6
#3  0x0817a2ae in sched_run ()
#4  0x0818fe8b in dummy_start ()
#5  0x00abd832 in start_thread () from /lib/libpthread.so.0
#6  0x00a2845e in clone () from /lib/libc.so.6

Thread 23 (Thread 25398):
#0  0x00261402 in __kernel_vsyscall ()
#1  0x00ac1bc5 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
#2  0x00a34b2d in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libc.so.6
#3  0x0465863f in iax2_process_thread (data=0x949d680) at chan_iax2.c:11622
#4  0x0818fe8b in dummy_start ()
#5  0x00abd832 in start_thread () from /lib/libpthread.so.0
#6  0x00a2845e in clone () from /lib/libc.so.6

Thread 22 (Thread 25399):
#0  0x00261402 in __kernel_vsyscall ()
#1  0x00ac1bc5 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
#2  0x00a34b2d in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libc.so.6
#3  0x0465863f in iax2_process_thread (data=0x949e7e0) at chan_iax2.c:11622
#4  0x0818fe8b in dummy_start ()
#5  0x00abd832 in start_thread () from /lib/libpthread.so.0
#6  0x00a2845e in clone () from /lib/libc.so.6

Thread 21 (Thread 25400):
#0  0x00261402 in __kernel_vsyscall ()
#1  0x00ac1bc5 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
#2  0x00a34b2d in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libc.so.6
#3  0x0465863f in iax2_process_thread (data=0x949f940) at chan_iax2.c:11622
#4  0x0818fe8b in dummy_start ()
#5  0x00abd832 in start_thread () from /lib/libpthread.so.0
#6  0x00a2845e in clone () from /lib/libc.so.6

Thread 20 (Thread 25401):
#0  0x00261402 in __kernel_vsyscall ()
#1  0x00ac1bc5 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
#2  0x00a34b2d in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libc.so.6
#3  0x0465863f in iax2_process_thread (data=0x94a0aa0) at chan_iax2.c:11622
#4  0x0818fe8b in dummy_start ()
#5  0x00abd832 in start_thread () from /lib/libpthread.so.0
#6  0x00a2845e in clone () from /lib/libc.so.6

Thread 19 (Thread 25402):
#0  0x00261402 in __kernel_vsyscall ()
#1  0x00ac1bc5 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
#2  0x00a34b2d in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libc.so.6
#3  0x0465863f in iax2_process_thread (data=0x94a1c00) at chan_iax2.c:11622
#4  0x0818fe8b in dummy_start ()
#5  0x00abd832 in start_thread () from /lib/libpthread.so.0
#6  0x00a2845e in clone () from /lib/libc.so.6

Thread 18 (Thread 25403):
#0  0x00261402 in __kernel_vsyscall ()
#1  0x00ac1bc5 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
#2  0x00a34b2d in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libc.so.6
#3  0x0465863f in iax2_process_thread (data=0x94a2d60) at chan_iax2.c:11622
#4  0x0818fe8b in dummy_start ()
#5  0x00abd832 in start_thread () from /lib/libpthread.so.0
#6  0x00a2845e in clone () from /lib/libc.so.6

Thread 17 (Thread 25404):
#0  0x00261402 in __kernel_vsyscall ()
#1  0x00ac1bc5 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
#2  0x00a34b2d in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libc.so.6
#3  0x0465863f in iax2_process_thread (data=0x94a3ec0) at chan_iax2.c:11622
#4  0x0818fe8b in dummy_start ()
#5  0x00abd832 in start_thread () from /lib/libpthread.so.0
#6  0x00a2845e in clone () from /lib/libc.so.6

Thread 16 (Thread 25405):
#0  0x00261402 in __kernel_vsyscall ()
#1  0x00ac1bc5 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
#2  0x00a34b2d in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libc.so.6
#3  0x0465863f in iax2_process_thread (data=0x94a5020) at chan_iax2.c:11622
#4  0x0818fe8b in dummy_start ()
#5  0x00abd832 in start_thread () from /lib/libpthread.so.0
#6  0x00a2845e in clone () from /lib/libc.so.6

Thread 15 (Thread 25406):
#0  0x00261402 in __kernel_vsyscall ()
#1  0x00ac1bc5 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
#2  0x00a34b2d in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libc.so.6
#3  0x0465863f in iax2_process_thread (data=0x94a6180) at chan_iax2.c:11622
#4  0x0818fe8b in dummy_start ()
#5  0x00abd832 in start_thread () from /lib/libpthread.so.0
#6  0x00a2845e in clone () from /lib/libc.so.6

Thread 14 (Thread 25407):
#0  0x00261402 in __kernel_vsyscall ()
#1  0x00ac1bc5 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
#2  0x00a34b2d in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libc.so.6
#3  0x0465863f in iax2_process_thread (data=0x94a72e0) at chan_iax2.c:11622
#4  0x0818fe8b in dummy_start ()
#5  0x00abd832 in start_thread () from /lib/libpthread.so.0
#6  0x00a2845e in clone () from /lib/libc.so.6

Thread 13 (Thread 25408):
#0  0x00261402 in __kernel_vsyscall ()
#1  0x00a1e753 in poll () from /lib/libc.so.6
#2  0x08112fc4 in ast_io_wait ()
#3  0x046202eb in network_thread (ignore=0x0) at chan_iax2.c:12085
#4  0x0818fe8b in dummy_start ()
#5  0x00abd832 in start_thread () from /lib/libpthread.so.0
#6  0x00a2845e in clone () from /lib/libc.so.6

Thread 12 (Thread 25409):
#0  0x00261402 in __kernel_vsyscall ()
#1  0x00a1e753 in poll () from /lib/libc.so.6
#2  0x0058a2e0 in do_monitor (data=0x0) at chan_phone.c:1085
#3  0x0818fe8b in dummy_start ()
#4  0x00abd832 in start_thread () from /lib/libpthread.so.0
#5  0x00a2845e in clone () from /lib/libc.so.6

Thread 11 (Thread 25410):
#0  0x00261402 in __kernel_vsyscall ()
#1  0x00a1e753 in poll () from /lib/libc.so.6
#2  0x08112fc4 in ast_io_wait ()
#3  0x008b555e in do_monitor (data=0x0) at chan_unistim.c:4650
#4  0x0818fe8b in dummy_start ()
#5  0x00abd832 in start_thread () from /lib/libpthread.so.0
#6  0x00a2845e in clone () from /lib/libc.so.6

Thread 10 (Thread 25411):
#0  0x00261402 in __kernel_vsyscall ()
#1  0x00ac1bc5 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
#2  0x00a34b2d in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libc.so.6
#3  0x08182d34 in tps_processing_function ()
#4  0x0818fe8b in dummy_start ()
#5  0x00abd832 in start_thread () from /lib/libpthread.so.0
#6  0x00a2845e in clone () from /lib/libc.so.6

Thread 9 (Thread 25412):
#0  0x00261402 in __kernel_vsyscall ()
#1  0x009e7d26 in nanosleep () from /lib/libc.so.6
#2  0x00e573b5 in cleanup (unused=0x0) at pbx_realtime.c:130
#3  0x0818fe8b in dummy_start ()
#4  0x00abd832 in start_thread () from /lib/libpthread.so.0
#5  0x00a2845e in clone () from /lib/libc.so.6

Thread 8 (Thread 25413):
#0  0x00261402 in __kernel_vsyscall ()
#1  0x00a1e753 in poll () from /lib/libc.so.6
#2  0x08112fc4 in ast_io_wait ()
#3  0x002f0a8d in network_thread (ignore=0x0) at pbx_dundi.c:2178
#4  0x0818fe8b in dummy_start ()
#5  0x00abd832 in start_thread () from /lib/libpthread.so.0
#6  0x00a2845e in clone () from /lib/libc.so.6

Thread 7 (Thread 25414):
#0  0x00261402 in __kernel_vsyscall ()
#1  0x009e7d26 in nanosleep () from /lib/libc.so.6
#2  0x009e7b4f in sleep () from /lib/libc.so.6
#3  0x003012ca in process_precache (ign=0x0) at pbx_dundi.c:2255
#4  0x0818fe8b in dummy_start ()
#5  0x00abd832 in start_thread () from /lib/libpthread.so.0
#6  0x00a2845e in clone () from /lib/libc.so.6

Thread 6 (Thread 25415):
#0  0x00261402 in __kernel_vsyscall ()
#1  0x009e7d26 in nanosleep () from /lib/libc.so.6
#2  0x009e7b4f in sleep () from /lib/libc.so.6
#3  0x002f0988 in process_clearcache (ignore=0x0) at pbx_dundi.c:2218
#4  0x0818fe8b in dummy_start ()
#5  0x00abd832 in start_thread () from /lib/libpthread.so.0
#6  0x00a2845e in clone () from /lib/libc.so.6

Thread 5 (Thread 25416):
#0  0x00261402 in __kernel_vsyscall ()
#1  0x00a1e753 in poll () from /lib/libc.so.6
#2  0x008c68b2 in scan_thread (unused=0x0) at pbx_spool.c:634
#3  0x0818fe8b in dummy_start ()
#4  0x00abd832 in start_thread () from /lib/libpthread.so.0
#5  0x00a2845e in clone () from /lib/libc.so.6

Thread 4 (Thread 25417):
#0  0x00261402 in __kernel_vsyscall ()
#1  0x009e7d26 in nanosleep () from /lib/libc.so.6
#2  0x00896fc9 in lock_broker (unused=0x0) at func_lock.c:192
#3  0x0818fe8b in dummy_start ()
#4  0x00abd832 in start_thread () from /lib/libpthread.so.0
#5  0x00a2845e in clone () from /lib/libc.so.6

Thread 3 (Thread 25418):
#0  0x00261402 in __kernel_vsyscall ()
#1  0x00ac1bc5 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
#2  0x00a34b2d in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libc.so.6
#3  0x08182d34 in tps_processing_function ()
#4  0x0818fe8b in dummy_start ()
#5  0x00abd832 in start_thread () from /lib/libpthread.so.0
#6  0x00a2845e in clone () from /lib/libc.so.6

Thread 2 (Thread 25419):
#0  0x00261402 in __kernel_vsyscall ()
#1  0x00a1e753 in poll () from /lib/libc.so.6
#2  0x0807b0ee in monitor_sig_flags ()
#3  0x0818fe8b in dummy_start ()
#4  0x00abd832 in start_thread () from /lib/libpthread.so.0
#5  0x00a2845e in clone () from /lib/libc.so.6

Thread 1 (Thread 25562):
#0  0x00261402 in __kernel_vsyscall ()
#1  0x0097edf0 in raise () from /lib/libc.so.6
#2  0x00980701 in abort () from /lib/libc.so.6
#3  0x009b73ab in __libc_message () from /lib/libc.so.6
#4  0x009bf6c5 in _int_free () from /lib/libc.so.6
#5  0x009bfb09 in free () from /lib/libc.so.6
#6  0x0810bdc9 in ast_http_send ()
#7  0x08125d4f in generic_http_callback ()
#8  0x08126480 in rawman_http_callback ()
#9  0x0810e629 in httpd_helper_thread ()
#10 0x08183732 in handle_tcptls_connection ()
#11 0x0818fe8b in dummy_start ()
#12 0x00abd832 in start_thread () from /lib/libpthread.so.0
#13 0x00a2845e in clone () from /lib/libc.so.6

BT


(gdb) bt
#0  0x00261402 in __kernel_vsyscall ()
#1  0x0097edf0 in raise () from /lib/libc.so.6
#2  0x00980701 in abort () from /lib/libc.so.6
#3  0x009b73ab in __libc_message () from /lib/libc.so.6
#4  0x009bf6c5 in _int_free () from /lib/libc.so.6
#5  0x009bfb09 in free () from /lib/libc.so.6
#6  0x0810bdc9 in ast_http_send ()
#7  0x08125d4f in generic_http_callback ()
#8  0x08126480 in rawman_http_callback ()
#9  0x0810e629 in httpd_helper_thread ()
#10 0x08183732 in handle_tcptls_connection ()
#11 0x0818fe8b in dummy_start ()
#12 0x00abd832 in start_thread () from /lib/libpthread.so.0
#13 0x00a2845e in clone () from /lib/libc.so.6

BT Full

(gdb) bt full
#0  0x00261402 in __kernel_vsyscall ()
No symbol table info available.
#1  0x0097edf0 in raise () from /lib/libc.so.6
No symbol table info available.
#2  0x00980701 in abort () from /lib/libc.so.6
No symbol table info available.
#3  0x009b73ab in __libc_message () from /lib/libc.so.6
No symbol table info available.
#4  0x009bf6c5 in _int_free () from /lib/libc.so.6
No symbol table info available.
#5  0x009bfb09 in free () from /lib/libc.so.6
No symbol table info available.
#6  0x0810bdc9 in ast_http_send ()
No symbol table info available.
#7  0x08125d4f in generic_http_callback ()
No symbol table info available.
#8  0x08126480 in rawman_http_callback ()
No symbol table info available.
#9  0x0810e629 in httpd_helper_thread ()
No symbol table info available.
#10 0x08183732 in handle_tcptls_connection ()
No symbol table info available.
#11 0x0818fe8b in dummy_start ()
No symbol table info available.
#12 0x00abd832 in start_thread () from /lib/libpthread.so.0
No symbol table info available.
#13 0x00a2845e in clone () from /lib/libc.so.6
No symbol table info available.
13

You haven’t issued

bt

and

bt full

so you have no more information than the summary one. Ideally you also issue those preceded by “thread apply all”, although some dumps don’t seem to provide anything except the running thread.

In practice, though it is telling you that you need to follow the guidance in the valgrind wiki page.

14

Yeah I just posted the stuff I noticed I forgot.

For valgrind I have to download source because since I have RPM installtion, I have no source. Already tried to do it but missing the suppression file (obviously).

15

If you don’t have source, the chances are you have an optimised binary, which means even the backtrace (when you run it properly) won’t be acceptable.

Generally, for crashes, people with binary distributions need to:

  1. get support through the packager;
  2. wait for someone building from source to provide enough information for the bug to get fixed; or
  3. rebuild from source.
16

Valgrind log


==27183== Memcheck, a memory error detector
==27183== Copyright (C) 2002-2009, and GNU GPL'd, by Julian Seward et al.
==27183== Using Valgrind-3.5.0 and LibVEX; rerun with -h for copyright info
==27183== Command: asterisk -vvvvcg
==27183== 
==27183== Invalid read of size 1
==27183==    at 0x94A57D: _dl_close (in /lib/ld-2.5.so)
==27183==    by 0xAB1D03: dlclose_doit (in /lib/libdl-2.5.so)
==27183==    by 0x945035: _dl_catch_error (in /lib/ld-2.5.so)
==27183==    by 0xAB22EB: _dlerror_run (in /lib/libdl-2.5.so)
==27183==    by 0xAB1D39: dlclose (in /lib/libdl-2.5.so)
==27183==    by 0x81159AA: load_dynamic_module (in /usr/sbin/asterisk)
==27183==    by 0x81163C6: load_resource (in /usr/sbin/asterisk)
==27183==    by 0x8116912: load_resource_list (in /usr/sbin/asterisk)
==27183==    by 0x8116D63: load_modules (in /usr/sbin/asterisk)
==27183==    by 0x8081E68: main (in /usr/sbin/asterisk)
==27183==  Address 0x546052c is 516 bytes inside a block of size 657 free'd
==27183==    at 0x400551D: free (vg_replace_malloc.c:325)
==27183==    by 0x94A0E4: _dl_close_worker (in /lib/ld-2.5.so)
==27183==    by 0x94A5A6: _dl_close (in /lib/ld-2.5.so)
==27183==    by 0xAB1D03: dlclose_doit (in /lib/libdl-2.5.so)
==27183==    by 0x945035: _dl_catch_error (in /lib/ld-2.5.so)
==27183==    by 0xAB22EB: _dlerror_run (in /lib/libdl-2.5.so)
==27183==    by 0xAB1D39: dlclose (in /lib/libdl-2.5.so)
==27183==    by 0x81159AA: load_dynamic_module (in /usr/sbin/asterisk)
==27183==    by 0x81163C6: load_resource (in /usr/sbin/asterisk)
==27183==    by 0x8116912: load_resource_list (in /usr/sbin/asterisk)
==27183==    by 0x8116D63: load_modules (in /usr/sbin/asterisk)
==27183==    by 0x8081E68: main (in /usr/sbin/asterisk)
==27183== 
==27183== Invalid read of size 4
==27183==    at 0x94A586: _dl_close (in /lib/ld-2.5.so)
==27183==    by 0xAB1D03: dlclose_doit (in /lib/libdl-2.5.so)
==27183==    by 0x945035: _dl_catch_error (in /lib/ld-2.5.so)
==27183==    by 0xAB22EB: _dlerror_run (in /lib/libdl-2.5.so)
==27183==    by 0xAB1D39: dlclose (in /lib/libdl-2.5.so)
==27183==    by 0x81159AA: load_dynamic_module (in /usr/sbin/asterisk)
==27183==    by 0x81163C6: load_resource (in /usr/sbin/asterisk)
==27183==    by 0x8116912: load_resource_list (in /usr/sbin/asterisk)
==27183==    by 0x8116D63: load_modules (in /usr/sbin/asterisk)
==27183==    by 0x8081E68: main (in /usr/sbin/asterisk)
==27183==  Address 0x54604b0 is 392 bytes inside a block of size 657 free'd
==27183==    at 0x400551D: free (vg_replace_malloc.c:325)
==27183==    by 0x94A0E4: _dl_close_worker (in /lib/ld-2.5.so)
==27183==    by 0x94A5A6: _dl_close (in /lib/ld-2.5.so)
==27183==    by 0xAB1D03: dlclose_doit (in /lib/libdl-2.5.so)
==27183==    by 0x945035: _dl_catch_error (in /lib/ld-2.5.so)
==27183==    by 0xAB22EB: _dlerror_run (in /lib/libdl-2.5.so)
==27183==    by 0xAB1D39: dlclose (in /lib/libdl-2.5.so)
==27183==    by 0x81159AA: load_dynamic_module (in /usr/sbin/asterisk)
==27183==    by 0x81163C6: load_resource (in /usr/sbin/asterisk)
==27183==    by 0x8116912: load_resource_list (in /usr/sbin/asterisk)
==27183==    by 0x8116D63: load_modules (in /usr/sbin/asterisk)
==27183==    by 0x8081E68: main (in /usr/sbin/asterisk)
==27183== 
==27183== Invalid read of size 4
==27183==    at 0x94A5BD: _dl_close (in /lib/ld-2.5.so)
==27183==    by 0xAB1D03: dlclose_doit (in /lib/libdl-2.5.so)
==27183==    by 0x945035: _dl_catch_error (in /lib/ld-2.5.so)
==27183==    by 0xAB22EB: _dlerror_run (in /lib/libdl-2.5.so)
==27183==    by 0xAB1D39: dlclose (in /lib/libdl-2.5.so)
==27183==    by 0x81159AA: load_dynamic_module (in /usr/sbin/asterisk)
==27183==    by 0x81163C6: load_resource (in /usr/sbin/asterisk)
==27183==    by 0x8116912: load_resource_list (in /usr/sbin/asterisk)
==27183==    by 0x8116D63: load_modules (in /usr/sbin/asterisk)
==27183==    by 0x8081E68: main (in /usr/sbin/asterisk)
==27183==  Address 0x546032c is 4 bytes inside a block of size 657 free'd
==27183==    at 0x400551D: free (vg_replace_malloc.c:325)
==27183==    by 0x94A0E4: _dl_close_worker (in /lib/ld-2.5.so)
==27183==    by 0x94A5A6: _dl_close (in /lib/ld-2.5.so)
==27183==    by 0xAB1D03: dlclose_doit (in /lib/libdl-2.5.so)
==27183==    by 0x945035: _dl_catch_error (in /lib/ld-2.5.so)
==27183==    by 0xAB22EB: _dlerror_run (in /lib/libdl-2.5.so)
==27183==    by 0xAB1D39: dlclose (in /lib/libdl-2.5.so)
==27183==    by 0x81159AA: load_dynamic_module (in /usr/sbin/asterisk)
==27183==    by 0x81163C6: load_resource (in /usr/sbin/asterisk)
==27183==    by 0x8116912: load_resource_list (in /usr/sbin/asterisk)
==27183==    by 0x8116D63: load_modules (in /usr/sbin/asterisk)
==27183==    by 0x8081E68: main (in /usr/sbin/asterisk)
==27183== 
==27183== Invalid read of size 1
==27183==    at 0x4006828: strlen (mc_replace_strmem.c:277)
==27183==    by 0x945131: _dl_signal_error (in /lib/ld-2.5.so)
==27183==    by 0x94A5D2: _dl_close (in /lib/ld-2.5.so)
==27183==    by 0xAB1D03: dlclose_doit (in /lib/libdl-2.5.so)
==27183==    by 0x945035: _dl_catch_error (in /lib/ld-2.5.so)
==27183==    by 0xAB22EB: _dlerror_run (in /lib/libdl-2.5.so)
==27183==    by 0xAB1D39: dlclose (in /lib/libdl-2.5.so)
==27183==    by 0x81159AA: load_dynamic_module (in /usr/sbin/asterisk)
==27183==    by 0x81163C6: load_resource (in /usr/sbin/asterisk)
==27183==    by 0x8116912: load_resource_list (in /usr/sbin/asterisk)
==27183==    by 0x8116D63: load_modules (in /usr/sbin/asterisk)
==27183==    by 0x8081E68: main (in /usr/sbin/asterisk)
==27183==  Address 0x54602c8 is 0 bytes inside a block of size 45 free'd
==27183==    at 0x400551D: free (vg_replace_malloc.c:325)
==27183==    by 0x94A062: _dl_close_worker (in /lib/ld-2.5.so)
==27183==    by 0x94A5A6: _dl_close (in /lib/ld-2.5.so)
==27183==    by 0xAB1D03: dlclose_doit (in /lib/libdl-2.5.so)
==27183==    by 0x945035: _dl_catch_error (in /lib/ld-2.5.so)
==27183==    by 0xAB22EB: _dlerror_run (in /lib/libdl-2.5.so)
==27183==    by 0xAB1D39: dlclose (in /lib/libdl-2.5.so)
==27183==    by 0x81159AA: load_dynamic_module (in /usr/sbin/asterisk)
==27183==    by 0x81163C6: load_resource (in /usr/sbin/asterisk)
==27183==    by 0x8116912: load_resource_list (in /usr/sbin/asterisk)
==27183==    by 0x8116D63: load_modules (in /usr/sbin/asterisk)
==27183==    by 0x8081E68: main (in /usr/sbin/asterisk)
==27183== 
==27183== Invalid read of size 1
==27183==    at 0x4006833: strlen (mc_replace_strmem.c:277)
==27183==    by 0x945131: _dl_signal_error (in /lib/ld-2.5.so)
==27183==    by 0x94A5D2: _dl_close (in /lib/ld-2.5.so)
==27183==    by 0xAB1D03: dlclose_doit (in /lib/libdl-2.5.so)
==27183==    by 0x945035: _dl_catch_error (in /lib/ld-2.5.so)
==27183==    by 0xAB22EB: _dlerror_run (in /lib/libdl-2.5.so)
==27183==    by 0xAB1D39: dlclose (in /lib/libdl-2.5.so)
==27183==    by 0x81159AA: load_dynamic_module (in /usr/sbin/asterisk)
==27183==    by 0x81163C6: load_resource (in /usr/sbin/asterisk)
==27183==    by 0x8116912: load_resource_list (in /usr/sbin/asterisk)
==27183==    by 0x8116D63: load_modules (in /usr/sbin/asterisk)
==27183==    by 0x8081E68: main (in /usr/sbin/asterisk)
==27183==  Address 0x54602c9 is 1 bytes inside a block of size 45 free'd
==27183==    at 0x400551D: free (vg_replace_malloc.c:325)
==27183==    by 0x94A062: _dl_close_worker (in /lib/ld-2.5.so)
==27183==    by 0x94A5A6: _dl_close (in /lib/ld-2.5.so)
==27183==    by 0xAB1D03: dlclose_doit (in /lib/libdl-2.5.so)
==27183==    by 0x945035: _dl_catch_error (in /lib/ld-2.5.so)
==27183==    by 0xAB22EB: _dlerror_run (in /lib/libdl-2.5.so)
==27183==    by 0xAB1D39: dlclose (in /lib/libdl-2.5.so)
==27183==    by 0x81159AA: load_dynamic_module (in /usr/sbin/asterisk)
==27183==    by 0x81163C6: load_resource (in /usr/sbin/asterisk)
==27183==    by 0x8116912: load_resource_list (in /usr/sbin/asterisk)
==27183==    by 0x8116D63: load_modules (in /usr/sbin/asterisk)
==27183==    by 0x8081E68: main (in /usr/sbin/asterisk)
==27183== 
==27183== Invalid read of size 1
==27183==    at 0x94CEA5: memcpy (in /lib/ld-2.5.so)
==27183==    by 0x94A5D2: _dl_close (in /lib/ld-2.5.so)
==27183==    by 0xAB1D03: dlclose_doit (in /lib/libdl-2.5.so)
==27183==    by 0x945035: _dl_catch_error (in /lib/ld-2.5.so)
==27183==    by 0xAB22EB: _dlerror_run (in /lib/libdl-2.5.so)
==27183==    by 0xAB1D39: dlclose (in /lib/libdl-2.5.so)
==27183==    by 0x81159AA: load_dynamic_module (in /usr/sbin/asterisk)
==27183==    by 0x81163C6: load_resource (in /usr/sbin/asterisk)
==27183==    by 0x8116912: load_resource_list (in /usr/sbin/asterisk)
==27183==    by 0x8116D63: load_modules (in /usr/sbin/asterisk)
==27183==    by 0x8081E68: main (in /usr/sbin/asterisk)
==27183==  Address 0x54602c8 is 0 bytes inside a block of size 45 free'd
==27183==    at 0x400551D: free (vg_replace_malloc.c:325)
==27183==    by 0x94A062: _dl_close_worker (in /lib/ld-2.5.so)
==27183==    by 0x94A5A6: _dl_close (in /lib/ld-2.5.so)
==27183==    by 0xAB1D03: dlclose_doit (in /lib/libdl-2.5.so)
==27183==    by 0x945035: _dl_catch_error (in /lib/ld-2.5.so)
==27183==    by 0xAB22EB: _dlerror_run (in /lib/libdl-2.5.so)
==27183==    by 0xAB1D39: dlclose (in /lib/libdl-2.5.so)
==27183==    by 0x81159AA: load_dynamic_module (in /usr/sbin/asterisk)
==27183==    by 0x81163C6: load_resource (in /usr/sbin/asterisk)
==27183==    by 0x8116912: load_resource_list (in /usr/sbin/asterisk)
==27183==    by 0x8116D63: load_modules (in /usr/sbin/asterisk)
==27183==    by 0x8081E68: main (in /usr/sbin/asterisk)
==27183== 
==27183== Invalid read of size 4
==27183==    at 0x94CEAC: memcpy (in /lib/ld-2.5.so)
==27183==    by 0x94A5D2: _dl_close (in /lib/ld-2.5.so)
==27183==    by 0xAB1D03: dlclose_doit (in /lib/libdl-2.5.so)
==27183==    by 0x945035: _dl_catch_error (in /lib/ld-2.5.so)
==27183==    by 0xAB22EB: _dlerror_run (in /lib/libdl-2.5.so)
==27183==    by 0xAB1D39: dlclose (in /lib/libdl-2.5.so)
==27183==    by 0x81159AA: load_dynamic_module (in /usr/sbin/asterisk)
==27183==    by 0x81163C6: load_resource (in /usr/sbin/asterisk)
==27183==    by 0x8116912: load_resource_list (in /usr/sbin/asterisk)
==27183==    by 0x8116D63: load_modules (in /usr/sbin/asterisk)
==27183==    by 0x8081E68: main (in /usr/sbin/asterisk)
==27183==  Address 0x54602c9 is 1 bytes inside a block of size 45 free'd
==27183==    at 0x400551D: free (vg_replace_malloc.c:325)
==27183==    by 0x94A062: _dl_close_worker (in /lib/ld-2.5.so)
==27183==    by 0x94A5A6: _dl_close (in /lib/ld-2.5.so)
==27183==    by 0xAB1D03: dlclose_doit (in /lib/libdl-2.5.so)
==27183==    by 0x945035: _dl_catch_error (in /lib/ld-2.5.so)
==27183==    by 0xAB22EB: _dlerror_run (in /lib/libdl-2.5.so)
==27183==    by 0xAB1D39: dlclose (in /lib/libdl-2.5.so)
==27183==    by 0x81159AA: load_dynamic_module (in /usr/sbin/asterisk)
==27183==    by 0x81163C6: load_resource (in /usr/sbin/asterisk)
==27183==    by 0x8116912: load_resource_list (in /usr/sbin/asterisk)
==27183==    by 0x8116D63: load_modules (in /usr/sbin/asterisk)
==27183==    by 0x8081E68: main (in /usr/sbin/asterisk)
==27183== 
==27183== Invalid read of size 2
==27183==    at 0x94CEAA: memcpy (in /lib/ld-2.5.so)
==27183==    by 0x94A5D2: _dl_close (in /lib/ld-2.5.so)
==27183==    by 0xAB1D03: dlclose_doit (in /lib/libdl-2.5.so)
==27183==    by 0x945035: _dl_catch_error (in /lib/ld-2.5.so)
==27183==    by 0xAB22EB: _dlerror_run (in /lib/libdl-2.5.so)
==27183==    by 0xAB1D39: dlclose (in /lib/libdl-2.5.so)
==27183==    by 0x81159D6: load_dynamic_module (in /usr/sbin/asterisk)
==27183==    by 0x81163C6: load_resource (in /usr/sbin/asterisk)
==27183==    by 0x8116912: load_resource_list (in /usr/sbin/asterisk)
==27183==    by 0x8116D63: load_modules (in /usr/sbin/asterisk)
==27183==    by 0x8081E68: main (in /usr/sbin/asterisk)
==27183==  Address 0x54608a0 is 0 bytes inside a block of size 46 free'd
==27183==    at 0x400551D: free (vg_replace_malloc.c:325)
==27183==    by 0x94A062: _dl_close_worker (in /lib/ld-2.5.so)
==27183==    by 0x94A5A6: _dl_close (in /lib/ld-2.5.so)
==27183==    by 0xAB1D03: dlclose_doit (in /lib/libdl-2.5.so)
==27183==    by 0x945035: _dl_catch_error (in /lib/ld-2.5.so)
==27183==    by 0xAB22EB: _dlerror_run (in /lib/libdl-2.5.so)
==27183==    by 0xAB1D39: dlclose (in /lib/libdl-2.5.so)
==27183==    by 0x81159D6: load_dynamic_module (in /usr/sbin/asterisk)
==27183==    by 0x81163C6: load_resource (in /usr/sbin/asterisk)
==27183==    by 0x8116912: load_resource_list (in /usr/sbin/asterisk)
==27183==    by 0x8116D63: load_modules (in /usr/sbin/asterisk)
==27183==    by 0x8081E68: main (in /usr/sbin/asterisk)
==27183== 
==27183== Thread 44:
==27183== Invalid read of size 4
==27183==    at 0x810BCC0: ast_http_send (in /usr/sbin/asterisk)
==27183==    by 0x8125D4E: generic_http_callback (in /usr/sbin/asterisk)
==27183==    by 0x812647F: rawman_http_callback (in /usr/sbin/asterisk)
==27183==    by 0x810E628: httpd_helper_thread (in /usr/sbin/asterisk)
==27183==    by 0x8183731: handle_tcptls_connection (in /usr/sbin/asterisk)
==27183==    by 0x818FE8A: dummy_start (in /usr/sbin/asterisk)
==27183==    by 0xABD831: start_thread (in /lib/libpthread-2.5.so)
==27183==    by 0xA2845D: clone (in /lib/libc-2.5.so)
==27183==  Address 0x4db95b4 is 12 bytes inside a block of size 2,060 free'd
==27183==    at 0x4005A02: realloc (vg_replace_malloc.c:476)
==27183==    by 0x81808A8: __ast_str_helper (in /usr/sbin/asterisk)
==27183==    by 0x8190BA0: ast_str_append (in /usr/sbin/asterisk)
==27183==    by 0x811D6D7: process_output (in /usr/sbin/asterisk)
==27183==    by 0x8125C1E: generic_http_callback (in /usr/sbin/asterisk)
==27183==    by 0x812647F: rawman_http_callback (in /usr/sbin/asterisk)
==27183==    by 0x810E628: httpd_helper_thread (in /usr/sbin/asterisk)
==27183==    by 0x8183731: handle_tcptls_connection (in /usr/sbin/asterisk)
==27183==    by 0x818FE8A: dummy_start (in /usr/sbin/asterisk)
==27183==    by 0xABD831: start_thread (in /lib/libpthread-2.5.so)
==27183==    by 0xA2845D: clone (in /lib/libc-2.5.so)
==27183== 
==27183== Invalid read of size 4
==27183==    at 0x810BCD5: ast_http_send (in /usr/sbin/asterisk)
==27183==    by 0x8125D4E: generic_http_callback (in /usr/sbin/asterisk)
==27183==    by 0x812647F: rawman_http_callback (in /usr/sbin/asterisk)
==27183==    by 0x810E628: httpd_helper_thread (in /usr/sbin/asterisk)
==27183==    by 0x8183731: handle_tcptls_connection (in /usr/sbin/asterisk)
==27183==    by 0x818FE8A: dummy_start (in /usr/sbin/asterisk)
==27183==    by 0xABD831: start_thread (in /lib/libpthread-2.5.so)
==27183==    by 0xA2845D: clone (in /lib/libc-2.5.so)
==27183==  Address 0x4db95bc is 20 bytes inside a block of size 2,060 free'd
==27183==    at 0x4005A02: realloc (vg_replace_malloc.c:476)
==27183==    by 0x81808A8: __ast_str_helper (in /usr/sbin/asterisk)
==27183==    by 0x8190BA0: ast_str_append (in /usr/sbin/asterisk)
==27183==    by 0x811D6D7: process_output (in /usr/sbin/asterisk)
==27183==    by 0x8125C1E: generic_http_callback (in /usr/sbin/asterisk)
==27183==    by 0x812647F: rawman_http_callback (in /usr/sbin/asterisk)
==27183==    by 0x810E628: httpd_helper_thread (in /usr/sbin/asterisk)
==27183==    by 0x8183731: handle_tcptls_connection (in /usr/sbin/asterisk)
==27183==    by 0x818FE8A: dummy_start (in /usr/sbin/asterisk)
==27183==    by 0xABD831: start_thread (in /lib/libpthread-2.5.so)
==27183==    by 0xA2845D: clone (in /lib/libc-2.5.so)
==27183== 
==27183== Invalid read of size 1
==27183==    at 0x4006808: strlen (mc_replace_strmem.c:275)
==27183==    by 0x9AEAE7: fputs (in /lib/libc-2.5.so)
==27183==    by 0x810BD99: ast_http_send (in /usr/sbin/asterisk)
==27183==    by 0x8125D4E: generic_http_callback (in /usr/sbin/asterisk)
==27183==    by 0x812647F: rawman_http_callback (in /usr/sbin/asterisk)
==27183==    by 0x810E628: httpd_helper_thread (in /usr/sbin/asterisk)
==27183==    by 0x8183731: handle_tcptls_connection (in /usr/sbin/asterisk)
==27183==    by 0x818FE8A: dummy_start (in /usr/sbin/asterisk)
==27183==    by 0xABD831: start_thread (in /lib/libpthread-2.5.so)
==27183==    by 0xA2845D: clone (in /lib/libc-2.5.so)
==27183==  Address 0x4db95b4 is 12 bytes inside a block of size 2,060 free'd
==27183==    at 0x4005A02: realloc (vg_replace_malloc.c:476)
==27183==    by 0x81808A8: __ast_str_helper (in /usr/sbin/asterisk)
==27183==    by 0x8190BA0: ast_str_append (in /usr/sbin/asterisk)
==27183==    by 0x811D6D7: process_output (in /usr/sbin/asterisk)
==27183==    by 0x8125C1E: generic_http_callback (in /usr/sbin/asterisk)
==27183==    by 0x812647F: rawman_http_callback (in /usr/sbin/asterisk)
==27183==    by 0x810E628: httpd_helper_thread (in /usr/sbin/asterisk)
==27183==    by 0x8183731: handle_tcptls_connection (in /usr/sbin/asterisk)
==27183==    by 0x818FE8A: dummy_start (in /usr/sbin/asterisk)
==27183==    by 0xABD831: start_thread (in /lib/libpthread-2.5.so)
==27183==    by 0xA2845D: clone (in /lib/libc-2.5.so)
==27183== 
==27183== Invalid read of size 1
==27183==    at 0x4006813: strlen (mc_replace_strmem.c:275)
==27183==    by 0x9AEAE7: fputs (in /lib/libc-2.5.so)
==27183==    by 0x810BD99: ast_http_send (in /usr/sbin/asterisk)
==27183==    by 0x8125D4E: generic_http_callback (in /usr/sbin/asterisk)
==27183==    by 0x812647F: rawman_http_callback (in /usr/sbin/asterisk)
==27183==    by 0x810E628: httpd_helper_thread (in /usr/sbin/asterisk)
==27183==    by 0x8183731: handle_tcptls_connection (in /usr/sbin/asterisk)
==27183==    by 0x818FE8A: dummy_start (in /usr/sbin/asterisk)
==27183==    by 0xABD831: start_thread (in /lib/libpthread-2.5.so)
==27183==    by 0xA2845D: clone (in /lib/libc-2.5.so)
==27183==  Address 0x4db95b5 is 13 bytes inside a block of size 2,060 free'd
==27183==    at 0x4005A02: realloc (vg_replace_malloc.c:476)
==27183==    by 0x81808A8: __ast_str_helper (in /usr/sbin/asterisk)
==27183==    by 0x8190BA0: ast_str_append (in /usr/sbin/asterisk)
==27183==    by 0x811D6D7: process_output (in /usr/sbin/asterisk)
==27183==    by 0x8125C1E: generic_http_callback (in /usr/sbin/asterisk)
==27183==    by 0x812647F: rawman_http_callback (in /usr/sbin/asterisk)
==27183==    by 0x810E628: httpd_helper_thread (in /usr/sbin/asterisk)
==27183==    by 0x8183731: handle_tcptls_connection (in /usr/sbin/asterisk)
==27183==    by 0x818FE8A: dummy_start (in /usr/sbin/asterisk)
==27183==    by 0xABD831: start_thread (in /lib/libpthread-2.5.so)
==27183==    by 0xA2845D: clone (in /lib/libc-2.5.so)
==27183== 
==27183== Syscall param write(buf) points to unaddressable byte(s)
==27183==    at 0xA1908B: ??? (in /lib/libc-2.5.so)
==27183==    by 0x9B8AF4: new_do_write (in /lib/libc-2.5.so)
==27183==    by 0x9B8DAE: _IO_file_xsputn@@GLIBC_2.1 (in /lib/libc-2.5.so)
==27183==    by 0x9AEB49: fputs (in /lib/libc-2.5.so)
==27183==    by 0x810BD99: ast_http_send (in /usr/sbin/asterisk)
==27183==    by 0x8125D4E: generic_http_callback (in /usr/sbin/asterisk)
==27183==    by 0x812647F: rawman_http_callback (in /usr/sbin/asterisk)
==27183==    by 0x810E628: httpd_helper_thread (in /usr/sbin/asterisk)
==27183==    by 0x8183731: handle_tcptls_connection (in /usr/sbin/asterisk)
==27183==    by 0x818FE8A: dummy_start (in /usr/sbin/asterisk)
==27183==    by 0xABD831: start_thread (in /lib/libpthread-2.5.so)
==27183==    by 0xA2845D: clone (in /lib/libc-2.5.so)
==27183==  Address 0x4db95b4 is 12 bytes inside a block of size 2,060 free'd
==27183==    at 0x4005A02: realloc (vg_replace_malloc.c:476)
==27183==    by 0x81808A8: __ast_str_helper (in /usr/sbin/asterisk)
==27183==    by 0x8190BA0: ast_str_append (in /usr/sbin/asterisk)
==27183==    by 0x811D6D7: process_output (in /usr/sbin/asterisk)
==27183==    by 0x8125C1E: generic_http_callback (in /usr/sbin/asterisk)
==27183==    by 0x812647F: rawman_http_callback (in /usr/sbin/asterisk)
==27183==    by 0x810E628: httpd_helper_thread (in /usr/sbin/asterisk)
==27183==    by 0x8183731: handle_tcptls_connection (in /usr/sbin/asterisk)
==27183==    by 0x818FE8A: dummy_start (in /usr/sbin/asterisk)
==27183==    by 0xABD831: start_thread (in /lib/libpthread-2.5.so)
==27183==    by 0xA2845D: clone (in /lib/libc-2.5.so)
==27183== 
==27183== Invalid free() / delete / delete[]
==27183==    at 0x400551D: free (vg_replace_malloc.c:325)
==27183==    by 0x810BDC8: ast_http_send (in /usr/sbin/asterisk)
==27183==    by 0x8125D4E: generic_http_callback (in /usr/sbin/asterisk)
==27183==    by 0x812647F: rawman_http_callback (in /usr/sbin/asterisk)
==27183==    by 0x810E628: httpd_helper_thread (in /usr/sbin/asterisk)
==27183==    by 0x8183731: handle_tcptls_connection (in /usr/sbin/asterisk)
==27183==    by 0x818FE8A: dummy_start (in /usr/sbin/asterisk)
==27183==    by 0xABD831: start_thread (in /lib/libpthread-2.5.so)
==27183==    by 0xA2845D: clone (in /lib/libc-2.5.so)
==27183==  Address 0x4db95a8 is 0 bytes inside a block of size 2,060 free'd
==27183==    at 0x4005A02: realloc (vg_replace_malloc.c:476)
==27183==    by 0x81808A8: __ast_str_helper (in /usr/sbin/asterisk)
==27183==    by 0x8190BA0: ast_str_append (in /usr/sbin/asterisk)
==27183==    by 0x811D6D7: process_output (in /usr/sbin/asterisk)
==27183==    by 0x8125C1E: generic_http_callback (in /usr/sbin/asterisk)
==27183==    by 0x812647F: rawman_http_callback (in /usr/sbin/asterisk)
==27183==    by 0x810E628: httpd_helper_thread (in /usr/sbin/asterisk)
==27183==    by 0x8183731: handle_tcptls_connection (in /usr/sbin/asterisk)
==27183==    by 0x818FE8A: dummy_start (in /usr/sbin/asterisk)
==27183==    by 0xABD831: start_thread (in /lib/libpthread-2.5.so)
==27183==    by 0xA2845D: clone (in /lib/libc-2.5.so)
==27183== 
==27183== Thread 16:
==27183== Conditional jump or move depends on uninitialised value(s)
==27183==    at 0x8132DD0: _ast_sockaddr_port (in /usr/sbin/asterisk)
==27183==    by 0x5EA48B0: check_via (chan_sip.c:15343)
==27183==    by 0x5EF6B6E: handle_request_register (chan_sip.c:24140)
==27183==    by 0x5F10487: handle_incoming (chan_sip.c:24413)
==27183==    by 0x5F119DF: handle_request_do (chan_sip.c:24575)
==27183==    by 0x5F12C03: sipsock_read (chan_sip.c:24509)
==27183==    by 0x8113033: ast_io_wait (in /usr/sbin/asterisk)
==27183==    by 0x5ED0B4B: do_monitor (chan_sip.c:25004)
==27183==    by 0x818FE8A: dummy_start (in /usr/sbin/asterisk)
==27183==    by 0xABD831: start_thread (in /lib/libpthread-2.5.so)
==27183==    by 0xA2845D: clone (in /lib/libc-2.5.so)
==27183== 
==27183== Conditional jump or move depends on uninitialised value(s)
==27183==    at 0x8132DD6: _ast_sockaddr_port (in /usr/sbin/asterisk)
==27183==    by 0x5EA48B0: check_via (chan_sip.c:15343)
==27183==    by 0x5EF6B6E: handle_request_register (chan_sip.c:24140)
==27183==    by 0x5F10487: handle_incoming (chan_sip.c:24413)
==27183==    by 0x5F119DF: handle_request_do (chan_sip.c:24575)
==27183==    by 0x5F12C03: sipsock_read (chan_sip.c:24509)
==27183==    by 0x8113033: ast_io_wait (in /usr/sbin/asterisk)
==27183==    by 0x5ED0B4B: do_monitor (chan_sip.c:25004)
==27183==    by 0x818FE8A: dummy_start (in /usr/sbin/asterisk)
==27183==    by 0xABD831: start_thread (in /lib/libpthread-2.5.so)
==27183==    by 0xA2845D: clone (in /lib/libc-2.5.so)
==27183== 
==27183== 
==27183== HEAP SUMMARY:
==27183==     in use at exit: 4,479,727 bytes in 76,533 blocks
==27183==   total heap usage: 495,899 allocs, 419,371 frees, 39,072,773 bytes allocated
==27183== 
==27183== LEAK SUMMARY:
==27183==    definitely lost: 125,617 bytes in 989 blocks
==27183==    indirectly lost: 3,205 bytes in 196 blocks
==27183==      possibly lost: 3,209,289 bytes in 69,034 blocks
==27183==    still reachable: 1,141,616 bytes in 6,314 blocks
==27183==         suppressed: 0 bytes in 0 blocks
==27183== Rerun with --leak-check=full to see details of leaked memory
==27183== 
==27183== For counts of detected and suppressed errors, rerun with: -v
==27183== Use --track-origins=yes to see where uninitialised values come from
==27183== ERROR SUMMARY: 33805 errors from 16 contexts (suppressed: 1787 from 13)
17

[quote=“david55”]If you don’t have source, the chances are you have an optimised binary, which means even the backtrace (when you run it properly) won’t be acceptable.

Generally, for crashes, people with binary distributions need to:

  1. get support through the packager;
  2. wait for someone building from source to provide enough information for the bug to get fixed; or
  3. rebuild from source.[/quote]

Oh well then, guess its a wash. It will remain broken then because I’m not going to start all over, its forever broken then. Guess I’ll just switch us back over to SIPX I guess, if I have to start all over.

18

If SIPX is all you need, then don’t waste your time with Asterisk. Asterisk is for people who need a real IP telephony platform, not a legacy PBX implemented in Java.