Asterisk and MySQL - high CPU after connection problem

I’m currently using Asterisk 11.12 in production and have had issues where if latency is higher than about 50-60ms Asterisk goes to 100% CPU on all cores and doesn’t return to normal until restarting asterisk completely.

I’ve seen this behavior with several 11.x and 12.x versions and haven’t been able to find any bugs on it.

Has anybody seen this and are there any fixes for it?

When you obtain a backtrace (https://wiki.asterisk.org/wiki/display/AST/Getting+a+Backtrace) where does it show the code to be looping?

I was able to generate backtrace.txt I hope this points you in the right direction. I got a lot of “Missing separate debuginfo for” messages when using gdb so not sure how useful the backtrace is. Also this is on a system using AEL pretty heavily if that makes a difference.

Program terminated with signal 11, Segmentation fault. #0 0x00007fcfa0358091 in SQLCloseCursor () from /usr/lib64/libodbc.so.2 #0 0x00007fcfa0358091 in SQLCloseCursor () from /usr/lib64/libodbc.so.2 No symbol table info available.

Thread 1 (Thread 0x7fcf44cd0700 (LWP 8275)): #0 0x00007fcfa0358091 in SQLCloseCursor () from /usr/lib64/libodbc.so.2 #1 0x00007fcf4fd1c4c5 in acf_odbc_read (chan=<value optimized out>, cmd=0x7fcf44cc83c0 "ODBC_TENANTINFO_FROM_ID", s=<value optimized out>, buf=0x7fcf44cc8460 "*********", len=4096) at func_odbc.c:764 #2 0x00000000005244c7 in ast_func_read (chan=0x7fcf28be1808, function=<value optimized out>, workspace=0x7fcf44cc8460 "********", len=4096) at pbx.c:4312 #3 0x0000000000529542 in pbx_substitute_variables_helper_full (c=0x7fcf28be1808, headp=0x7fcf28be1e80, cp1=<value optimized out>, cp2=0x7fcf44ccd68b "", count=8180, used=0x7fcf44ccfb48) at pbx.c:4700 #4 0x000000000052cacf in pbx_substitute_variables_helper (c=0x7fcf28be1808, con=0x0, context=0x7fcf28be2658 "fromoutside", exten=0x7fcf28be26a8 "4082130350", priority=21, label=0x0, callerid=0x7fcf284e4950 "7146907000", action=E_SPAWN,$ #5 pbx_extension_helper (c=0x7fcf28be1808, con=0x0, context=0x7fcf28be2658 "fromoutside", exten=0x7fcf28be26a8 "4082130350", priority=21, label=0x0, callerid=0x7fcf284e4950 "7146907000", action=E_SPAWN, found=0x7fcf44ccfcfc, combined_f$ #6 0x0000000000530ff5 in ast_spawn_extension (c=0x7fcf28be1808, args=0x0) at pbx.c:6037 #7 __ast_pbx_run (c=0x7fcf28be1808, args=0x0) at pbx.c:6512 #8 0x00000000005327fb in pbx_thread (data=<value optimized out>) at pbx.c:6842 #9 0x00000000005745bb in dummy_start (data=<value optimized out>) at utils.c:1169 #10 0x00007fcfa2a349d1 in start_thread () from /lib64/libpthread.so.0 #11 0x00007fcfa3ffe86d in clone () from /lib64/libc.so.6

That’s a crash (0% CPU) not a loop (100%).

The lack of symbol table information for the third party library is expected. Your real debugging problem is that Asterisk was compiled with optimisation, which means it is difficult to see internal variables. You will need to change that before it would be accepted for a but report.

I’m not sufficiently familiar with that part of the code to take it any furrther.