Today, our Asterisk system coredumped while doing a logrotate (with the message “exceeded file size limit”). I’m not 100% sure what caused this problem, there are two issues I suppose are linked to each other, but it would be great if someone with more knowledge about asterisk could give me some ideas where to look at:
We had a SIP call that didn’t hang up since yesterday for reasons I can’t backtrace which caused the recording file of a conference to grow to 2 GB.
Also, Asterisk produced a couple of ten thousand message logrotate files (message.1 - message.999999) at the same time, so I guess it ended up in a loop as a result of the exceeded file size limit.
I have now configured logrotade/cron to monitor and delete old/oversized/too many messages in the spool/log directories. We also have turned off verbose logging althoug I guess this rather causes the problem to occur less frequently. I wonder if there are other measures we should take - is there a way to prevent the asterisk logrotator to produce 99999 backlog files? Am I even looking at the correct files?
I’m using Asterisk This is the backtrace:
Thread 1 (process 6082):
#0 ast_log (level=5, file=0x80f83af “logger.c”, line=906, function=0x80f8a02 “ast_verbose”, fmt=0x8109477 “%s”) at include/asterisk/lock.h:601
#1 0x08056db3 in ast_verbose (fmt=0x80f86e4 “Asterisk Event Logger restarted\n”) at logger.c:906
#2 0x08057584 in reload_logger (rotate=1) at logger.c:462
#3 0x08057942 in ast_log (level=5, file=0x80f83af “logger.c”, line=906, function=0x80f8a02 “ast_verbose”, fmt=0x8109477 “%s”) at logger.c:813
#4 0x08056db3 in ast_verbose (fmt=0x80f8708 “Rotated Logs Per SIGXFSZ (Exceeded file size limit)\n”) at logger.c:906
#5 0x08057987 in ast_log (level=5, file=0x80f83af “logger.c”, line=906, function=0x80f8a02 “ast_verbose”, fmt=0x8109477 “%s”) at logger.c:816
#6 0x08056db3 in ast_verbose (fmt=0x80f8708 “Rotated Logs Per SIGXFSZ (Exceeded file size limit)\n”) at logger.c:906
#7 0x08057987 in ast_log (level=5, file=0x80f83af “logger.c”, line=906, function=0x80f8a02 “ast_verbose”, fmt=0x8109477 “%s”) at logger.c:816
#8 0x08056db3 in ast_verbose (fmt=0x80f8708 “Rotated Logs Per SIGXFSZ (Exceeded file size limit)\n”) at logger.c:906
#9 0x08057987 in ast_log (level=5, file=0x80f83af “logger.c”, line=906, function=0x80f8a02 “ast_verbose”, fmt=0x8109477 “%s”) at logger.c:816
#10 0x08056db3 in ast_verbose (fmt=0x80f8708 “Rotated Logs Per SIGXFSZ (Exceeded file size limit)\n”) at logger.c:906
#11 0x08057987 in ast_log (level=5, file=0x80f83af “logger.c”, line=906, function=0x80f8a02 “ast_verbose”, fmt=0x8109477 “%s”) at logger.c:816
#12 0x08056db3 in ast_verbose (fmt=0x80f8708 “Rotated Logs Per SIGXFSZ (Exceeded file size limit)\n”) at logger.c:906
#13 0x08057987 in ast_log (level=5, file=0x80f83af “logger.c”, line=906, function=0x80f8a02 “ast_verbose”, fmt=0x8109477 “%s”) at logger.c:816
#14 0x08056db3 in ast_verbose (fmt=0x80f8708 “Rotated Logs Per SIGXFSZ (Exceeded file size limit)\n”) at logger.c:906
#15 0x08057987 in ast_log (level=5, file=0x80f83af “logger.c”, line=906, function=0x80f8a02 “ast_verbose”, fmt=0x8109477 “%s”) at logger.c:816
#16 0x08056db3 in ast_verbose (fmt=0x80f8708 “Rotated Logs Per SIGXFSZ (Exceeded file size limit)\n”) at logger.c:906
#17 0x08057987 in ast_log (level=5, file=0x80f83af “logger.c”, line=906, function=0x80f8a02 “ast_verbose”, fmt=0x8109477 “%s”) at logger.c:816
#18 0x08056db3 in ast_verbose (fmt=0x80f8708 “Rotated Logs Per SIGXFSZ (Exceeded file size limit)\n”) at logger.c:906
#19 0x08057987 in ast_log (level=5, file=0x80f83af “logger.c”, line=906, function=0x80f8a02 “ast_verbose”, fmt=0x8109477 “%s”) at logger.c:816
#20 0x08056db3 in ast_verbose (fmt=0x80f8708 “Rotated Logs Per SIGXFSZ (Exceeded file size limit)\n”) at logger.c:906
#21 0x08057987 in ast_log (level=0, file=0x36c88d “chan_sip.c”, line=6137, function=0x375454 “build_route”, fmt=0x37170c “build_route: Retaining previous route: <%s>\n”) at logger.c:816
#22 0x003423b1 in build_route (p=0x8e49848, req=0xb7b25c00, backwards=1) at chan_sip.c:6137
#23 0x0034c00f in handle_response_invite (p=0x8e49848, resp=29114, rest=0xb7b25e24 “OK”, req=0xb7b25c00, ignore=0, seqno=102) at chan_sip.c:9712
#24 0x00358b49 in handle_response (p=0x8e49848, resp=200, rest=0xb7b25e24 “OK”, req=0xb7b25c00, ignore=0, seqno=102) at chan_sip.c:10222
#25 0x0036250d in handle_request (p=0x8e49848, req=0xb7b25c00, sin=0xb7b25bf0, recount=0x1, nounlock=0x1) at chan_sip.c:11214
#26 0x003648da in sipsock_read (id=0x8d0ac38, fd=141, events=1, ignore=0x0) at chan_sip.c:11421
#27 0x08055bef in ast_io_wait (ioc=0x8d0d1a0, howlong=1) at io.c:284
#28 0x0036acde in do_monitor (data=0x0) at chan_sip.c:11584
#29 0x00b41371 in start_thread () from /lib/tls/libpthread.so.0
#30 0x00a69ffe in clone () from /lib/tls/libc.so.6