Astobj2.c:131 INTERNAL_OBJ: FRACK!, Failed assertion bad magic number

Asterisk Asterisk Support
1

Hi
We are using:

  • Asterisk 13.19.0 compiled from source
  • sip channel driver
  • Red Hat 7.4 OS Linux 3.10.0-693.11.6.el7.x86_64 x86_64
  • Bare hardware (no VM) 16 CPUs, 1200 MHz, 15.39 GB memory
  • asterisk-unimrcp 1.5 modules
  • Lumenvox 15.1.200 (64-bit) for speech recognition

We get fault messages showing up in the /var/log/asterisk/messages file and the asterisk console periodically and haven’t been able to determine why or how to fix this problem:

[Feb  6 10:48:35] ERROR[17692][C-00000021] astobj2.c: FRACK!, Failed assertion bad magic number 0x0 for object 0x31990d8 (0)
[Feb  6 10:48:35] ERROR[17692][C-00000021] astobj2.c: FRACK!, Failed assertion bad magic number 0x0 for object 0x31990d8 (0)
[Feb  6 10:48:35] ERROR[17692][C-00000021] astobj2.c: FRACK!, Failed assertion bad magic number 0x0 for object 0x31990d8 (0)
[Feb  6 10:48:35] ERROR[17692][C-00000021] astobj2.c: FRACK!, Failed assertion bad magic number 0x0 for object 0x31990d8 (0)
[Feb  6 10:48:35] ERROR[17692][C-00000021] astobj2.c: FRACK!, Failed assertion bad magic number 0x0 for object 0x31990d8 (0)
[Feb  6 10:48:35] ERROR[17692][C-00000021] astobj2.c: FRACK!, Failed assertion bad magic number 0x0 for object 0x31990d8 (0)
[Feb  6 10:48:35] ERROR[17692][C-00000021] astobj2.c: FRACK!, Failed assertion bad magic number 0x0 for object 0x31990d8 (0)
[Feb  6 10:48:35] ERROR[17692][C-00000021] astobj2.c: FRACK!, Failed assertion bad magic number 0x0 for object 0x31990d8 (0)
[Feb  6 10:48:35] ERROR[17692][C-00000021] astobj2.c: FRACK!, Failed assertion bad magic number 0x0 for object 0x31990d8 (0)
[Feb  6 10:48:35] ERROR[17692][C-00000021] astobj2.c: FRACK!, Failed assertion bad magic number 0x0 for object 0x31990d8 (0)
[Feb  6 10:48:35] ERROR[17692][C-00000021] astobj2.c: FRACK!, Failed assertion bad magic number 0x0 for object 0x31990d8 (0)

This is some of the console output:

[Feb  6 10:48:35] ERROR[17692][C-00000021]: astobj2.c:131 INTERNAL_OBJ: FRACK!, Failed assertion bad magic number 0x0 for object 0x31990d8 (0)
Got 19 backtrace records
#0: [0x45b116] /usr/sbin/asterisk(__ao2_ref+0x1a6) [0x45b116]
#1: [0x51b0a0] /usr/sbin/asterisk(ast_frdup+0x100) [0x51b0a0]
#2: [0x5e2fb6] /usr/sbin/asterisk(ast_translate+0x306) [0x5e2fb6]
#3: [0x4bec32] /usr/sbin/asterisk() [0x4bec32]
#4: [0x7f741a88f5e8] /usr/lib64/asterisk/modules/app_speech_utils.so(+0x35e8) [0x7f741a88f5e8]
#5: [0x57eae9] /usr/sbin/asterisk(pbx_exec+0xb9) [0x57eae9]
#6: [0x7f744b3ae432] /usr/lib64/asterisk/modules/res_agi.so(+0x8432) [0x7f744b3ae432]
#7: [0x7f744b3b3b92] /usr/lib64/asterisk/modules/res_agi.so(+0xdb92) [0x7f744b3b3b92]
#8: [0x7f744b3b47f6] /usr/lib64/asterisk/modules/res_agi.so(+0xe7f6) [0x7f744b3b47f6]
#9: [0x57eae9] /usr/sbin/asterisk(pbx_exec+0xb9) [0x57eae9]
#10: [0x573835] /usr/sbin/asterisk() [0x573835]
#11: [0x5757b4] /usr/sbin/asterisk() [0x5757b4]
#12: [0x576d7b] /usr/sbin/asterisk() [0x576d7b]
#13: [0x5e87ca] /usr/sbin/asterisk() [0x5e87ca]
[Feb  6 10:48:35] ERROR[17692][C-00000021]: astobj2.c:131 INTERNAL_OBJ: FRACK!, Failed assertion bad magic number 0x0 for object 0x31990d8 (0)
Got 17 backtrace records
#0: [0x45b5e1] /usr/sbin/asterisk(__ao2_cleanup+0x111) [0x45b5e1]
#1: [0x51ad41] /usr/sbin/asterisk(ast_frame_free+0x131) [0x51ad41]
#2: [0x7f741a88f6b4] /usr/lib64/asterisk/modules/app_speech_utils.so(+0x36b4) [0x7f741a88f6b4]
#3: [0x57eae9] /usr/sbin/asterisk(pbx_exec+0xb9) [0x57eae9]
#4: [0x7f744b3ae432] /usr/lib64/asterisk/modules/res_agi.so(+0x8432) [0x7f744b3ae432]
#5: [0x7f744b3b3b92] /usr/lib64/asterisk/modules/res_agi.so(+0xdb92) [0x7f744b3b3b92]
#6: [0x7f744b3b47f6] /usr/lib64/asterisk/modules/res_agi.so(+0xe7f6) [0x7f744b3b47f6]
#7: [0x57eae9] /usr/sbin/asterisk(pbx_exec+0xb9) [0x57eae9]
#8: [0x573835] /usr/sbin/asterisk() [0x573835]
#9: [0x5757b4] /usr/sbin/asterisk() [0x5757b4]
#10: [0x576d7b] /usr/sbin/asterisk() [0x576d7b]
#11: [0x5e87ca] /usr/sbin/asterisk() [0x5e87ca]

This is the latest version of Asterisk I have tried (previously tried 13.18-cert2), I also previously tried using pjsip instead of sip. I also confirmed that once it happens it continues to happen for every speech recognition call until asterisk is restarted. The error shows up for speech recognition based apps but doesn’t for non-speech recognition calls. I have confirmed this with simple bash scripts (ie: exec speechbackground vs exec playback).

Any suggestions/help is appreciated.
Thanks.

2

The message means that something is attempting to access an object that has already been freed. It’s a sign that reference counting is incorrect somewhere. It’s likely either in asterisk-unimrcp or app_speech_utils.

3

Thanks.
Based on these https://wiki.asterisk.org/wiki/display/AST/Getting+a+Backtrace instructions, I was able to get a core dump file associated with the start of the event and I ran the dump file through that ast_coredumper utility to produce some .txt files. Would you (or anyone) be able to take a look at the details? If so, where would I post the tarball?

4

Crashes are reported on the issue tracker[1] but we generally don’t accept crash reports when third party modules are involved, as we have no control over them and can’t generally look at them.

[1] https://issues.asterisk.org/jira

5

Looks like Arsen Chaloyan was able to replicate the issue with Asterisk segfaulting in res_speech with “Object is likely destroyed” message, he provides a patch here: UniMRCP Discussion Board

6

See https://wiki.asterisk.org/wiki/display/AST/Patch+Contribution+Process

7

What this says is that:

  1. if you don’t submit properly, it will never go in official versions of Asterisk;

  2. Only Arsen Chaloyan or his employer have the legal authority to submit it, so no-one should simply copy the patch and try to submit it.