I’m having a problem where asterisk hangs shortly after startup. It appears to be during the module loading process. The problem is, it’s only logging specific warnings or errors, not when a module may be loading okay and gets in a loop. This seems to happen to people a lot. It doesn’t help that Ubuntu includes 200 modules, and has most of them enabled by default (with autoload=yes)
Why doesn’t asterisk log the beginning of each module load to the log file? I have full verbosity on, and am not getting much more detail. Is there some other way to enable this? I would think that something loading a module should be logged at least with debug level logging.
It always hangs somewhere during the loading of modules. Unfortunately, even verbose logging isn’t very verbose as far as program status goes. It is helpful for call status information though.
Here’s what I see when I call with -c:
root@jupiter:/etc/asterisk# asterisk -c
Asterisk 1.6.2.9-2ubuntu2, Copyright (C) 1999 - 2010 Digium, Inc. and others.
Created by Mark Spencer <markster@digium.com>
Asterisk comes with ABSOLUTELY NO WARRANTY; type 'core show warranty' for details.
This is free software, with components licensed under the GNU General Public
License version 2 and other licenses; you are welcome to redistribute it under
certain conditions. Type 'core show license' for details.
=========================================================================
[ Booting...
[ Reading Master Configuration ]
[ Initializing Custom Configuration Options ]
[May 16 13:53:07] NOTICE[29806]: cdr.c:1471 do_reload: CDR simple logging enabled.
[May 16 13:53:07] NOTICE[29806]: loader.c:1064 load_modules: 201 modules will be loaded.
[May 16 13:53:07] WARNING[29806]: loader.c:393 load_dynamic_module: Error loading module 'chan_gtalk.so': /usr/lib/asterisk/modules/chan_gtalk.so: undefined symbol: ast_aji_get_client
^CKilled
The issue here isn’t necessarily ‘why is the program hanging’, but more so ‘the program isn’t telling me what it’s doing’. I can manually enable each module until I find the culprit. It would be nice, and if modules are easily able to hang the whole program, expected, that certain critical points in the program be logged.
I think the lack of 1.8 packages is going to be a problem for some time. There has been a change in user culture from build from source to install OS vendors’ packages, at the same time as the relatively short notice death of the 1.6 series. Packagers tend to lag a long way behind developers.
I actually just went through the same problem on two of my debian systems running the 1.8.4-1 packages. The update from the 1.6.2.9-2 packages went well but when it would try to start it would hang at the module loading stage with no indicator what was the hold up. After all kinds of tweaks and things I was able to get one system to run fine but couldn’t get the second (practically the same setup) to work. Looks like I finally nailed it down to chan_h323.so. If I prevent that from loading in modules.conf then everything works well
Let me know if there’s any helpful information I can provide
Yes, I forgot to mention I removed /usr/lib64/asterisk/modules prior to install of 1.8.8.2. It still hangs. I’m stumped because 1.8.7.0 ran fine on an older OpenSUSE 10.3 x64/Athlon 6000+ x2 Server. I thought OpenSUSE 12.1 x64 (new kernel 3.1.0), new hardware [ASUS Sabertooth 990FX, AMD FX-8120 8-core processor] would be a slam-dunk. I’ll keep looking. I’ve spent over 10 hours on this already. Thank you for your suggestion.
The list of loaded modules from the core dump should also help, and gdb can tell you were it is looping. You can atttach gdb before or after running, or use gcore.
The Asterisk team has been friendly and helpful, so I hope my small contribution will help us get to the bottom of this quickly.
Just guessing, but I wonder with 8 cores and a lot of parallel processing going on if this might be a deadlock situation among the 12 or so threads Asterisk uses? If true, then I wonder what surprises are in store when 10-core processors come out later this year. I’m not likely to be an early adopter… too much headache and cost in $ and time.