Could fork failure be non-memory related issue?

I don’t see this every time I try replicating this issue, but I have seen this a couple times now in the exact same dialplan spot. I made some dialplan changes elsewhere today (not in the problematic area).

[Feb  1 21:49:39]     -- Executing [h@dtroute-connect:1] System("Local/3475000@dtroute-connect-00000df1;2", "asterisk -rx "confbridge kick channel161223415916835 all"") in new stack
[Feb  1 21:49:39]     -- Executing [s@generate-call-file-singular:1] System("Local/from-internal@localdt-00000dee;2", "echo Channel: Local/2311127@lineclear >> /tmp/1612234153.16829.call") in new stack
[Feb  1 21:49:39]   == Spawn extension (from-internal, 01, 8) exited non-zero on 'SIP/ATAxLA2-0000031e'
[2021-02-01 21:49:39] WARNING[22943][C-00000e4a]: asterisk.c:1143 safe_exec_prep: Fork failed: Cannot allocate memory
[2021-02-01 21:49:39] WARNING[22943][C-00000e4a]: app_system.c:142 system_exec_helper: Unable to execute 'echo Channel: Local/2311127@lineclear >> /tmp/1612234153.16829.call'
[Feb  1 21:49:39]     -- Executing [3475000@npstnticket:1] NoOp("Local/3@dtroute-00000df0;2", "DIALSTATUS: CANCEL / HANGUPCAUSE: 0 / CDR: duration: 21 / billsec: 0 / start: 2021-02-01 21:49:18 / answer:  / end: 2021-02-01 21:49:39") in new stack

This happens on call teardown on Asterisk 18.2. I looked up the issue and it’s suggested that this error is memory related.

However, I can’t find any evidence for that. If I run top immediately after seeing this, Asterisk is idling at around 17% CPU and 54% memory, well below any the threshold of being out of memory, though I suppose it could be a memory leak, though that doesn’t explain why it can’t allocate memory. memory show summary doesn’t return anything rather suspicious and I compiled with MALLOC_DEBUG and mmlog remains void of activity. It isn’t causing any noticeable issues as far as I can tell, but I’ve not seen this before until today, and I’m wondering if something else could be causing the issue and how I might trace this, since it’s not causing crashes and it’s not raising any memory red flags. Thanks!

We have an issue quite like this. Using system to hang up channels on asterisk 13.
It generally works however occasionally we have this ‘memory’ issue.

This error is the result of using fork, which the system application uses:
https://github.com/asterisk/asterisk/blob/master/main/asterisk.c#L1143
https://man7.org/linux/man-pages/man2/fork.2.html

A fix for you may be checking ulimit on the user if not already, this has not helped ours.
Still trying to replicate the situation that this fails, got a feeling it is something to do with removing channels whilst they are just coming up, or something about calling system too often.

You should have a channel variable SYSTEMSTATUS which will be a failure. You may be able to try the system application again. However the auto fall through does not seem to work.
https://wiki.asterisk.org/wiki/display/AST/Application_System

Have you tried not using system and instead ARI/AMI?
https://wiki.asterisk.org/wiki/display/AST/Asterisk+18+ManagerAction_ConfbridgeKick

P.S. Sorry only allowed 2 links in a reply.

Sorry, somehow I didn’t get a notification about this and just saw this now.

Did not know about either of these, thanks for bringing them up!

Over time, I have noticed that it always (not always, but whenever this happens, it tends to be the case) that this happens in the middle of creating a call file. Not for anything in particular. Not sure why.

I haven’t seen it as much lately, though maybe I just haven’t been trolling the console for it.

SYSTEMSTATUS is good to know about but it might be impractical now to go and add that everywhere. I have thousands of System and Shell calls in my dialplan. I don’t use ARI/AMI at all. I try to stick with native dialplan functions if available, as much as possible, but otherwise doing a shell call is what I do, often for simplicity, other times because I pipe things around, like grepping channels or something, so it makes sense.

This sounds fragile and resource intensive. Creating a process is one of the most expensive tasks you can do. I would suggest that you consider using ‘real’ programming languages and AGI/AMI/ARI instead of doing everything in dialplan. You’ll end up with a more robust, maintainable, and performant system.

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.