The 'sorcery/contact-00000015' task processor queue reached 1500 scheduled tasks

Asterisk Asterisk SIP
1

We have an IVR built on Asterisk 13.15.0 and PJSIP version 2.7. The IVR handles 5000 calls a day connected to a Twilio Elastic SIP trunk. It also has 10 SIP desk phones configured which also use the Twilio trunk. All the endpoints are configured in the pjsip.conf file (not Realtime).

It’s been running great for 6 months but had it’s only crash this week with the warning that I have posted as the title. Asterisk simply stopped accepting new calls and the existing calls ran their course and hung up. I rebooted linux and inbound calls resumed.

I saw a helpful response to someone with the same failure that said the purpose of the taskprocessors is to notify registered observers in another thread whenever an item changes. He said the queue maxed out because one of the observers had locked up and prevented the rest from running. I do not know enough about Asterisk architecture to know what the observers are or how I would determine which observer failed. Is there a likely culprit? Are there more diagnostics I can put in place to catch the problem if it happens again?

Additional notes on our configuration is that we have not configured either pjproject.conf or sorcery.conf (I have not altered pjproject.conf or sorcery.conf at all).

I really appreciate your help!

Thanks!

2

There isn’t really a likely culprit or anything else you can do. Something that was interested in the contacts was slow in handling it, it could have been qualify support for example.

3

Thank you JColp.

I notice that in normal operation new tasks arrive into that queue for processing about once every 10 seconds. In order for 1500 unprocessed tasks to accumulate something must have been shut down for a few hours. Is that useful information for what might have happened?

4

Not particularly. Observers don’t just shut down, but they can be blocked for some reason. What that reason is - I do not know.