DB Connection Issues

So what am I missing? It seems like Asterisk still doesn’t accept new messages until the queue is cleared.

Your response was unclear. It wasn’t stated whether the change did or did not have an impact. I thought you were referring to before the change was made.

If you mean that even after the change has been made then PJSIP is still not accepting new INVITE requests, further investigation would likely need to be done for the specific scenario. A full debug log for example may provide more information.

Sorry about that. Yeah, the change has no affect.

what level of verbosity should the debug be set to?

And even with a full debug log it likely requires reproducing the specific scenario, stepping through the code, making a database behave the same way. You could file an issue for such but there is no time frame on when or if it would get looked at.

Understood.

So if I set PJSIP to only care about PJSIP taskprocessors, then does that mean that other taskprocessor overloads would not show up in the log? Or it will continue showing in the log, but in reality it won’t pause distributing messages.

They will continue showing up in the log. That is unrelated to the setting in PJSIP. The setting in PJSIP just tells PJSIP which ones to pay attention to when it happens. If it’s pjsip_only then it means only taskprocessors created in PJSIP world. As well the overload stuff only applies to new INVITEs and new stuff - SIP messages for existing dialogs and calls continue to be processed.

So let me make sure I correctly understand this.

If I set taskprocessor_overload_trigger=none then PJSIP will accept new INVITEs regardless of any overloads happening. Is that correct?

Thanks much

That is correct.

Got it. Well, I did that, but unfortunately PJSIP is not accepting new INVITEs when overloads happen.

I will try to set up a clean setup and create an issue in the tracker. (I am currently using v18.12.1)

That code hasn’t changed and has been confirmed to work, so it likely does work - but what you are experiencing is something after it and thus unrelated.

I didn’t get a chance to file an issue yet…

I worked on this a bit more, I still have taskprocessor_overload_trigger set to none. It seema to me like PJSIP is not ignoring the taskprocessor overloads.

For example, when creating a big amount of calls and the Confbridge queue gets overloaded, Asterisk (PJSIP) will not accept any new calls, until the alarm is cleared.

I watched it by running:

[root@yplab ~]# asterisk -x"core show taskprocessors" | grep 'Processor\|Confbridge'
Processor                                                               Processed   In Queue  Max Depth  Low water High water
Confbridge/1000-00000421                                                      366        533        695        450        500

Here’s when it started processing new INVITEs

[root@yplab ~]# asterisk -x"core show taskprocessors" | grep 'Processor\|Confbridge'
Processor                                                               Processed   In Queue  Max Depth  Low water High water
Confbridge/1000-00000421                                                      475        424        695        450        500

Further to this, I disabled CDR, I can destroy 2k channels at once and PJSIP is instantly processing the new INVITEs, meaning, I no longer get stasis/m:cdr overloads. I do see overloads during the process of the channels getting destroyed, such as stasis/m:devicestate and stasis/m:manager, but their queues clear so quick that I wasn’t able to capture the overloads and they had no impact on the calls whatsoever.

So I am afraid that PJSIP doesn’t actually ignore the overloads.

Did you enable debug level logging with at least a level of 3? The debug log will state that it is ignoring traffic due to the taskprocessor overload.

When you created the large number of channels, what was the state of the PJSIP traskprocessors as well?

I did not check. I’ll run this again tomorrow morning and check.

This produces a whole lot of messages, any particular strings to grep so I can locate the message a bit easier?

Thanks again for your time, Josh!

“Taskprocessor overload” will find it if present.

Indeed (pastebin):

[root@yplab ~]# tail -f /var/log/asterisk/full | grep -i processor
[2023-03-20 12:08:43] DEBUG[10291] res_pjsip/pjsip_distributor.c: Taskprocessor overload alert: Ignoring 'Request msg REGISTER/cseq=15 (rdata0x7f8e7c000948)'.
[2023-03-20 12:08:46] DEBUG[10291] res_pjsip/pjsip_distributor.c: Taskprocessor overload alert: Ignoring 'Request msg REGISTER/cseq=15 (rdata0x7f8e7c000948)'.
[2023-03-20 12:08:47] DEBUG[10291] res_pjsip/pjsip_distributor.c: Taskprocessor overload alert: Ignoring 'Request msg INVITE/cseq=1 (rdata0x7f8e7c000948)'.
[2023-03-20 12:08:50] DEBUG[10291] res_pjsip/pjsip_distributor.c: Taskprocessor overload alert: Ignoring 'Request msg REGISTER/cseq=15 (rdata0x7f8e7c000948)'.
[2023-03-20 12:08:54] DEBUG[10291] res_pjsip/pjsip_distributor.c: Taskprocessor overload alert: Ignoring 'Request msg REGISTER/cseq=15 (rdata0x7f8e7c000948)'.
[2023-03-20 12:08:58] DEBUG[10291] res_pjsip/pjsip_distributor.c: Taskprocessor overload alert: Ignoring 'Request msg REGISTER/cseq=15 (rdata0x7f8e7c000948)'.
[2023-03-20 12:09:02] DEBUG[10291] res_pjsip/pjsip_distributor.c: Taskprocessor overload alert: Ignoring 'Request msg REGISTER/cseq=15 (rdata0x7f8e7c000948)'.
[2023-03-20 12:09:03] DEBUG[10291] res_pjsip/pjsip_distributor.c: Taskprocessor overload alert: Ignoring 'Request msg INVITE/cseq=1 (rdata0x7f8e7c000948)'.
[2023-03-20 12:09:06] DEBUG[10291] res_pjsip/pjsip_distributor.c: Taskprocessor overload alert: Ignoring 'Request msg REGISTER/cseq=15 (rdata0x7f8e7c000948)'.
[2023-03-20 12:09:10] DEBUG[10291] res_pjsip/pjsip_distributor.c: Taskprocessor overload alert: Ignoring 'Request msg REGISTER/cseq=15 (rdata0x7f8e7c000948)'.
[2023-03-20 12:09:14] DEBUG[10291] res_pjsip/pjsip_distributor.c: Taskprocessor overload alert: Ignoring 'Request msg REGISTER/cseq=15 (rdata0x7f8e7c000948)'.
[2023-03-20 12:09:46] DEBUG[10291] res_pjsip/pjsip_distributor.c: Taskprocessor overload alert: Ignoring 'Request msg REGISTER/cseq=1 (rdata0x7f8e7c000948)'.
[2023-03-20 12:09:47] DEBUG[10291] res_pjsip/pjsip_distributor.c: Taskprocessor overload alert: Ignoring 'Request msg REGISTER/cseq=1 (rdata0x7f8e7c000948)'.
[2023-03-20 12:09:48] DEBUG[10291] res_pjsip/pjsip_distributor.c: Taskprocessor overload alert: Ignoring 'Request msg REGISTER/cseq=1 (rdata0x7f8e7c000948)'.
[2023-03-20 12:09:50] DEBUG[10291] res_pjsip/pjsip_distributor.c: Taskprocessor overload alert: Ignoring 'Request msg REGISTER/cseq=1 (rdata0x7f8e7c000948)'.
[2023-03-20 12:09:54] DEBUG[10291] res_pjsip/pjsip_distributor.c: Taskprocessor overload alert: Ignoring 'Request msg REGISTER/cseq=1 (rdata0x7f8e7c000948)'.
[2023-03-20 12:09:58] DEBUG[10291] res_pjsip/pjsip_distributor.c: Taskprocessor overload alert: Ignoring 'Request msg REGISTER/cseq=1 (rdata0x7f8e7c000948)'.
[2023-03-20 12:10:02] DEBUG[10291] res_pjsip/pjsip_distributor.c: Taskprocessor overload alert: Ignoring 'Request msg REGISTER/cseq=1 (rdata0x7f8e7c000948)'.
[2023-03-20 12:10:05] DEBUG[3175] taskprocessor.c: Taskprocessor 'Confbridge/1000-00000428' cleared the high water alert.

These were all local channels. Here’s the output of the the PJSIP task processors: https://pastebin.com/tCVfsgV6

[root@yplab ~]# asterisk -x"core show taskprocessors" | grep 'Processor\|default\|distributor\|exten_state\|messaging\|pool'
Processor                                                               Processed   In Queue  Max Depth  Low water High water
pjsip/default-0000000a                                                         29          0          1        450        500
pjsip/default-0000000b                                                          0          0          0        450        500
pjsip/default-0000000c                                                          0          0          0        450        500
pjsip/default-0000000d                                                          0          0          0        450        500
pjsip/default-0000000e                                                          0          0          0        450        500
pjsip/default-0000000f                                                          0          0          0        450        500
pjsip/default-00000010                                                          0          0          0        450        500
pjsip/default-00000011                                                          0          0          0        450        500
pjsip/distributor-000001fa                                                     22          0          2        450        500
pjsip/distributor-000001fb                                                     38          0          2        450        500
pjsip/distributor-000001fc                                                     25          0          2        450        500
pjsip/distributor-000001fd                                                     50          0          2        450        500
pjsip/distributor-000001fe                                                     36          0          2        450        500
pjsip/distributor-000001ff                                                     36          0          1        450        500
pjsip/distributor-00000200                                                     38          0          2        450        500
pjsip/distributor-00000201                                                     60          0          1        450        500
pjsip/distributor-00000202                                                     28          0          1        450        500
pjsip/distributor-00000203                                                     36          0          2        450        500
pjsip/distributor-00000204                                                     28          0          1        450        500
pjsip/distributor-00000205                                                     16          0          1        450        500
pjsip/distributor-00000206                                                     36          0          2        450        500
pjsip/distributor-00000207                                                     58          0          2        450        500
pjsip/distributor-00000208                                                     36          0          2        450        500
pjsip/distributor-00000209                                                     18          0          1        450        500
pjsip/distributor-0000020a                                                     45          0          2        450        500
pjsip/distributor-0000020b                                                     26          0          2        450        500
pjsip/distributor-0000020c                                                     28          0          2        450        500
pjsip/distributor-0000020d                                                     26          0          2        450        500
pjsip/distributor-0000020e                                                  20264          0          2        450        500
pjsip/distributor-0000020f                                                     52          0          2        450        500
pjsip/distributor-00000210                                                     37          0          2        450        500
pjsip/distributor-00000211                                                     42          0          2        450        500
pjsip/distributor-00000212                                                     40          0          2        450        500
pjsip/distributor-00000213                                                     28          0          1        450        500
pjsip/distributor-00000214                                                     57          0          2        450        500
pjsip/distributor-00000215                                                     22          0          1        450        500
pjsip/distributor-00000216                                                     50          0          2        450        500
pjsip/distributor-00000217                                                     26          0          1        450        500
pjsip/distributor-00000218                                                     39          0          2        450        500
pjsip/exten_state                                                               0          0          0        450        500
pjsip/messaging                                                                 0          0          0        450        500
pjsip/pool                                                                 122676          0          3        450        500
pjsip/pool-control                                                         245740          0          5        450        500
sorcery/pool                                                                10541          0          2        450        500
sorcery/pool-control                                                        21443          0          2        450        500
stasis/pool                                                                  3008          0          2        450        500
stasis/pool-control                                                          6068          0          3        450        500

It looks to me like all other PJSIP traskprocessors are endpoints and options.

What am I missing?

Then it would seem as though they were indeed ignored due to other taskprocessors, though the code is so simple I don’t see how[1]… unless something is fundamentally wrong or an assumption is incorrect. I’ve orchestrated the code to make sure that the configuration is read in correctly and applied, and it is. This isn’t possibly a case where FreePBX is changing it underneath you is it?

[1] asterisk/pjsip_distributor.c at 20 · asterisk/asterisk · GitHub

I’ve also put my instance into a permanent state of perceived overload, and with “none” or “pjsip_only” messages are not dropped. Setting it to “global” they do get dropped.

Tested on both 18 and 20.

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.