Multiple subp taskprocessors for single extension

vlchris · March 14, 2018, 10:03pm

Hey Everyone,

I’m hoping for some direction on an issue I’m having. I have a system with about 400 phones on it, and every once in a while a single phone will start running up a ton of subp processes.

The rest of the subp processes are fine, with 1 per extension. Once it hits about 3000, asterisk dumps, and it has to be restarted. The question is, how are these generated, and can someone give some clue as to how to stop it. This is currently running on 13.19.2 and have tested with 13.17.1 and 14.6.2. I have not tested latest 15 yet. Any direction would be awesome. (snippet below of the offending phone) This is chan_sip btw…

(from core show subp:246@R10T105-0000107a subp:246@R10T105-00001080 subp:246@R10T105-00001081 subp:246@R10T105-00001084 subp:246@R10T105-0000108a subp:246@R10T105-0000108b subp:246@R10T105-0000108d subp:246@R10T105-00001091 subp:246@R10T105-00001093 subp:246@R10T105-00001095 subp:246@R10T105-00001099 subp:246@R10T105-0000109a subp:246@R10T105-0000109b subp:246@R10T105-0000109c subp:246@R10T105-0000109d subp:246@R10T105-0000109e subp:246@R10T105-0000109f subp:246@R10T105-000010a0 subp:246@R10T105-000010a1 subp:246@R10T105-000010a3 subp:246@R10T105-000010a4 subp:246@R10T105-000010a8 taskprocessors x 2000+ times)
81 0 1 450 500
80 0 1 450 500
79 0 1 450 500
78 0 1 450 500
77 0 1 450 500
76 0 1 450 500
75 0 1 450 500
74 0 1 450 500
73 0 1 450 500
72 0 1 450 500
71 0 1 450 500
70 0 1 450 500
69 0 1 450 500
68 0 1 450 500
67 0 1 450 500
66 0 1 450 500
65 0 1 450 500
64 0 1 450 500
63 0 1 450 500
62 0 1 450 500
61 0 1 450 500
60 0 1 450 500

vlchris · March 15, 2018, 11:48pm

Interestingly enough, the MWI and Registrations are set to 120… I started a poll to see how often they are incrementing and every 5 minutes it shows +1…

So… re-registers are creating a new task processor for this one phone, I guess? I know I’m just talking to myself, but hey, maybe someone has an idea.

vlchris · March 17, 2018, 6:58pm

Making progress I think, I switched the offending phone to subscribemwi=no just as a test, and restarted asterisk… The task processors stayed level. Today I added a few new phones, and they started rising again, doing their normal little crashes it does before it finally dumps the core…

[2018-03-17 14:49:15] ERROR[13952]: astobj2.c:131 INTERNAL_OBJ: FRACK!, Failed assertion bad magic number 0x0 for object 0x7fd84c431fa0 (0)
Got 13 backtrace records
#0: [0x45bc37] asterisk(__ao2_ref+0x1a7) [0x45bc37]
#1: [0x7fd87cfa943a] /usr/lib/asterisk/modules/chan_sip.so(+0x6343a) [0x7fd87cfa943a]
#2: [0x5bc10b] asterisk() [0x5bc10b]
#3: [0x5d475e] asterisk(ast_taskprocessor_execute+0x15e) [0x5d475e]
#4: [0x5dc070] asterisk() [0x5dc070]
#5: [0x5d46be] asterisk(ast_taskprocessor_execute+0xbe) [0x5d46be]
#6: [0x5dc948] asterisk() [0x5dc948]
#7: [0x5e61ea] asterisk() [0x5e61ea]

It’s really bothering me, that a single phone can take down asterisk, but I feel like I’m getting closer. As you can see, after making the change a few days ago and restarted, the processors stayed normal until today when I added a few new phones to the mix, now they are climbing fast and throwing "FRACK!"s again…