Help finding a memory leak

I apologize, but I’m not sure I understand the question. The agents are using sip. They log in from a web app (or dialplan), and it creates or deletes a record in queue_member_table. I use as much realtime as possible. For the callers in to the queue, if that’s what you meant, I never thought to look. I’ll take a look and respond in a bit. Heading in to the office now.

I meant the agents which have been added into the queue. Do you directly add their SIP dialing string (SIP/bob) or does it go through the dialplan first using a Local channel.I ask because the issue I am working on should (I think) be isolated to Local channels and their behavior, if you don’t use them then your issue may be different or a different scenario I haven’t found yet.

That’s what I thought you meant, but yes, they’re all SIP/${EXTEN}.

membername     queue_name     interface     
Kozusnik Mark     71000        SIP/12966

Understood! I don’t think what I’m working on is the cause of that, checking versions this code didn’t exist in 13.8 and may actually have solved your queue problem as well.

I’ve been using Asterisk since v1.2. We still have two 1.4 machines running as well as a few 1.8. But I’ve never had to dig this deep before. Thanks for bearing with me.

I set up a scenario on the test box for a call to come in, create a timestamp variable, insert that into the database, then query for a count of records in the table. I ran 3 simultaneous instances of sipp, was able to see where multiple channels were inserting at the same time, and the memory for the asterisk daemon is still at 2%. I did get the cpu up to 38%.

[testing]
exten	=> service,1,Answer()
same	=> n,Set(calldt=${STRFTIME(${EPOCH},,%Y-%m-%d_%H:%M:%S)})
same	=> n,Set(ODBC_WRITESQL()=insert into testing (dt) values(\'${calldt}\'\))
same	=> n,Set(cnt=${ODBC_READSQL(select count(*) from testing)})
same	=> n,Set(cnt=${ODBC_READSQL(select count(*) from testing)})
same	=> n,Set(cnt=${ODBC_READSQL(select count(*) from testing)})
same	=> n,Set(cnt=${ODBC_READSQL(select count(*) from testing)})
same	=> n,Set(cnt=${ODBC_READSQL(select count(*) from testing)})
same	=> n,Set(cnt=${ODBC_READSQL(select count(*) from testing)})
same	=> n,Set(cnt=${ODBC_READSQL(select count(*) from testing)})
same	=> n,Set(cnt=${ODBC_READSQL(select count(*) from testing)})
same	=> n,Set(cnt=${ODBC_READSQL(select count(*) from testing)})
same	=> n,Set(cnt=${ODBC_READSQL(select count(*) from testing)})
same	=> n,Set(cnt=${ODBC_READSQL(select count(*) from testing)})
same	=> n,Set(cnt=${ODBC_READSQL(select count(*) from testing)})
;same	=> n,Playback(cannot-complete-as-dialed)
;same	=> n,Hangup()

exten	=> i,1,Hangup()
exten	=> t,1,Hangup()
exten	=> h,1,Hangup()


[sipp]
type=friend
context=testing
host=10.23.0.232
user=sipp
canreinvite=no
disallow=all
allow=ulaw

sipp -sn uac 172.18.96.138 -l 10 -d 100000

asterisk-whconfig01*CLI> core show channels
Channel Location State Application(Data)
0 active channels
0 active calls
10969 calls processed

asterisk-whconfig01*CLI> core show taskprocessors

Processor                                      Processed   In Queue  Max Depth  Low water High water
app_voicemail                                          0          0          0        450        500
ast_msg_queue                                          0          0          0        450        500
CCSS_core                                              0          0          0        450        500
Sorcery                                                0          0          0        450        500
Sorcery-control                                        1          0          1        450        500
sorcery/bucket-00000000                                0          0          0        450        500
sorcery/file-00000001                                  0          0          0        450        500
stasis-core                                       381644          0          8        450        500
stasis-core-control                               763455          0          8        450        500
subm:ast_bridge_topic_all-cached-0000000c            223          0         14        450        500
subm:ast_channel_topic_all-00000010               512521          0         28        450        500
subm:ast_channel_topic_all-cached-0000000b        672947          0         35        450        500
subm:ast_channel_topic_all-cached-0000000d        672946          0         35        450        500
subm:ast_device_state_topic-00000002               59171          0          4        450        500
subm:ast_device_state_topic-00000004               59170          0          4        450        500
subm:ast_device_state_topic-0000000f               59147          0          4        450        500
subm:ast_parking-00000009                              3          0          1        450        500
subm:ast_presence_state_topic_all-00000005             1          0          1        450        500
subm:ast_security-0000000e                         87358          0          3        450        500
subm:ast_system-00000008                              31          0         29        450        500
subm:ast_system-0000000a                              30          0          1        450        500
subm:cdr_engine-00000003                          673174          0         83       4500       5000
subm:endpoint_topic_all-cached-00000007           717252          0         35        450        500
subm:manager_topic-00000006                       820145          0        236       2700       3000
subp:12966@from-internal-sip-0000002a                  2          0          1        450        500
subp:12969@from-internal-sip-00000024                  4          0          2        450        500
subp:12969@from-internal-sip-00000025                  2          0          2        450        500
subp:SIP/12966-00000027                                2          0          2        450        500
subp:SIP/12966-00000029                                8          0          2        450        500
subp:SIP/12967-00000026                                8          0          2        450        500
subp:SIP/12968-0000002c                                8          0          2        450        500
subp:SIP/12969-00000020                                2          0          2        450        500
subp:SIP/12969-00000021                                2          0          2        450        500
subp:SIP/12969-00000023                                8          0          2        450        500
subp:SIP/12970-0000002b                                8          0          2        450        500
subp:SIP/sipp-0000001f                            689437          0         54        450        500

36 taskprocessors

Sounds like you need to potentially send more calls to it.

As well - while this app_queue fix I don’t think will impact you it has gone through code review and will be in the 13.11 release.

The leak is pretty slow. I can manage it until 13.11 is released. I’ll also look in to updating the unixodbc driver and dependencies. Is there an expected release date? I see the release candidate is available. I’ll download it and try it out in test.

A week or two I’d expect. I have no explicit date as it depends on things.

Sounds good. Thanks.

Tested 13.11-rc1 in test, and same as before, I can’t find a problem. I’ll wait for the official release, then try it in live. I was also thinking about the ODBC connections. I do run quite a few, but I also need them to run in series so I’m not sure that multi-threading would be good for this dialplan. I haven’t seen any bottlenecks yet.

asterisk-whconfig01*CLI> core show version
Asterisk 13.11.0-rc1 built by root @ asterisk-whconfig01 on a x86_64 running Linux on 2016-08-29 18:15:29 UTC
asterisk-whconfig01*CLI> core show taskprocessors

Processor                                      Processed   In Queue  Max Depth  Low water High water
app_voicemail                                          0          0          0        450        500
ast_msg_queue                                          0          0          0        450        500
CCSS_core                                              0          0          0        450        500
Sorcery                                                0          0          0        450        500
Sorcery-control                                        1          0          1        450        500
sorcery/bucket-00000000                                0          0          0        450        500
sorcery/file-00000001                                  0          0          0        450        500
stasis-core                                         6312          0          4        450        500
stasis-core-control                                12684          0          4        450        500
subm:ast_bridge_topic_all-cached-0000000c            545          0         14        450        500
subm:ast_channel_topic_all-00000010                 5320          0         23        450        500
subm:ast_channel_topic_all-cached-0000000b          7386          0         38        450        500
subm:ast_channel_topic_all-cached-0000000d          7385          0         38        450        500
subm:ast_device_state_topic-00000002                 695          0          4        450        500
subm:ast_device_state_topic-00000004                 694          0          4        450        500
subm:ast_device_state_topic-0000000f                 671          0          6        450        500
subm:ast_parking-00000009                              7          0          1        450        500
subm:ast_presence_state_topic_all-00000005             1          0          1        450        500
subm:ast_security-0000000e                          4809          0          1        450        500
subm:ast_system-00000008                              32          0         30        450        500
subm:ast_system-0000000a                              31          0          1        450        500
subm:cdr_engine-00000003                            7939          0         42       4500       5000
subm:endpoint_topic_all-cached-00000007             7607          0         38        450        500
subm:manager_topic-00000006                        13663          0         42       2700       3000
subp:12966@from-internal-sip-00000013                  4          0          2        450        500
subp:12966@from-internal-sip-00000015                  3          0          1        450        500
subp:12969@from-internal-sip-00000017                  1          0          1        450        500
subp:SIP/12966-00000012                               47          0         13        450        500
subp:SIP/12966-00000014                             1017          0         23        450        500
subp:SIP/12967-00000011                             3995          0         38        450        500
subp:SIP/12968-0000001c                                6          0          2        450        500
subp:SIP/12969-00000016                             1606          0         12        450        500
subp:SIP/12970-00000018                              519          0         13        450        500
subp:SIP/gateway-whconfig01-0000001b                 271          0         12        450        500

34 taskprocessors

Glad to hear it! And yes, multiple connections are good for a lot of cases but certainly not if you need to ensure stuff is serialized.

Update: I added two more offices yesterday. It now has 60 phones registered. Set noload for several more modules that I’m not using. Restarted asterisk last night. The server (well, NUC - mini-pc) ran over 1000 calls today, and the memory increased to 9.1% (of 16 GB) today. It is still on 13.5. Will be adding 75 more phones in the near future, so hopefully 13.11 will fix the memory problem I seem to have.

Aside from the memory leak(s), the system seems to run really well. I haven’t seen the cpu spike over 5% of one core yet.

So I see 13.11 is officially released. Downloading now. Are there any “gotchas” you can thing of off the top of your head? I’ll install it next week, in test first.

Nope, none come to mind. If there were they would be in the UPGRADE.txt document but we rarely make any breaking changes between releases like that unless we absolutely have to.

Well, it’s still a no-go. I upgraded to 13.11, ran two calls and got the same warning:
[Sep 6 20:21:31] WARNING[11668][C-00000001]: taskprocessor.c:887 taskprocessor_push: The ‘subp:SIP/gateway-twc01-00000011’ task processor queue reached 500 scheduled tasks.

asterisk-twc01*CLI> core show taskprocessors

Processor                                      Processed   In Queue  Max Depth  Low water High water
app_voicemail                                          0          0          0        450        500
ast_msg_queue                                          0          0          0        450        500
CCSS_core                                              0          0          0        450        500
Sorcery                                                0          0          0        450        500
Sorcery-control                                        1          0          1        450        500
sorcery/bucket-00000000                                0          0          0        450        500
sorcery/file-00000001                                  0          0          0        450        500
stasis-core                                          248        117          2        450        500
stasis-core-control                                  674          0          4        450        500
subm:ast_bridge_topic_all-cached-0000000b              1          0          1        450        500
subm:ast_channel_topic_all-00000010                  561          0          6        450        500
subm:ast_channel_topic_all-cached-0000000c           754          0          8        450        500
subm:ast_channel_topic_all-cached-0000000d           753          0          8        450        500
subm:ast_device_state_topic-00000002                 135          0          6        450        500
subm:ast_device_state_topic-00000004                 134          0          5        450        500
subm:ast_device_state_topic-0000000f                  49          0          2        450        500
subm:ast_parking-00000009                              1          0          1        450        500
subm:ast_presence_state_topic_all-00000005             1          0          1        450        500
subm:ast_security-0000000e                           225          0         10        450        500
subm:ast_system-00000008                              32          0         30        450        500
subm:ast_system-0000000a                              31          0          1        450        500
subm:cdr_engine-00000003                             757          0          8       4500       5000
subm:endpoint_topic_all-cached-00000007             1496          0          6        450        500
subm:manager_topic-00000006                         1232          0          8       2700       3000
subp:61001@from-internal-sip-00000085                  0          2          0        450        500
subp:61001@from-internal-sip-00000087                  0          2          0        450        500
subp:61001@from-internal-sip-000000b4                  0          2          0        450        500
subp:61001@from-internal-sip-000000c2                  0          1          0        450        500
subp:61008@from-internal-sip-0000001b                  0          2          0        450        500
subp:61008@from-internal-sip-0000001d                  0          2          0        450        500
subp:61008@from-internal-sip-0000007f                  0          2          0        450        500
subp:61008@from-internal-sip-00000081                  0          2          0        450        500
subp:61008@from-internal-sip-00000099                  0          2          0        450        500
subp:61008@from-internal-sip-0000009b                  0          2          0        450        500
subp:61008@from-internal-sip-000000df                  0          2          0        450        500
subp:61008@from-internal-sip-000000e3                  0          1          0        450        500
subp:61009@from-internal-sip-000000c6                  0          1          0        450        500
subp:61013@from-internal-sip-000000e5                  0          1          0        450        500
subp:61015@from-internal-sip-0000003a                  0          2          0        450        500
subp:61015@from-internal-sip-0000003c                  0          2          0        450        500
subp:61015@from-internal-sip-0000006a                  0          2          0        450        500
subp:61015@from-internal-sip-0000006c                  0          2          0        450        500
subp:61015@from-internal-sip-000000fd                  0          1          0        450        500
subp:61016@from-internal-sip-0000003e                  1          0          1        450        500
subp:61017@from-internal-sip-000000e1                  0          1          0        450        500
subp:61018@from-internal-sip-0000009d                  0          1          0        450        500
subp:61023@from-internal-sip-0000005e                  0          2          0        450        500
subp:61023@from-internal-sip-00000060                  0          2          0        450        500
subp:61023@from-internal-sip-00000109                  0          1          0        450        500
subp:61027@from-internal-sip-0000002c                  0          2          0        450        500
subp:61027@from-internal-sip-00000033                  0          2          0        450        500
subp:61027@from-internal-sip-00000091                  0          2          0        450        500
subp:61027@from-internal-sip-00000093                  0          2          0        450        500
subp:61027@from-internal-sip-000000a1                  0          2          0        450        500
subp:61027@from-internal-sip-000000b2                  0          1          0        450        500
subp:61028@from-internal-sip-00000072                  1          0          1        450        500
subp:61029@from-internal-sip-00000022                  0          2          0        450        500
subp:61029@from-internal-sip-00000024                  1          0          1        450        500
subp:61030@from-internal-sip-0000002a                  0          2          0        450        500
subp:61030@from-internal-sip-0000002e                  0          2          0        450        500
subp:61030@from-internal-sip-0000004b                  0          2          0        450        500
subp:61030@from-internal-sip-0000004d                  0          2          0        450        500
subp:61030@from-internal-sip-000000d8                  0          1          0        450        500
subp:61031@from-internal-sip-00000089                  0          2          0        450        500
subp:61031@from-internal-sip-0000008b                  0          2          0        450        500
subp:61031@from-internal-sip-000000fb                  0          1          0        450        500
subp:61033@from-internal-sip-000000da                  0          1          0        450        500
subp:61034@from-internal-sip-00000074                  0          2          0        450        500
subp:61034@from-internal-sip-00000079                  0          2          0        450        500
subp:61034@from-internal-sip-0000009f                  0          1          0        450        500
subp:61035@from-internal-sip-000000ec                  0          1          0        450        500
subp:61036@from-internal-sip-0000006e                  0          2          0        450        500
subp:61036@from-internal-sip-00000070                  0          2          0        450        500
subp:61036@from-internal-sip-000000ce                  0          1          0        450        500
subp:61037@from-internal-sip-000000c4                  0          1          0        450        500
subp:61038@from-internal-sip-00000095                  0          2          0        450        500
subp:61038@from-internal-sip-00000097                  0          2          0        450        500
subp:61038@from-internal-sip-000000f6                  0          1          0        450        500
subp:61039@from-internal-sip-00000066                  0          2          0        450        500
subp:61039@from-internal-sip-00000068                  1          1          1        450        500
subp:61039@from-internal-sip-000000f2                  0          1          0        450        500
subp:71001@from-internal-sip-00000014                  0          2          0        450        500
subp:71001@from-internal-sip-00000016                  0          2          0        450        500
subp:71001@from-internal-sip-00000040                  0          2          0        450        500
subp:71001@from-internal-sip-00000042                  0          2          0        450        500
subp:71001@from-internal-sip-00000054                  0          2          0        450        500
subp:71001@from-internal-sip-00000056                  9          3          1        450        500
subp:71001@from-internal-sip-00000062                  0          2          0        450        500
subp:71001@from-internal-sip-00000064                  0          2          0        450        500
subp:71001@from-internal-sip-0000007b                  0          2          0        450        500
subp:71001@from-internal-sip-0000007d                  0          2          0        450        500
subp:71001@from-internal-sip-000000a9                  0          3          0        450        500
subp:71001@from-internal-sip-000000af                  0          2          0        450        500
subp:71001@from-internal-sip-000000d0                  0          1          0        450        500
subp:71002@from-internal-sip-00000047                  0          2          0        450        500
subp:71002@from-internal-sip-00000049                  5          1          1        450        500
subp:71002@from-internal-sip-0000008d                  0          2          0        450        500
subp:71002@from-internal-sip-0000008f                  0          2          0        450        500
subp:71002@from-internal-sip-000000a4                  0          1          0        450        500
subp:71003@from-internal-sip-00000026                  0          2          0        450        500
subp:71003@from-internal-sip-00000028                  0          2          0        450        500
subp:71003@from-internal-sip-00000036                  0          2          0        450        500
subp:71003@from-internal-sip-00000038                  0          2          0        450        500
subp:71003@from-internal-sip-00000052                  0          2          0        450        500
subp:71003@from-internal-sip-0000005b                  0          2          0        450        500
subp:71003@from-internal-sip-000000bd                  0          4          0        450        500
subp:71003@from-internal-sip-000000c8                  0          3          0        450        500
subp:71003@from-internal-sip-000000dc                  0          2          0        450        500
subp:71003@from-internal-sip-000000e8                  0          1          0        450        500
subp:SIP/61000-00000051                                2          0          2        450        500
subp:SIP/61000-0000005a                                2          0          2        450        500
subp:SIP/61000-000000bc                                0          4          0        450        500
subp:SIP/61001-00000084                                2          0          2        450        500
subp:SIP/61001-00000086                                2          0          2        450        500
subp:SIP/61001-000000b3                                0          2          0        450        500
subp:SIP/61002-000000b6                                0          5          0        450        500
subp:SIP/61002-000000b7                                0          5          0        450        500
subp:SIP/61002-000000b8                                0          5          0        450        500
subp:SIP/61002-000000c1                                0          2          0        450        500
subp:SIP/61003-00000034                                2          0          2        450        500
subp:SIP/61004-0000005c                                2          0          2        450        500
subp:SIP/61005-000000a2                                0          2          0        450        500
subp:SIP/61006-000000b5                                0          4          0        450        500
subp:SIP/61007-00000098                                2          0          2        450        500
subp:SIP/61007-0000009a                                2          0          2        450        500
subp:SIP/61007-000000ab                                0          5          0        450        500
subp:SIP/61007-000000ac                                0          5          0        450        500
subp:SIP/61007-000000ad                                0          5          0        450        500
subp:SIP/61007-000000de                                0          2          0        450        500
subp:SIP/61008-0000001a                                2          0          2        450        500
subp:SIP/61008-0000001c                                2          0          2        450        500
subp:SIP/61008-0000007e                                2          0          2        450        500
subp:SIP/61008-00000080                                2          0          2        450        500
subp:SIP/61008-000000e2                                0          2          0        450        500
subp:SIP/61009-000000b9                                0          5          0        450        500
subp:SIP/61009-000000ba                                0          5          0        450        500
subp:SIP/61009-000000bb                                0          5          0        450        500
subp:SIP/61009-000000c5                                0          4          0        450        500
subp:SIP/61011-00000046                                2          0          2        450        500
subp:SIP/61011-00000048                                2          0          2        450        500
subp:SIP/61012-0000008c                                2          0          2        450        500
subp:SIP/61012-0000008e                                2          0          2        450        500
subp:SIP/61012-000000a3                                0          2          0        450        500
subp:SIP/61013-000000e4                                0          2          0        450        500
subp:SIP/61014-000000d1                                0          5          0        450        500
subp:SIP/61014-000000d2                                0          5          0        450        500
subp:SIP/61014-000000d3                                0          5          0        450        500
subp:SIP/61014-000000e6                                0          2          0        450        500
subp:SIP/61015-00000039                                2          0          2        450        500
subp:SIP/61015-0000003b                                2          0          2        450        500
subp:SIP/61015-00000069                                2          0          2        450        500
subp:SIP/61015-0000006b                                2          0          2        450        500
subp:SIP/61015-000000fc                                0          2          0        450        500
subp:SIP/61016-0000003d                                2          0          2        450        500
subp:SIP/61017-000000e0                                0          2          0        450        500
subp:SIP/61018-0000009c                                0          2          0        450        500
subp:SIP/61019-00000012                                2          0          2        450        500
subp:SIP/61020-000000aa                                0          2          0        450        500
subp:SIP/61021-000000d4                                0          5          0        450        500
subp:SIP/61021-000000d5                                0          5          0        450        500
subp:SIP/61021-000000d6                                0          5          0        450        500
subp:SIP/61021-000000f9                                0          4          0        450        500
subp:SIP/61022-000000a5                                0          5          0        450        500
subp:SIP/61022-000000a6                                0          5          0        450        500
subp:SIP/61022-000000a7                                0          5          0        450        500
subp:SIP/61022-000000dd                                0          2          0        450        500
subp:SIP/61023-0000005d                                2          0          2        450        500
subp:SIP/61023-0000005f                                2          0          2        450        500
subp:SIP/61023-000000ca                                0          5          0        450        500
subp:SIP/61023-000000cb                                0          5          0        450        500
subp:SIP/61023-000000cc                                0          5          0        450        500
subp:SIP/61023-000000e9                                0          5          0        450        500
subp:SIP/61023-000000ea                                0          5          0        450        500
subp:SIP/61023-000000f3                                0          5          0        450        500
subp:SIP/61023-000000f4                                0          5          0        450        500
subp:SIP/61023-000000f7                                0          5          0        450        500
subp:SIP/61023-000000f8                                0          5          0        450        500
subp:SIP/61023-000000fe                                0          5          0        450        500
subp:SIP/61023-000000ff                                0          5          0        450        500
subp:SIP/61023-00000100                                0          5          0        450        500
subp:SIP/61023-00000101                                0          5          0        450        500
subp:SIP/61023-00000102                                0          5          0        450        500
subp:SIP/61023-00000103                                0          5          0        450        500
subp:SIP/61023-00000104                                0          5          0        450        500
subp:SIP/61023-00000105                                0          5          0        450        500
subp:SIP/61023-00000106                                0          5          0        450        500
subp:SIP/61023-00000107                                0          5          0        450        500
subp:SIP/61023-00000108                                0          4          0        450        500
subp:SIP/61024-0000007a                                2          0          2        450        500
subp:SIP/61024-0000007c                                2          0          2        450        500
subp:SIP/61024-000000ae                                0          2          0        450        500
subp:SIP/61025-00000013                                2          0          2        450        500
subp:SIP/61025-00000015                                2          0          2        450        500
subp:SIP/61025-00000053                                2          0          2        450        500
subp:SIP/61025-00000055                                2          2          2        450        500
subp:SIP/61026-0000003f                                2          0          2        450        500
subp:SIP/61026-00000041                                2          0          2        450        500
subp:SIP/61026-000000cf                                0          4          0        450        500
subp:SIP/61027-00000090                                2          0          2        450        500
subp:SIP/61027-00000092                                2          0          2        450        500
subp:SIP/61027-000000a0                                0          2          0        450        500
subp:SIP/61028-00000071                                2          2          2        450        500
subp:SIP/61029-00000021                                2          0          2        450        500
subp:SIP/61029-00000023                                2          0          2        450        500
subp:SIP/61030-00000029                                2          0          2        450        500
subp:SIP/61030-0000002d                                2          0          2        450        500
subp:SIP/61030-0000004a                                2          0          2        450        500
subp:SIP/61030-0000004c                                2          0          2        450        500
subp:SIP/61030-000000d7                                0          4          0        450        500
subp:SIP/61031-00000088                                2          0          2        450        500
subp:SIP/61031-0000008a                                2          0          2        450        500
subp:SIP/61031-000000be                                0          5          0        450        500
subp:SIP/61031-000000bf                                0          5          0        450        500
subp:SIP/61031-000000c0                                0          5          0        450        500
subp:SIP/61031-000000ed                                0          5          0        450        500
subp:SIP/61031-000000ee                                0          5          0        450        500
subp:SIP/61031-000000ef                                0          5          0        450        500
subp:SIP/61031-000000f0                                0          5          0        450        500
subp:SIP/61031-000000fa                                0          2          0        450        500
subp:SIP/61032-0000002b                                2          0          2        450        500
subp:SIP/61032-00000032                                2          0          2        450        500
subp:SIP/61032-000000b1                                0          2          0        450        500
subp:SIP/61033-000000d9                                0          4          0        450        500
subp:SIP/61034-00000073                                2          0          2        450        500
subp:SIP/61034-00000078                                2          0          2        450        500
subp:SIP/61034-0000009e                                0          4          0        450        500
subp:SIP/61035-000000eb                                0          4          0        450        500
subp:SIP/61036-0000006d                                2          0          2        450        500
subp:SIP/61036-0000006f                                2          0          2        450        500
subp:SIP/61036-000000cd                                0          4          0        450        500
subp:SIP/61037-000000c3                                0          4          0        450        500
subp:SIP/61038-00000094                                2          0          2        450        500
subp:SIP/61038-00000096                                2          0          2        450        500
subp:SIP/61038-000000f5                                0          2          0        450        500
subp:SIP/61039-000000f1                                0          2          0        450        500
subp:SIP/61040-000000c9                                0          2          0        450        500
subp:SIP/61041-00000065                                2          0          2        450        500
subp:SIP/61041-00000067                                2          0          2        450        500
subp:SIP/61043-000000b0                                0          2          0        450        500
subp:SIP/61044-0000010a                                0          4          0        450        500
subp:SIP/61047-000000e7                                0          2          0        450        500
subp:SIP/61048-00000025                                2          0          2        450        500
subp:SIP/61048-00000027                                2          0          2        450        500
subp:SIP/61048-000000db                                0          2          0        450        500
subp:SIP/61049-00000035                                2          0          2        450        500
subp:SIP/61049-00000037                                2          0          2        450        500
subp:SIP/61049-000000c7                                0          2          0        450        500
subp:SIP/62966-00000061                                2          0          2        450        500
subp:SIP/62966-00000063                                2          0          2        450        500
subp:SIP/62966-000000a8                                0          2          0        450        500
subp:SIP/gateway-twc01-00000011                      209        545          9        450        500

251 taskprocessors

I ran a backtrace after the warning, but nothing jumps out at me.
https://drive.google.com/open?id=0BxUsrDyPCTFALVpkb0xyYUZ1TGs

Rolled back to 13.5 for now.

Looking at the backtrace this appears to be an already reported deadlock in chan_sip[1] when using realtime and mailboxes.

[1] https://issues.asterisk.org/jira/browse/ASTERISK-25468

When I ran “core show locks” after the warning last night, it was blank. I’ll try it again tonight. There doesn’t appear to be a fix except to change to pjsip or dump realtime voicemail (the aft isnot an option for me). Considering that, I may be opening another can of worms. It appears that this problem was introduced in V12. Perhaps I should move to V11. The only change I’d need to make is the res_parking -> features.

Asterisk 11 does not use the same event system and does not have the same code in that area, so it probably wouldn’t exhibit the same problem. Just be aware that 11 is going security fix only the end of October. I’ve also bumped up the issue for chan_sip against 13 internally.