PJSIP/ARA - ASTERISK frozen (but not crashed) after UnixODBC/backend interaction

Asterisk Asterisk Support
1

Hi,
in our scenario we have 4000 extensions generated by pjsip_wizard and ARA is enabled for PJSIP.

At the moment we have registered only two extensions, as per testing purpose, and when we run the command “pjsip show endpoints” from CLI, ASTERISK freezes but no taskprocessor logs found.

The SIP clients are still registered, but unabled to make calls.
In the part of Dialplan where we allow the internal calls between extension, there are some FUNC_ODBC functions.

Here the “pjsip set logger on” and VERBOSE when problems occur.
We can see that asterisk can handle the incoming request, but it can’t handle the outgoing one.

— LOG PJSIP —

[Aug 25 07:23:26] VERBOSE[1657] res_pjsip_logger.c: <--- Received SIP request (894 bytes) from UDP:10.1.1.2:5061 --->
INVITE sip:1564@10.80.1.50 SIP/2.0
Via: SIP/2.0/UDP 10.1.1.2:5061;rport;branch=z9hG4bKPj28852148a1714e8b8043e8a901593f93
Max-Forwards: 70
From: "1077" <sip:1077@10.80.1.50>;tag=3d277c9f9f334cf59fad1a2f5f0859c4
To: <sip:1564@10.80.1.50>
Contact: "1077" <sip:1077@10.1.1.2:5061;ob>
Call-ID: 69f6c934cadd4bdaab1aea3cdf48bf92
CSeq: 28631 INVITE
Allow: PRACK, INVITE, ACK, BYE, CANCEL, UPDATE, INFO, SUBSCRIBE, NOTIFY, REFER, MESSAGE, OPTIONS
Supported: replaces, 100rel, norefersub
User-Agent: MicroSIP/3.21.2
Content-Type: application/sdp
Content-Length:   315

v=0
o=- 3870404605 3870404605 IN IP4 10.1.1.2
s=pjmedia
b=AS:84
t=0 0
a=X-nat:0
m=audio 10000 RTP/AVP 8 101
c=IN IP4 10.1.1.2
b=TIAS:64000
a=rtcp:10001 IN IP4 10.1.1.2
a=sendrecv
a=rtpmap:8 PCMA/8000
a=rtpmap:101 telephone-event/8000
a=fmtp:101 0-16
a=ssrc:1917858426 cname:399b45190a1c2e48

[Aug 25 07:23:26] VERBOSE[2757] res_pjsip_logger.c: <--- Transmitting SIP response (541 bytes) to UDP:10.1.1.2:5061 --->
SIP/2.0 401 Unauthorized
Via: SIP/2.0/UDP 10.1.1.2:5061;rport=5061;received=10.1.1.2;branch=z9hG4bKPj28852148a1714e8b8043e8a901593f93
Call-ID: 69f6c934cadd4bdaab1aea3cdf48bf92
From: "1077" <sip:1077@10.80.1.50>;tag=3d277c9f9f334cf59fad1a2f5f0859c4
To: <sip:1564@10.80.1.50>;tag=z9hG4bKPj28852148a1714e8b8043e8a901593f93
CSeq: 28631 INVITE
WWW-Authenticate: Digest realm="asterisk",nonce="1661408606/6f960416faf2f29a37736017365d673d",opaque="3ff26712095ad2ae",algorithm=MD5,qop="auth"
Server: asterisk
Content-Length:  0


[Aug 25 07:23:26] VERBOSE[1657] res_pjsip_logger.c: <--- Received SIP request (369 bytes) from UDP:10.1.1.2:5061 --->
ACK sip:1564@10.80.1.50 SIP/2.0
Via: SIP/2.0/UDP 10.1.1.2:5061;rport;branch=z9hG4bKPj28852148a1714e8b8043e8a901593f93
Max-Forwards: 70
From: "1077" <sip:1077@10.80.1.50>;tag=3d277c9f9f334cf59fad1a2f5f0859c4
To: <sip:1564@10.80.1.50>;tag=z9hG4bKPj28852148a1714e8b8043e8a901593f93
Call-ID: 69f6c934cadd4bdaab1aea3cdf48bf92
CSeq: 28631 ACK
Content-Length:  0


[Aug 25 07:23:26] VERBOSE[1657] res_pjsip_logger.c: <--- Received SIP request (1184 bytes) from UDP:10.1.1.2:5061 --->
INVITE sip:1564@10.80.1.50 SIP/2.0
Via: SIP/2.0/UDP 10.1.1.2:5061;rport;branch=z9hG4bKPj0e394bf5c55444cdaf404bb52914f517
Max-Forwards: 70
From: "1077" <sip:1077@10.80.1.50>;tag=3d277c9f9f334cf59fad1a2f5f0859c4
To: <sip:1564@10.80.1.50>
Contact: "1077" <sip:1077@10.1.1.2:5061;ob>
Call-ID: 69f6c934cadd4bdaab1aea3cdf48bf92
CSeq: 28632 INVITE
Allow: PRACK, INVITE, ACK, BYE, CANCEL, UPDATE, INFO, SUBSCRIBE, NOTIFY, REFER, MESSAGE, OPTIONS
Supported: replaces, 100rel, norefersub
User-Agent: MicroSIP/3.21.2
Authorization: Digest username="1077", realm="asterisk", nonce="1661408606/6f960416faf2f29a37736017365d673d", uri="sip:1564@10.80.1.50", response="b6521c4996dcb69388122e2eed68d790", algorithm=MD5, cnonce="6d5fb172c74643acae6eb015844e0d50", opaque="3ff26712095ad2ae", qop=auth, nc=00000001
Content-Type: application/sdp
Content-Length:   315

v=0
o=- 3870404605 3870404605 IN IP4 10.1.1.2
s=pjmedia
b=AS:84
t=0 0
a=X-nat:0
m=audio 10000 RTP/AVP 8 101
c=IN IP4 10.1.1.2
b=TIAS:64000
a=rtcp:10001 IN IP4 10.1.1.2
a=sendrecv
a=rtpmap:8 PCMA/8000
a=rtpmap:101 telephone-event/8000
a=fmtp:101 0-16
a=ssrc:1917858426 cname:399b45190a1c2e48

[Aug 25 07:23:27] VERBOSE[2757] res_pjsip_logger.c: <--- Transmitting SIP response (343 bytes) to UDP:10.1.1.2:5061 --->
SIP/2.0 100 Trying
Via: SIP/2.0/UDP 10.1.1.2:5061;rport=5061;received=10.1.1.2;branch=z9hG4bKPj0e394bf5c55444cdaf404bb52914f517
Call-ID: 69f6c934cadd4bdaab1aea3cdf48bf92
From: "1077" <sip:1077@10.80.1.50>;tag=3d277c9f9f334cf59fad1a2f5f0859c4
To: <sip:1564@10.80.1.50>
CSeq: 28632 INVITE
Server: asterisk
Content-Length:  0


[... DP VERBOSE ...]
[Aug 25 07:23:27] VERBOSE[2793][C-00000001] pbx.c: Executing [start@subcall:4] Answer("PJSIP/1077-00000000", "") in new stack

[Aug 25 07:23:27] VERBOSE[2757] res_rtp_asterisk.c: 0x14b6f0102a50 -- Strict RTP learning after remote address set to: 10.1.1.2:10000
[Aug 25 07:23:27] VERBOSE[2757] res_pjsip_logger.c: <--- Transmitting SIP response (782 bytes) to UDP:10.1.1.2:5061 --->
SIP/2.0 200 OK
Via: SIP/2.0/UDP 10.1.1.2:5061;rport=5061;received=10.1.1.2;branch=z9hG4bKPj0e394bf5c55444cdaf404bb52914f517
Call-ID: 69f6c934cadd4bdaab1aea3cdf48bf92
From: "1077" <sip:1077@10.80.1.50>;tag=3d277c9f9f334cf59fad1a2f5f0859c4
To: <sip:1564@10.80.1.50>;tag=09ce900f-2902-41fe-b61a-e88c6c48a349
CSeq: 28632 INVITE
Server: asterisk
Contact: <sip:10.80.1.50:5060>
Allow: OPTIONS, REGISTER, SUBSCRIBE, NOTIFY, PUBLISH, INVITE, ACK, BYE, CANCEL, UPDATE, PRACK, MESSAGE, REFER
Supported: 100rel, timer, replaces, norefersub
Content-Type: application/sdp
Content-Length:   177

v=0
o=- 3870404605 3870404607 IN IP4 10.80.1.50
s=Asterisk
c=IN IP4 10.80.1.50
t=0 0
m=audio 18602 RTP/AVP 8
a=rtpmap:8 PCMA/8000
a=ptime:20
a=maxptime:150
a=sendrecv

[Aug 25 07:23:27] VERBOSE[1657] res_pjsip_logger.c: <--- Received SIP request (364 bytes) from UDP:10.1.1.2:5061 --->
ACK sip:10.80.1.50:5060 SIP/2.0
Via: SIP/2.0/UDP 10.1.1.2:5061;rport;branch=z9hG4bKPjafd4a1534cf84bac85af8b993beb975f
Max-Forwards: 70
From: "1077" <sip:1077@10.80.1.50>;tag=3d277c9f9f334cf59fad1a2f5f0859c4
To: <sip:1564@10.80.1.50>;tag=09ce900f-2902-41fe-b61a-e88c6c48a349
Call-ID: 69f6c934cadd4bdaab1aea3cdf48bf92
CSeq: 28632 ACK
Content-Length:  0

[... DP VERBOSE ...]

[Aug 25 07:23:27] VERBOSE[2793][C-00000001] pbx.c: Executing [start@subcall:22] Dial("PJSIP/1077-00000000", "PJSIP/1564,20,rtTIi") in new stack

We have the following environment:

Ubuntu 20.04 - Linux/5.4.0-124-generic built by root on x86_64 2022-08-24 22:18:22 UTC
ASTERISK 18.12.1
PJPROJECT version currently running against: 2.12
UnixODBC 2.3.6
odbc-postgresql 1:12.01.0000-1

Limit:

Limit                     Soft Limit           Hard Limit           Units
Max cpu time              unlimited            unlimited            seconds
Max file size             unlimited            unlimited            bytes
Max data size             unlimited            unlimited            bytes
Max stack size            unlimited            unlimited            bytes
Max core file size        unlimited            unlimited            bytes
Max resident set          unlimited            unlimited            bytes
Max processes             unlimited            unlimited            processes
Max open files            1024000              1024000              files
Max locked memory         unlimited            unlimited            bytes
Max address space         unlimited            unlimited            bytes
Max file locks            unlimited            unlimited            locks
Max pending signals       unlimited            unlimited            signals
Max msgqueue size         unlimited            unlimited            bytes
Max nice priority         0                    0
Max realtime priority     70                   70
Max realtime timeout      unlimited            unlimited            us

PBX Settings:

PBX Core settings
-----------------
  Version:                     18.12.1
  Build Options:               DONT_OPTIMIZE, COMPILE_DOUBLE, DEBUG_THREADS, DEBUG_FD_LEAKS, BETTER_BACKTRACES, OPTIONAL_API
  Maximum calls:               Not set
  Maximum open file handles:   1024000
  Root console verbosity:      6
  Current console verbosity:   6
  Debug level:                 6
  Trace level:                 0
  Dump core on crash:          Yes
  Core dump file:              |/usr/share/apport/apport -p%p -s%s -c%c -d%d -P%P -u%u -g%g -- %E
  Maximum load average:        0.000000
  Minimum free memory:         0 MB
  Startup time:                00:06:37
  Last reload time:            00:06:37
  System:                      Linux/5.4.0-124-generic built by root on x86_64 2022-08-24 22:18:22 UTC
  System name:
  Entity ID:                   00:50:56:9f:04:56
  PBX UUID:                    6b52f041-e0ae-4f32-893c-f61f34368752
  Default language:            es
  Language prefix:             Enabled
  User name and group:         /
  Running directory:           /var/lib/asterisk
  Executable includes:         Disabled
  Transcode via SLIN:          Enabled
  Transmit silence during rec: Disabled
  Generic PLC:                 Enabled
  Generic PLC on equal codecs: Disabled
  Hide Msg Chan AMI events:    Disabled
  Min DTMF duration::          80
  Cache media frames:          Enabled
  RTP use dynamic payloads:    1
  RTP dynamic payload types:   35-63,96-127

* Subsystems
  -------------
  Manager (AMI):               Disabled
  Web Manager (AMI/HTTP):      Disabled
  Call data records:           Disabled
  Realtime Architecture (ARA): Enabled

* Directories
  -------------
  Configuration file:          /etc/asterisk/asterisk.conf
  Configuration directory:     /etc/asterisk
  Module directory:            /usr/lib/asterisk/modules
  Spool directory:             /var/spool/asterisk
  Log directory:               /var/log/asterisk
  Run/Sockets directory:       /var/run/asterisk
  PID file:                    /var/run/asterisk/asterisk.pid
  VarLib directory:            /var/lib/asterisk
  Data directory:              /var/lib/asterisk
  ASTDB:                       /var/lib/asterisk/astdb
  IAX2 Keys directory:         /var/lib/asterisk/keys
  AGI Scripts directory:       /var/lib/asterisk/agi-bin

We can reproduce the error whenever we want and we are able to export a coredump with ast_coredumper (–running).

Please, for more configurations see the comment below (limit of 32000 characteres on this topic)

What could be the problem of this freeze?
Could be a ODBC driver issue?

Any help will be appreciated

Many thanks

2

Modules:

Module                         Description                              Use Count  Status      Support Level
acl                            Named ACL system                         2          Running              core
app_attended_transfer.so       Attended transfer to the given extension 0          Running          extended
app_authenticate.so            Authentication Application               0          Running              core
app_blind_transfer.so          Blind transfer channel to the given dest 0          Running          extended
app_bridgeaddchan.so           Bridge Add Channel Application           0          Running              core
app_bridgewait.so              Place the channel into a holding bridge  0          Running              core
app_cdr.so                     Tell Asterisk to not maintain a CDR for  0          Running              core
app_celgenuserevent.so         Generate an User-Defined CEL event       0          Running              core
app_chanisavail.so             Check channel availability               0          Running          extended
app_channelredirect.so         Redirects a given channel to a dialplan  0          Running              core
app_chanspy.so                 Listen to the audio of an active channel 1          Running              core
app_controlplayback.so         Control Playback Application             0          Running              core
app_dial.so                    Dialing Application                      1          Running              core
app_dictate.so                 Virtual Dictation Machine                0          Running          extended
app_directed_pickup.so         Directed Call Pickup Application         0          Running              core
app_directory.so               Extension Directory                      0          Running              core
app_dumpchan.so                Dump Info About The Calling Channel      0          Running              core
app_echo.so                    Simple Echo Application                  0          Running              core
app_exec.so                    Executes dialplan applications           0          Running              core
app_forkcdr.so                 Fork The CDR into 2 separate entities    0          Running              core
app_mixmonitor.so              Mixed Audio Monitoring Application       0          Running              core
app_originate.so               Originate call                           0          Running              core
app_playback.so                Sound File Playback Application          0          Running              core
app_playtones.so               Playtones Application                    0          Running              core
app_privacy.so                 Require phone number to be entered, if n 0          Running              core
app_queue.so                   True Call Queueing                       0          Running              core
app_read.so                    Read Variable Application                0          Running              core
app_readexten.so               Read and evaluate extension validity     0          Running              core
app_record.so                  Trivial Record Application               0          Running              core
app_reload.so                  Reload module(s)                         0          Running          extended
app_sayunixtime.so             Say time                                 0          Running              core
app_senddtmf.so                Send DTMF digits Application             0          Running              core
app_speech_utils.so            Dialplan Speech Applications             0          Running              core
app_stack.so                   Dialplan subroutines (Gosub, Return, etc 1          Running              core
app_stasis.so                  Stasis dialplan application              0          Running              core
app_stream_echo.so             Stream Echo Application                  0          Running              core
app_system.so                  Generic System() application             0          Running              core
app_talkdetect.so              Playback with Talk Detection             0          Running              core
app_test.so                    Interface Test Application               0          Running          extended
app_transfer.so                Transfers a caller to another extension  0          Running              core
app_userevent.so               Custom User Event Application            0          Running              core
app_verbose.so                 Send verbose output                      0          Running              core
app_voicemail.so               Comedian Mail (Voicemail System)         0          Running              core
app_while.so                   While Loops and Conditional Execution    0          Running              core
bridge_builtin_features.so     Built in bridging features               1          Running              core
bridge_builtin_interval_features.so Built in bridging interval features      0          Running              core
bridge_holding.so              Holding bridge module                    0          Running              core
bridge_native_rtp.so           Native RTP bridging module               0          Running              core
bridge_simple.so               Simple two channel bridging module       0          Running              core
bridge_softmix.so              Multi-party software based channel mixin 0          Running              core
ccss                           Call Completion Supplementary Services   2          Running              core
cdr                            CDR Engine                               5          Running              core
cdr_adaptive_odbc.so           Adaptive ODBC CDR backend                0          Running              core
cdr_odbc.so                    ODBC CDR Backend                         0          Running          extended
cel                            CEL Engine                               2          Running              core
chan_bridge_media.so           Bridge Media Channel Driver              0          Running              core
chan_pjsip.so                  PJSIP Channel Driver                     5          Running              core
codec_alaw.so                  A-law Coder/Decoder                      0          Running              core
dnsmgr                         DNS Manager                              2          Running              core
dsp                            DSP                                      1          Running              core
enum                           ENUM Support                             1          Running              core
extconfig                      Configuration                            14         Running              core
features                       Call Features                            1          Running              core
format_g723.so                 G.723.1 Simple Timestamp File Format     0          Running              core
format_g726.so                 Raw G.726 (16/24/32/40kbps) data         0          Running              core
format_g729.so                 Raw G.729 data                           0          Running              core
format_gsm.so                  Raw GSM data                             0          Running              core
format_h263.so                 Raw H.263 data                           0          Running              core
format_h264.so                 Raw H.264 data                           0          Running              core
format_ilbc.so                 Raw iLBC data                            0          Running              core
format_ogg_vorbis.so           OGG/Vorbis audio                         0          Running              core
format_pcm.so                  Raw/Sun uLaw/ALaw 8KHz (PCM,PCMA,AU), G. 0          Running              core
format_siren14.so              ITU G.722.1 Annex C (Siren14, licensed f 0          Running              core
format_siren7.so               ITU G.722.1 (Siren7, licensed from Polyc 0          Running              core
format_sln.so                  Raw Signed Linear Audio support (SLN) 8k 0          Running              core
format_vox.so                  Dialogic VOX (ADPCM) File Format         0          Running          extended
format_wav.so                  Microsoft WAV/WAV16 format (8kHz/16kHz S 0          Running              core
format_wav_gsm.so              Microsoft WAV format (Proprietary GSM)   0          Running              core
func_callerid.so               Party ID related dialplan functions (Cal 0          Running              core
func_cdr.so                    Call Detail Record (CDR) dialplan functi 1          Running              core
func_channel.so                Channel information dialplan functions   0          Running              core
func_config.so                 Asterisk configuration file variable acc 0          Running              core
func_cut.so                    Cut out information from a string        1          Running              core
func_devstate.so               Gets or sets a device state in the dialp 0          Running              core
func_dialgroup.so              Dialgroup dialplan function              0          Running              core
func_dialplan.so               Dialplan Context/Extension/Priority Chec 0          Running              core
func_env.so                    Environment/filesystem dialplan function 0          Running              core
func_extstate.so               Gets an extension's state in the dialpla 0          Running              core
func_global.so                 Variable dialplan functions              0          Running              core
func_groupcount.so             Channel group dialplan functions         1          Running              core
func_hangupcause.so            HANGUPCAUSE related functions and applic 0          Running              core
func_jitterbuffer.so           Jitter buffer for read side of channel.  0          Running              core
func_logic.so                  Logical dialplan functions               0          Running              core
func_math.so                   Mathematical dialplan function           0          Running              core
func_module.so                 Checks if Asterisk module is loaded in m 0          Running              core
func_odbc.so                   ODBC lookups                             0          Running              core
func_periodic_hook.so          Periodic dialplan hooks.                 1          Running              core
func_pjsip_aor.so              Get information about a PJSIP AOR        0          Running              core
func_pjsip_contact.so          Get information about a PJSIP contact    0          Running              core
func_pjsip_endpoint.so         Get information about a PJSIP endpoint   0          Running              core
func_presencestate.so          Gets or sets a presence state in the dia 0          Running              core
func_shell.so                  Collects the output generated by a comma 0          Running              core
func_strings.so                String handling dialplan functions       0          Running              core
func_timeout.so                Channel timeout dialplan functions       0          Running              core
func_uri.so                    URI encode/decode dialplan functions     1          Running              core
http                           Built-in HTTP Server                     4          Running              core
indications                    Indication Tone Handling                 1          Running              core
logger                         Logger                                   1          Running              core
manager                        Asterisk Manager Interface               1          Running              core
pbx_config.so                  Text Extension Configuration             0          Running              core
pbx_loopback.so                Loopback Switch                          0          Running              core
pbx_realtime.so                Realtime Switch                          0          Running          extended
pbx_spool.so                   Outgoing Spool Support                   0          Running              core
plc                            PLC                                      1          Running              core
res_clialiases.so              CLI Aliases                              0          Running              core
res_clioriginate.so            Call origination and redirection from th 0          Running              core
res_config_odbc.so             Realtime ODBC configuration              0          Running              core
res_convert.so                 File format conversion CLI command       0          Running              core
res_crypto.so                  Cryptographic Digital Signatures         0          Running              core
res_curl.so                    cURL Resource Module                     1          Running              core
res_fax.so                     Generic FAX Applications                 1          Running              core
res_fax_spandsp.so             Spandsp G.711 and T.38 FAX Technologies  0          Running          extended
res_format_attr_celt.so        CELT Format Attribute Module             1          Running              core
res_format_attr_g729.so        G.729 Format Attribute Module            1          Running              core
res_format_attr_h263.so        H.263 Format Attribute Module            1          Running              core
res_format_attr_h264.so        H.264 Format Attribute Module            1          Running              core
res_format_attr_ilbc.so        iLBC Format Attribute Module             1          Running              core
res_format_attr_opus.so        Opus Format Attribute Module             1          Running              core
res_format_attr_silk.so        SILK Format Attribute Module             1          Running              core
res_format_attr_siren14.so     Siren14 Format Attribute Module          1          Running              core
res_format_attr_siren7.so      Siren7 Format Attribute Module           1          Running              core
res_format_attr_vp8.so         VP8 Format Attribute Module              1          Running              core
res_http_media_cache.so        HTTP Media Cache Backend                 1          Running              core
res_http_post.so               HTTP POST support                        0          Running              core
res_http_websocket.so          HTTP WebSocket Support                   1          Running              core
res_limit.so                   Resource limits                          0          Running              core
res_musiconhold.so             Music On Hold Resource                   0          Running              core
res_mutestream.so              Mute audio stream resources              0          Running              core
res_odbc.so                    ODBC resource                            5          Running              core
res_odbc_transaction.so        ODBC transaction resource                1          Running              core
res_parking.so                 Call Parking Resource                    0          Running              core
res_pjproject.so               PJPROJECT Log and Utility Support        4          Running              core
res_pjsip.so                   Basic SIP resource                       44         Running              core
res_pjsip_acl.so               PJSIP ACL Resource                       0          Running              core
res_pjsip_authenticator_digest.so PJSIP authentication resource            0          Running              core
res_pjsip_caller_id.so         PJSIP Caller ID Support                  2          Running              core
res_pjsip_config_wizard.so     PJSIP Config Wizard                      1          Running              core
res_pjsip_dialog_info_body_generator.so PJSIP Extension State Dialog Info+XML Pr 0          Running              core
res_pjsip_diversion.so         PJSIP Add Diversion Header Support       2          Running              core
res_pjsip_dlg_options.so       SIP OPTIONS in dialog handler            1          Running              core
res_pjsip_dtmf_info.so         PJSIP DTMF INFO Support                  1          Running              core
res_pjsip_empty_info.so        PJSIP Empty INFO Support                 1          Running              core
res_pjsip_endpoint_identifier_anonymous.so PJSIP Anonymous endpoint identifier      0          Running              core
res_pjsip_endpoint_identifier_ip.so PJSIP IP endpoint identifier             0          Running              core
res_pjsip_endpoint_identifier_user.so PJSIP username endpoint identifier       0          Running              core
res_pjsip_exten_state.so       PJSIP Extension State Notifications      0          Running              core
res_pjsip_header_funcs.so      PJSIP Header Functions                   1          Running              core
res_pjsip_history.so           PJSIP History                            0          Running          extended
res_pjsip_logger.so            PJSIP Packet Logger                      0          Running              core
res_pjsip_messaging.so         PJSIP Messaging Support                  1          Running              core
res_pjsip_nat.so               PJSIP NAT Support                        1          Running              core
res_pjsip_notify.so            CLI/AMI PJSIP NOTIFY Support             0          Running              core
res_pjsip_one_touch_record_info.so PJSIP INFO One Touch Recording Support   1          Running              core
res_pjsip_outbound_authenticator_digest.so PJSIP authentication resource            0          Running              core
res_pjsip_outbound_publish.so  PJSIP Outbound Publish Support           2          Running              core
res_pjsip_outbound_registration.so PJSIP Outbound Registration Support      0          Running              core
res_pjsip_path.so              PJSIP Path Header Support                1          Running              core
res_pjsip_pidf_body_generator.so PJSIP Extension State PIDF Provider      0          Running              core
res_pjsip_pidf_digium_body_supplement.so PJSIP PIDF Sangoma presence supplement   0          Running              core
res_pjsip_pidf_eyebeam_body_supplement.so PJSIP PIDF Eyebeam supplement            0          Running              core
res_pjsip_publish_asterisk.so  PJSIP Asterisk Event PUBLISH Support     0          Running              core
res_pjsip_pubsub.so            PJSIP event resource                     10         Running              core
res_pjsip_refer.so             PJSIP Blind and Attended Transfer Suppor 2          Running              core
res_pjsip_registrar.so         PJSIP Registrar Support                  0          Running              core
res_pjsip_rfc3326.so           PJSIP RFC3326 Support                    1          Running              core
res_pjsip_sdp_rtp.so           PJSIP SDP RTP/AVP stream handler         1          Running              core
res_pjsip_send_to_voicemail.so PJSIP REFER Send to Voicemail Support    1          Running              core
res_pjsip_session.so           PJSIP Session resource                   17         Running              core
res_pjsip_sips_contact.so      UAC SIPS Contact support                 0          Running              core
res_pjsip_t38.so               PJSIP T.38 UDPTL Support                 2          Running              core
res_pjsip_transport_websocket.so PJSIP WebSocket Transport Support        1          Running              core
res_pjsip_xpidf_body_generator.so PJSIP Extension State PIDF Provider      0          Running              core
res_realtime.so                Realtime Data Lookup/Rewrite             0          Running              core
res_resolver_unbound.so        Unbound DNS Resolver Support             1          Running              core
res_rtp_asterisk.so            Asterisk RTP Stack                       1          Running              core
res_rtp_multicast.so           Multicast RTP Engine                     0          Running              core
res_security_log.so            Security Event Logging                   0          Running              core
res_sorcery_astdb.so           Sorcery Astdb Object Wizard              2          Running              core
res_sorcery_config.so          Sorcery Configuration File Object Wizard 10         Running              core
res_sorcery_memory.so          Sorcery In-Memory Object Wizard          7          Running              core
res_sorcery_memory_cache.so    Sorcery Memory Cache Object Wizard       1          Running              core
res_sorcery_realtime.so        Sorcery Realtime Object Wizard           8          Running              core
res_speech.so                  Generic Speech Recognition API           1          Running              core
res_srtp.so                    Secure RTP (SRTP)                        0          Running              core
res_stasis.so                  Stasis application support               6          Running              core
res_stasis_answer.so           Stasis application answer support        0          Running              core
res_stasis_device_state.so     Stasis application device state support  0          Running              core
res_stasis_playback.so         Stasis application playback support      0          Running              core
res_stasis_recording.so        Stasis application recording support     1          Running              core
res_stasis_snoop.so            Stasis application snoop support         0          Running              core
sounds                         Sounds Index                             1          Running              core
udptl                          UDPTL                                    2          Running              core
202 modules loaded

res_odbc.conf file:

[asterisk]
enabled => yes
logging => yes
dsn => asterisk
max_connections => 9
sanitysql => select 1
pre-connect => yes

[asterisk-cdr]
enabled => yes
logging => no
dsn => asterisk-cdr
max_connections => 3
sanitysql => select 1
pre-connect => yes

[asterisk-internal]
enabled => yes
logging => yes
dsn => asterisk-internal
max_connections => 3
sanitysql => select 1
pre-connect => yes

extconfig.conf file:

[settings]
ps_endpoints => odbc,asterisk
ps_auths => odbc,asterisk
ps_aors => odbc,asterisk
ps_domain_aliases => odbc,asterisk
ps_endpoint_id_ips => odbc,asterisk
ps_outbound_publishes => odbc,asterisk
ps_inbound_publications => odbc,asterik
ps_asterisk_publications => odbc,asterisk
ps_transports => odbc,asterisk
ps_contacts => odbc,asterisk
ps_globals => odbc,asterisk
ps_registrations => odbc,asterisk
ps_resource_list => odbc,asterisk
ps_subscription_persistence => odbc,asterisk
ps_systems => odbc,asterisk

sorcery.conf file:

[res_pjsip]
; globals
;global/cache=memory_cache,object_lifetime_stale=300,object_lifetime_maximum=1800,expire_on_reload=yes
global=realtime,ps_globals
global=config,pjsip.conf,criteria=type=global

; systems
;system/cache=memory_cache,object_lifetime_stale=300,object_lifetime_maximum=1800,expire_on_reload=yes
system=realtime,ps_systems
system=config,pjsip.conf,criteria=type=system

; transports
;transport/cache=memory_cache,object_lifetime_stale=300,object_lifetime_maximum=0,expire_on_reload=yes
transport=realtime,ps_transports

; endpoints
;endpoint/cache=memory_cache,object_lifetime_stale=300,object_lifetime_maximum=0,expire_on_reload=yes
endpoint=realtime,ps_endpoints
endpoint=config,pjsip.conf,criteria=type=endpoint

; auths
;auth/cache=memory_cache,object_lifetime_stale=300,object_lifetime_maximum=0,expire_on_reload=yes
auth=realtime,ps_auths
auth=config,pjsip.conf,criteria=type=auth

; aors
;aor/cache=memory_cache,object_lifetime_stale=300,object_lifetime_maximum=0,expire_on_reload=yes
aor=realtime,ps_aors
aor=config,pjsip.conf,criteria=type=aor

; contacts
contact/cache=memory_cache,object_lifetime_stale=300,object_lifetime_maximum=1800,expire_on_reload=yes
contact=realtime,ps_contacts

; domain aliases
domain_alias=realtime,ps_domain_aliases

[res_pjsip_endpoint_identifier_ip]
identify=realtime,ps_endpoint_id_ips
identify=config,pjsip.conf,criteria=type=identify

[res_pjsip_outbound_publish]
outbound-publish=realtime,ps_outbound_publishes
outbound-publish=config,pjsip.conf,criteria=type=outbound-publish

[res_pjsip_pubsub]
inbound-publication=realtime,ps_inbound_publications

[res_pjsip_publish_asterisk]
asterisk-publication=realtime,ps_asterisk_publications

We removed the cache settings because we thought that was the problem.

3

A running backtrace would show where Asterisk is blocked. When realtime is involved, it is almost always the database itself in some way.

4

I took it; where can I attach full core-dump?

brief:

!@!@!@! brief.txt !@!@!@!

$2 = {si_signo = 19, si_errno = 0, si_code = 128, _sifields = {_pad = {0 <repeats 28 times>}, _kill = {si_pid = 0, si_uid = 0}, _timer = {si_tid = 0, si_overrun = 0, si_sigval = {sival_int = 0, sival_ptr = 0x0}}, _rt = {si_pid = 0, si_uid = 0, si_sigval = {sival_int = 0, sival_ptr = 0x0}}, _sigchld = {si_pid = 0, si_uid = 0, si_status = 0, si_utime = 0, si_stime = 0}, _sigfault = {si_addr = 0x0, _addr_lsb = 0, _addr_bnd = {_lower = 0x0, _upper = 0x0}}, _sigpoll = {si_band = 0, si_fd = 0}}}
Signal        Stop      Print   Pass to program Description

Thread 51 (Thread 0x14fd98d15700 (LWP 18230)):
#0  futex_abstimed_wait_cancelable (private=<optimized out>, abstime=0x14fd98d14d80, clockid=<optimized out>, expected=0, futex_word=0x14fdb4014850) at ../sysdeps/nptl/futex-internal.h:320
#1  __pthread_cond_wait_common (abstime=0x14fd98d14d80, clockid=<optimized out>, mutex=0x14fdb4014858, cond=0x14fdb4014828) at pthread_cond_wait.c:520
#2  __pthread_cond_timedwait (cond=0x14fdb4014828, mutex=0x14fdb4014858, abstime=0x14fd98d14d80) at pthread_cond_wait.c:665
#3  0x000055c27a7ecd51 in __ast_cond_timedwait (filename=0x55c27a994dde "threadpool.c", lineno=1163, func=0x55c27a9955a8 <__PRETTY_FUNCTION__.9408> "worker_idle", cond_name=0x55c27a99505f "&worker->cond", mutex_name=0x55c27a995051 "&worker->lock", cond=0x14fdb4014828, t=0x14fdb4014858, abstime=0x14fd98d14d80) at lock.c:653
#4  0x000055c27a8ba2f8 in worker_idle (worker=0x14fdb4014820) at threadpool.c:1163
#5  0x000055c27a8b9fc1 in worker_start (arg=0x14fdb4014820) at threadpool.c:1054
#6  0x000055c27a8c6e44 in dummy_start (data=0x14fdbc059110) at utils.c:1574
#7  0x000014fddbbeb609 in start_thread (arg=<optimized out>) at pthread_create.c:477
#8  0x000014fddb824133 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Thread 50 (Thread 0x14fd9810f700 (LWP 18229)):
#0  futex_abstimed_wait_cancelable (private=<optimized out>, abstime=0x14fd9810ed80, clockid=<optimized out>, expected=0, futex_word=0x14fdb40128d0) at ../sysdeps/nptl/futex-internal.h:320
#1  __pthread_cond_wait_common (abstime=0x14fd9810ed80, clockid=<optimized out>, mutex=0x14fdb40128d8, cond=0x14fdb40128a8) at pthread_cond_wait.c:520
#2  __pthread_cond_timedwait (cond=0x14fdb40128a8, mutex=0x14fdb40128d8, abstime=0x14fd9810ed80) at pthread_cond_wait.c:665
#3  0x000055c27a7ecd51 in __ast_cond_timedwait (filename=0x55c27a994dde "threadpool.c", lineno=1163, func=0x55c27a9955a8 <__PRETTY_FUNCTION__.9408> "worker_idle", cond_name=0x55c27a99505f "&worker->cond", mutex_name=0x55c27a995051 "&worker->lock", cond=0x14fdb40128a8, t=0x14fdb40128d8, abstime=0x14fd9810ed80) at lock.c:653
#4  0x000055c27a8ba2f8 in worker_idle (worker=0x14fdb40128a0) at threadpool.c:1163
#5  0x000055c27a8b9fc1 in worker_start (arg=0x14fdb40128a0) at threadpool.c:1054
#6  0x000055c27a8c6e44 in dummy_start (data=0x14fdc00c2810) at utils.c:1574
#7  0x000014fddbbeb609 in start_thread (arg=<optimized out>) at pthread_create.c:477
#8  0x000014fddb824133 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Thread 49 (Thread 0x14fd9818c700 (LWP 18228)):
#0  futex_abstimed_wait_cancelable (private=<optimized out>, abstime=0x14fd9818bd80, clockid=<optimized out>, expected=0, futex_word=0x14fdb4010550) at ../sysdeps/nptl/futex-internal.h:320
#1  __pthread_cond_wait_common (abstime=0x14fd9818bd80, clockid=<optimized out>, mutex=0x14fdb4010558, cond=0x14fdb4010528) at pthread_cond_wait.c:520
#2  __pthread_cond_timedwait (cond=0x14fdb4010528, mutex=0x14fdb4010558, abstime=0x14fd9818bd80) at pthread_cond_wait.c:665
#3  0x000055c27a7ecd51 in __ast_cond_timedwait (filename=0x55c27a994dde "threadpool.c", lineno=1163, func=0x55c27a9955a8 <__PRETTY_FUNCTION__.9408> "worker_idle", cond_name=0x55c27a99505f "&worker->cond", mutex_name=0x55c27a995051 "&worker->lock", cond=0x14fdb4010528, t=0x14fdb4010558, abstime=0x14fd9818bd80) at lock.c:653
#4  0x000055c27a8ba2f8 in worker_idle (worker=0x14fdb4010520) at threadpool.c:1163
#5  0x000055c27a8b9fc1 in worker_start (arg=0x14fdb4010520) at threadpool.c:1054
#6  0x000055c27a8c6e44 in dummy_start (data=0x14fda00185b0) at utils.c:1574
#7  0x000014fddbbeb609 in start_thread (arg=<optimized out>) at pthread_create.c:477
#8  0x000014fddb824133 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Thread 48 (Thread 0x14fd98209700 (LWP 18227)):
#0  futex_abstimed_wait_cancelable (private=<optimized out>, abstime=0x14fd98208d80, clockid=<optimized out>, expected=0, futex_word=0x14fdb400c910) at ../sysdeps/nptl/futex-internal.h:320
#1  __pthread_cond_wait_common (abstime=0x14fd98208d80, clockid=<optimized out>, mutex=0x14fdb400c918, cond=0x14fdb400c8e8) at pthread_cond_wait.c:520
#2  __pthread_cond_timedwait (cond=0x14fdb400c8e8, mutex=0x14fdb400c918, abstime=0x14fd98208d80) at pthread_cond_wait.c:665
#3  0x000055c27a7ecd51 in __ast_cond_timedwait (filename=0x55c27a994dde "threadpool.c", lineno=1163, func=0x55c27a9955a8 <__PRETTY_FUNCTION__.9408> "worker_idle", cond_name=0x55c27a99505f "&worker->cond", mutex_name=0x55c27a995051 "&worker->lock", cond=0x14fdb400c8e8, t=0x14fdb400c918, abstime=0x14fd98208d80) at lock.c:653
#4  0x000055c27a8ba2f8 in worker_idle (worker=0x14fdb400c8e0) at threadpool.c:1163
#5  0x000055c27a8b9fc1 in worker_start (arg=0x14fdb400c8e0) at threadpool.c:1054
#6  0x000055c27a8c6e44 in dummy_start (data=0x14fdbc00df10) at utils.c:1574
#7  0x000014fddbbeb609 in start_thread (arg=<optimized out>) at pthread_create.c:477
#8  0x000014fddb824133 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Thread 47 (Thread 0x14fdd96ba700 (LWP 18226)):
#0  futex_abstimed_wait_cancelable (private=<optimized out>, abstime=0x14fdd96b9d80, clockid=<optimized out>, expected=0, futex_word=0x14fdb40109f4) at ../sysdeps/nptl/futex-internal.h:320
#1  __pthread_cond_wait_common (abstime=0x14fdd96b9d80, clockid=<optimized out>, mutex=0x14fdb40109f8, cond=0x14fdb40109c8) at pthread_cond_wait.c:520
#2  __pthread_cond_timedwait (cond=0x14fdb40109c8, mutex=0x14fdb40109f8, abstime=0x14fdd96b9d80) at pthread_cond_wait.c:665
#3  0x000055c27a7ecd51 in __ast_cond_timedwait (filename=0x55c27a994dde "threadpool.c", lineno=1163, func=0x55c27a9955a8 <__PRETTY_FUNCTION__.9408> "worker_idle", cond_name=0x55c27a99505f "&worker->cond", mutex_name=0x55c27a995051 "&worker->lock", cond=0x14fdb40109c8, t=0x14fdb40109f8, abstime=0x14fdd96b9d80) at lock.c:653
#4  0x000055c27a8ba2f8 in worker_idle (worker=0x14fdb40109c0) at threadpool.c:1163
#5  0x000055c27a8b9fc1 in worker_start (arg=0x14fdb40109c0) at threadpool.c:1054
#6  0x000055c27a8c6e44 in dummy_start (data=0x14fda001b450) at utils.c:1574
#7  0x000014fddbbeb609 in start_thread (arg=<optimized out>) at pthread_create.c:477
#8  0x000014fddb824133 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Thread 46 (Thread 0x14fd98c1b700 (LWP 18209)):
#0  futex_wait_cancelable (private=<optimized out>, expected=0, futex_word=0x14fd98c16750) at ../sysdeps/nptl/futex-internal.h:183
#1  __pthread_cond_wait_common (abstime=0x0, clockid=0, mutex=0x14fd98c166f0, cond=0x14fd98c16728) at pthread_cond_wait.c:508
#2  __pthread_cond_wait (cond=0x14fd98c16728, mutex=0x14fd98c166f0) at pthread_cond_wait.c:647
#3  0x000055c27a7ec7b3 in __ast_cond_wait (filename=0x14fd9b2b90e9 "res_pjsip.c", lineno=1990, func=0x14fd9b2baff0 <__PRETTY_FUNCTION__.28492> "ast_sip_push_task_wait", cond_name=0x14fd9b2b9f23 "&std.cond", mutex_name=0x14fd9b2b9f19 "&std.lock", cond=0x14fd98c16728, t=0x14fd98c166f0) at lock.c:588
#4  0x000014fd9b27e888 in ast_sip_push_task_wait (serializer=0x0, sip_task=0x14fd9ae299e1 <request>, task_data=0x14fd98c16860) at res_pjsip.c:1990
#5  0x000014fd9b27e972 in ast_sip_push_task_wait_servant (serializer=0x0, sip_task=0x14fd9ae299e1 <request>, task_data=0x14fd98c16860) at res_pjsip.c:2005
#6  0x000014fd9ae2a6de in chan_pjsip_request_with_stream_topology (type=0x14fda005aba8 "PJSIP", topology=0x14fda0056c28, assignedids=0x0, requestor=0x14fdbd0f06e0, data=0x14fda005abb4 "6584", cause=0x14fd98c16a40) at chan_pjsip.c:2700
#7  0x000055c27a77282b in request_channel (type=0x14fda005aba8 "PJSIP", request_cap=0x0, topology=0x14fda0056c28, assignedids=0x0, requestor=0x14fdbd0f06e0, addr=0x14fda005abb4 "6584", cause=0x14fd98c16a40) at channel.c:6339
#8  0x000055c27a773024 in ast_request_with_stream_topology (type=0x14fda005aba8 "PJSIP", topology=0x14fda0056c28, assignedids=0x0, requestor=0x14fdbd0f06e0, addr=0x14fda005abb4 "6584", cause=0x14fd98c16a40) at channel.c:6439
#9  0x000014fdd8c1942a in dial_exec_full (chan=0x14fdbd0f06e0, data=0x14fd98c18ae0 "PJSIP/6584,20,rtTIi", peerflags=0x14fd98c17940, continue_exec=0x0) at app_dial.c:2696
#10 0x000014fdd8c1d8d4 in dial_exec (chan=0x14fdbd0f06e0, data=0x14fd98c18ae0 "PJSIP/6584,20,rtTIi") at app_dial.c:3508
#11 0x000055c27a8262d5 in pbx_exec (c=0x14fdbd0f06e0, app=0x55c28a8787e0, data=0x14fd98c18ae0 "PJSIP/6584,20,rtTIi") at pbx_app.c:492
#12 0x000055c27a80f2ce in pbx_extension_helper (c=0x14fdbd0f06e0, con=0x0, context=0x14fdbd0f10a0 "subcallv3", exten=0x14fdbd0f10f0 "start", priority=22, label=0x0, callerid=0x14fdbcede470 "5401", action=E_SPAWN, found=0x14fd98c1abb0, combined_find_spawn=1) at pbx.c:2945
#13 0x000055c27a8139e5 in ast_spawn_extension (c=0x14fdbd0f06e0, context=0x14fdbd0f10a0 "subcallv3", exten=0x14fdbd0f10f0 "start", priority=22, callerid=0x14fdbcede470 "5401", found=0x14fd98c1abb0, combined_find_spawn=1) at pbx.c:4200
#14 0x000055c27a8147ab in __ast_pbx_run (c=0x14fdbd0f06e0, args=0x0) at pbx.c:4374
#15 0x000055c27a816126 in pbx_thread (data=0x14fdbd0f06e0) at pbx.c:4698
#16 0x000055c27a8c6e44 in dummy_start (data=0x14fdbd0f3d00) at utils.c:1574
#17 0x000014fddbbeb609 in start_thread (arg=<optimized out>) at pthread_create.c:477
#18 0x000014fddb824133 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Thread 45 (Thread 0x14fd98b21700 (LWP 18193)):
#0  __lll_lock_wait (futex=futex@entry=0x55c27cacfa70, private=0) at lowlevellock.c:52
#1  0x000014fddbbee131 in __GI___pthread_mutex_lock (mutex=0x55c27cacfa70) at ../nptl/pthread_mutex_lock.c:115
#2  0x000055c27a7eb60e in __ast_pthread_mutex_lock (filename=0x14fd9b2bfe1f "res_pjsip/config_transport.c", lineno=654, func=0x14fd9b2c0d80 <__PRETTY_FUNCTION__.28128> "transport_apply", mutex_name=0x14fd9b2bfebc "states", t=0x55c27cacfa70) at lock.c:326
#3  0x000055c27a71da75 in __ao2_lock (user_data=0x55c27cacfac0, lock_how=AO2_LOCK_REQ_WRLOCK, file=0x14fd9b2bfe1f "res_pjsip/config_transport.c", func=0x14fd9b2c0d80 <__PRETTY_FUNCTION__.28128> "transport_apply", line=654, var=0x14fd9b2bfebc "states") at astobj2.c:241
#4  0x000014fd9b296b1a in transport_apply (sorcery=0x55c27c81d2e0, obj=0x14fdc002ede8) at res_pjsip/config_transport.c:654
#5  0x000055c27a87dbdc in ast_sorcery_objectset_apply (sorcery=0x55c27c81d2e0, object=0x14fdc002ede8, objectset=0x14fdc002eb90) at sorcery.c:1657
#6  0x000014fdd8e43a0e in sorcery_realtime_retrieve_fields (sorcery=0x55c27c81d2e0, data=0x55c27c06b470, type=0x55c27c069dd0 "transport", fields=0x14fdc0018d60) at res_sorcery_realtime.c:178
#7  0x000014fdd8e43b47 in sorcery_realtime_retrieve_id (sorcery=0x55c27c81d2e0, data=0x55c27c06b470, type=0x55c27c069dd0 "transport", id=0x14fd98b20630 "transport-udp") at res_sorcery_realtime.c:190
#8  0x000055c27a87e74e in ast_sorcery_retrieve_by_id (sorcery=0x55c27c81d2e0, type=0x14fd9b2b983b "transport", id=0x14fd98b20630 "transport-udp") at sorcery.c:1875
#9  0x000014fd9b27a588 in ast_sip_set_tpselector_from_transport_name (transport_name=0x14fd98b20630 "transport-udp", selector=0x14fd98b207b0) at res_pjsip.c:760
#10 0x000014fd9b27a68a in ast_sip_set_tpselector_from_ep_or_uri (endpoint=0x55c283e696f8, sip_uri=0x14fdc0018b58, selector=0x14fd98b207b0) at res_pjsip.c:779
#11 0x000014fd9b279d80 in ast_sip_dlg_set_transport (endpoint=0x55c283e696f8, dlg=0x14fdc00185f8, selector=0x14fd98b207b0) at res_pjsip.c:597
#12 0x000014fd9b27aa65 in ast_sip_create_dialog_uac (endpoint=0x55c283e696f8, uri=0x14fdc001729a "sip:6584@10.20.11.18:55468;rinstance=971ae3cd46e22a90", request_user=0x0) at res_pjsip.c:858
#13 0x000014fd9ae99fa7 in ast_sip_session_create_outgoing (endpoint=0x55c283e696f8, contact=0x0, location=0x55c283e63fdc "6584", request_user=0x0, req_topology=0x14fda0056c28) at res_pjsip_session.c:3355
#14 0x000014fd9ae2a2bc in request (obj=0x14fd98c16860) at chan_pjsip.c:2673
#15 0x000014fd9b27e642 in sync_task (data=0x14fd98c166f0) at res_pjsip.c:1956
#16 0x000055c27a8b026b in ast_taskprocessor_execute (tps=0x55c27c06ff00) at taskprocessor.c:1302
#17 0x000055c27a8baa5e in execute_tasks (data=0x55c27c06ff00) at threadpool.c:1350
#18 0x000055c27a8b026b in ast_taskprocessor_execute (tps=0x55c27c06fde0) at taskprocessor.c:1302
#19 0x000055c27a8b8379 in threadpool_execute (pool=0x55c27c0632e0) at threadpool.c:367
#20 0x000055c27a8ba203 in worker_active (worker=0x14fdb400b010) at threadpool.c:1137
#21 0x000055c27a8b9f66 in worker_start (arg=0x14fdb400b010) at threadpool.c:1056
#22 0x000055c27a8c6e44 in dummy_start (data=0x14fdbc006240) at utils.c:1574
#23 0x000014fddbbeb609 in start_thread (arg=<optimized out>) at pthread_create.c:477
#24 0x000014fddb824133 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Thread 44 (Thread 0x14fd98c98700 (LWP 18192)):
#0  futex_wait_cancelable (private=<optimized out>, expected=0, futex_word=0x55c27aa5d80c <as_cond+44>) at ../sysdeps/nptl/futex-internal.h:183
#1  __pthread_cond_wait_common (abstime=0x0, clockid=0, mutex=0x55c27aa37630 <aslist+16>, cond=0x55c27aa5d7e0 <as_cond>) at pthread_cond_wait.c:508
#2  __pthread_cond_wait (cond=0x55c27aa5d7e0 <as_cond>, mutex=0x55c27aa37630 <aslist+16>) at pthread_cond_wait.c:647
#3  0x000055c27a7ec7b3 in __ast_cond_wait (filename=0x55c27a95856a "autoservice.c", lineno=102, func=0x55c27a9586c0 <__PRETTY_FUNCTION__.15383> "autoservice_run", cond_name=0x55c27a958585 "&as_cond", mutex_name=0x55c27a958578 "&aslist.lock", cond=0x55c27aa5d7e0 <as_cond>, t=0x55c27aa37630 <aslist+16>) at lock.c:588
#4  0x000055c27a72a7d7 in autoservice_run (ign=0x0) at autoservice.c:102
#5  0x000055c27a8c6e44 in dummy_start (data=0x14fdb800f9f0) at utils.c:1574
#6  0x000014fddbbeb609 in start_thread (arg=<optimized out>) at pthread_create.c:477
#7  0x000014fddb824133 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Thread 43 (Thread 0x14fd98d92700 (LWP 18185)):
#0  futex_wait_cancelable (private=<optimized out>, expected=0, futex_word=0x14fd98d90840) at ../sysdeps/nptl/futex-internal.h:183
#1  __pthread_cond_wait_common (abstime=0x0, clockid=0, mutex=0x14fd98d907e0, cond=0x14fd98d90818) at pthread_cond_wait.c:508
#2  __pthread_cond_wait (cond=0x14fd98d90818, mutex=0x14fd98d907e0) at pthread_cond_wait.c:647
#3  0x000055c27a7ec7b3 in __ast_cond_wait (filename=0x14fd9b2b90e9 "res_pjsip.c", lineno=1990, func=0x14fd9b2baff0 <__PRETTY_FUNCTION__.28492> "ast_sip_push_task_wait", cond_name=0x14fd9b2b9f23 "&std.cond", mutex_name=0x14fd9b2b9f19 "&std.lock", cond=0x14fd98d90818, t=0x14fd98d907e0) at lock.c:588
#4  0x000014fd9b27e888 in ast_sip_push_task_wait (serializer=0x0, sip_task=0x14fd9b295ff2 <destroy_sip_transport_state>, task_data=0x14fdb034c550) at res_pjsip.c:1990
#5  0x000014fd9b27e972 in ast_sip_push_task_wait_servant (serializer=0x0, sip_task=0x14fd9b295ff2 <destroy_sip_transport_state>, task_data=0x14fdb034c550) at res_pjsip.c:2005
#6  0x000014fd9b2960d8 in sip_transport_state_destroy (obj=0x14fdb034c550) at res_pjsip/config_transport.c:440
#7  0x000055c27a71e919 in __ao2_ref (user_data=0x14fdb034c550, delta=-1, tag=0x0, file=0x14fd9b2bfe1f "res_pjsip/config_transport.c", line=450, func=0x14fd9b2c0c50 <__PRETTY_FUNCTION__.28046> "internal_state_destroy") at astobj2.c:615
#8  0x000055c27a71ecab in __ao2_cleanup_debug (obj=0x14fdb034c550, tag=0x0, file=0x14fd9b2bfe1f "res_pjsip/config_transport.c", line=450, function=0x14fd9b2c0c50 <__PRETTY_FUNCTION__.28046> "internal_state_destroy") at astobj2.c:673
#9  0x000014fd9b296165 in internal_state_destroy (obj=0x14fdb035b5c0) at res_pjsip/config_transport.c:450
#10 0x000055c27a71e919 in __ao2_ref (user_data=0x14fdb035b5c0, delta=-1, tag=0x0, file=0x14fd9b2bfe1f "res_pjsip/config_transport.c", line=638, func=0x14fd9b2c0da0 <__PRETTY_FUNCTION__.28118> "_dtor_perm_state") at astobj2.c:615
#11 0x000055c27a71ecab in __ao2_cleanup_debug (obj=0x14fdb035b5c0, tag=0x0, file=0x14fd9b2bfe1f "res_pjsip/config_transport.c", line=638, function=0x14fd9b2c0da0 <__PRETTY_FUNCTION__.28118> "_dtor_perm_state") at astobj2.c:673
#12 0x000014fd9b297d02 in _dtor_perm_state (v=0x14fd98d90c60) at res_pjsip/config_transport.c:638
#13 0x000014fd9b297c3c in transport_apply (sorcery=0x55c27c81d2e0, obj=0x14fdb0350128) at res_pjsip/config_transport.c:638
#14 0x000055c27a87dbdc in ast_sorcery_objectset_apply (sorcery=0x55c27c81d2e0, object=0x14fdb0350128, objectset=0x14fdb034a3e0) at sorcery.c:1657
#15 0x000014fdd8e43a0e in sorcery_realtime_retrieve_fields (sorcery=0x55c27c81d2e0, data=0x55c27c06b470, type=0x55c27c069dd0 "transport", fields=0x14fdb00a29b0) at res_sorcery_realtime.c:178
#16 0x000014fdd8e43b47 in sorcery_realtime_retrieve_id (sorcery=0x55c27c81d2e0, data=0x55c27c06b470, type=0x55c27c069dd0 "transport", id=0x55c282e94474 "transport-udp") at res_sorcery_realtime.c:190
#17 0x000055c27a87e74e in ast_sorcery_retrieve_by_id (sorcery=0x55c27c81d2e0, type=0x14fd9b2bfe4c "transport", id=0x55c282e94474 "transport-udp") at sorcery.c:1875
#18 0x000014fd9b29a67b in cli_iterate (container=0x55c282e9ac38, callback=0x14fd9b29a797 <cli_print_body>, args=0x14fd98d91360) at res_pjsip/config_transport.c:1488
#19 0x000014fd9b2a6a4a in cli_endpoint_print_child_body (type=0x14fd9b2c2821 "transport", obj=0x55c282e9ac38, context=0x14fd98d91360) at res_pjsip/pjsip_configuration.c:1908
#20 0x000014fd9b2a6dbf in cli_endpoint_print_body (obj=0x55c282e9ac38, arg=0x14fd98d91360, flags=2) at res_pjsip/pjsip_configuration.c:1954
#21 0x000055c27a720663 in internal_ao2_traverse (self=0x14fdb000d728, flags=OBJ_NODATA, cb_fn=0x14fd9b2a6aae <cli_endpoint_print_body>, arg=0x14fd98d91360, data=0x0, type=AO2_CALLBACK_DEFAULT, tag=0x0, file=0x14fd9b2c5ef7 "res_pjsip/pjsip_cli.c", line=215, func=0x14fd9b2c6200 <__PRETTY_FUNCTION__.28340> "ast_sip_cli_traverse_objects") at astobj2_container.c:328
#22 0x000055c27a720952 in __ao2_callback (c=0x14fdb000d728, flags=OBJ_NODATA, cb_fn=0x14fd9b2a6aae <cli_endpoint_print_body>, arg=0x14fd98d91360, tag=0x0, file=0x14fd9b2c5ef7 "res_pjsip/pjsip_cli.c", line=215, func=0x14fd9b2c6200 <__PRETTY_FUNCTION__.28340> "ast_sip_cli_traverse_objects") at astobj2_container.c:414
#23 0x000014fd9b2b4c5d in ast_sip_cli_traverse_objects (e=0x14fd9b2d6988 <cli_commands+200>, cmd=-4, a=0x14fd98d91440) at res_pjsip/pjsip_cli.c:215
#24 0x000055c27a794d38 in ast_cli_command_full (uid=0, gid=0, fd=17, s=0x14fd98d91710 "pjsip show endpoints ") at cli.c:3020
#25 0x000055c27a794f2e in ast_cli_command_multiple_full (uid=0, gid=0, fd=17, size=22, s=0x14fd98d919c0 "pjsip show endpoints ") at cli.c:3047
#26 0x000055c27a712152 in netconsole (vconsole=0x55c27aa66ea0 <consoles>) at asterisk.c:1464
#27 0x000055c27a8c6e44 in dummy_start (data=0x14fdb8002d20) at utils.c:1574
#28 0x000014fddbbeb609 in start_thread (arg=<optimized out>) at pthread_create.c:477
#29 0x000014fddb824133 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Thread 42 (Thread 0x14fd98286700 (LWP 18053)):
#0  futex_abstimed_wait_cancelable (private=0, abstime=0x0, clockid=0, expected=0, futex_word=0x55c28a9866b0) at ../sysdeps/nptl/futex-internal.h:320
#1  do_futex_wait (sem=sem@entry=0x55c28a9866b0, abstime=0x0, clockid=0) at sem_waitcommon.c:112
#2  0x000014fddbbf5548 in __new_sem_wait_slow (sem=0x55c28a9866b0, abstime=0x0, clockid=0) at sem_waitcommon.c:184
#3  0x000055c27a8acb20 in ast_sem_wait (sem=0x55c28a9866b0) at /usr/src/asterisk/asterisk-18.12.1/include/asterisk/sem.h:61
#4  0x000055c27a8acc11 in default_tps_processing_function (data=0x55c2896dd050) at taskprocessor.c:205
#5  0x000055c27a8c6e44 in dummy_start (data=0x55c28b52e3f0) at utils.c:1574
#6  0x000014fddbbeb609 in start_thread (arg=<optimized out>) at pthread_create.c:477
#7  0x000014fddb824133 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Thread 41 (Thread 0x14fd98303700 (LWP 18052)):
#0  futex_abstimed_wait_cancelable (private=0, abstime=0x0, clockid=0, expected=0, futex_word=0x55c28a95de30) at ../sysdeps/nptl/futex-internal.h:320
#1  do_futex_wait (sem=sem@entry=0x55c28a95de30, abstime=0x0, clockid=0) at sem_waitcommon.c:112
#2  0x000014fddbbf5548 in __new_sem_wait_slow (sem=0x55c28a95de30, abstime=0x0, clockid=0) at sem_waitcommon.c:184
#3  0x000055c27a8acb20 in ast_sem_wait (sem=0x55c28a95de30) at /usr/src/asterisk/asterisk-18.12.1/include/asterisk/sem.h:61
#4  0x000055c27a8acc11 in default_tps_processing_function (data=0x55c28a96f750) at taskprocessor.c:205
#5  0x000055c27a8c6e44 in dummy_start (data=0x55c2896dd0a0) at utils.c:1574
#6  0x000014fddbbeb609 in start_thread (arg=<optimized out>) at pthread_create.c:477
#7  0x000014fddb824133 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
5

The issue appears to be because you have set transports to pull from realtime. That’s not something commonly done, so it’s possible there’s a bug there. You really really want to just create your transports in the configuration file.

1 Like
6

You don’t. They are only useful on the machine that created them (or an almost identical one).

7

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.