Asterisk to Asterisk Trunk: Unable to create request with auth. & No matching endpoint found

Hello all,

I’m trying to connect two Asterisk boxes on my local network. Both Asterisks (Port:5080) are behind their respective Kamailios (Port 5060) and endpoints are connected through Websocket. Intra-Asterisk calls are working without an issue however I failed to place a call between asterisk boxes using trunk. I receive Unable to create request with auth. error on System A (10.0.0.13) & No matching endpoint found on System B (10.0.0.35)

I’ve followed the PJSIP trunk examples and created a trunk configuration as shown below. Since both boxes are on the same local network I do not want to create a password authentication for the trunk, instead I used identify. I am able to send the Invite packet from System A to System B however system B challenges my Invite with 401 Not Authorized for which my System A has no answer. I’ve also tried with password authentication without any luck. I normally use realtime PJSIP however for this issue I’ve used both config files and real time to configure the trunk but it made no difference.

I have a feeling that my identify settings are not loading OR for some reason not being associated with the trunk endpoint that I created. I feel like I’ve exhausted all the resources I could find about the topic and tried every possible combinations. I need your help. Please find my configurations and logs below.
System A Configs (10.0.0.13)


#####Sorcery#####
[res_pjsip] ; Realtime PJSIP configuration wizard
endpoint=realtime,ps_endpoints
endpoint=config,pjsip.conf,criteria=type=endpoint
auth=realtime,ps_auths
auth=config,pjsip.conf,criteria=type=auth
aor=realtime,ps_aors
aor=config,pjsip.conf,criteria=type=aor
domain_alias=realtime,ps_domain_aliases
contact=realtime,ps_contacts

[res_pjsip_endpoint_identifier_ip]
identify=realtime,ps_endpoint_id_ips
identify=config,pjsip.conf,criteria=type=identify

####Extconfig#####
[settings]
ps_endpoints => odbc,asterisk
ps_auths => odbc,asterisk
ps_aors => odbc,asterisk
ps_domain_aliases => odbc,asterisk
ps_endpoint_id_ips => odbc,asterisk
ps_contacts => odbc,asterisk
ps_registrations = odbc,asterisk ;Added 3-1-2021 for Asterisk - Asterisk Registrar  

#####PJSIP Conf#####
[trunk]
type=aor
contact=sip:10.0.0.35:5060

[trunk]
type=endpoint
context=incoming
disallow=all
allow=ulaw,vp9,vp8,h264
aors=trunk
identify_by=ip


[trunk]
type=identify
endpoint=trunk
match=10.0.0.35

#####Extensions #######
[outgoing]
exten => _62XX,1,Set(header=${PJSIP_HEADER(read,Session-ID)})
same => n,Dial(PJSIP/${EXTEN}@trunk,,b(handler^addheader^1(${header})))
same => n,Hangup()

[incoming]
exten => _61XX,1,Set(header=${PJSIP_HEADER(read,Session-ID)})
same => n,Dial(PJSIP/${EXTEN},,b(handler^addheader^1(${header})))
same => n,Hangup()


[endpoints]
include => outgoing
include => incoming

#####Endpoints (MySQL) #####
 Endpoint:  <Endpoint/CID.....................................>  <State.....>  <Channels.>
    I/OAuth:  <AuthId/UserName...........................................................>
        Aor:  <Aor............................................>  <MaxContact>
      Contact:  <Aor/ContactUri..........................> <Hash....> <Status> <RTT(ms)..>
  Transport:  <TransportId........>  <Type>  <cos>  <tos>  <BindAddress..................>
   Identify:  <Identify/Endpoint.........................................................>
        Match:  <criteria.........................>
    Channel:  <ChannelId......................................>  <State.....>  <Time.....>
        Exten: <DialedExten...........>  CLCID: <ConnectedLineCID.......>
==========================================================================================

 Endpoint:  6100                                                 Not in use    0 of inf
        Aor:  6100                                               5
      Contact:  6100/sip:6100@10.0.0.13:5060               7dd855e193 NonQual         nan
  Transport:  transport-wss             wss      0      0  0.0.0.0:5060


 ParameterName                      : ParameterValue
 ===================================================================================================
 100rel                             : yes
 accept_multiple_sdp_answers        : false
 accountcode                        : 
 acl                                : 
 aggregate_mwi                      : true
 allow                              : (ulaw|vp9|vp8|h264)
 allow_overlap                      : true
 allow_subscribe                    : true
 allow_transfer                     : true
 allow_unauthenticated_options      : false
 aors                               : 6100
 asymmetric_rtp_codec               : false
 auth                               : 
 bind_rtp_to_media_address          : false
 bundle                             : true
 call_group                         : 
 callerid                           : <unknown>
 callerid_privacy                   : allowed_not_screened
 callerid_tag                       : 
 codec_prefs_incoming_answer        : prefer:pending, operation:intersect, keep:all, transcode:allow
 codec_prefs_incoming_offer         : prefer:pending, operation:intersect, keep:all, transcode:allow
 codec_prefs_outgoing_answer        : prefer:pending, operation:intersect, keep:all, transcode:allow
 codec_prefs_outgoing_offer         : prefer:pending, operation:union, keep:all, transcode:allow
 connected_line_method              : invite
 contact_acl                        : 
 context                            : endpoints
 cos_audio                          : 0
 cos_video                          : 0
 device_state_busy_at               : 0
 direct_media                       : true
 direct_media_glare_mitigation      : none
 direct_media_method                : invite
 disable_direct_media_on_nat        : false
 dtls_auto_generate_cert            : Yes
 dtls_ca_file                       : 
 dtls_ca_path                       : 
 dtls_cert_file                     : 
 dtls_cipher                        : 
 dtls_fingerprint                   : SHA-256
 dtls_private_key                   : 
 dtls_rekey                         : 0
 dtls_setup                         : actpass
 dtls_verify                        : Yes
 dtmf_mode                          : rfc4733
 fax_detect                         : false
 fax_detect_timeout                 : 0
 follow_early_media_fork            : true
 force_avp                          : false
 force_rport                        : true
 from_domain                        : 
 from_user                          : 
 g726_non_standard                  : false
 ice_support                        : true
 identify_by                        : username,ip
 ignore_183_without_sdp             : false
 inband_progress                    : false
 incoming_call_offer_pref           : local
 incoming_mwi_mailbox               : 
 language                           : 
 mailboxes                          : 
 max_audio_streams                  : 1
 max_video_streams                  : 1
 media_address                      : 
 media_encryption                   : dtls
 media_encryption_optimistic        : false
 media_use_received_transport       : true
 message_context                    : 
 moh_passthrough                    : false
 moh_suggest                        : default
 mwi_from_user                      : 
 mwi_subscribe_replaces_unsolicited : no
 named_call_group                   : 
 named_pickup_group                 : 
 notify_early_inuse_ringing         : false
 one_touch_recording                : false
 outbound_auth                      : 
 outbound_proxy                     : 
 outgoing_call_offer_pref           : remote_merge
 pickup_group                       : 
 preferred_codec_only               : false
 record_off_feature                 : automixmon
 record_on_feature                  : automixmon
 refer_blind_progress               : true
 rewrite_contact                    : false
 rpid_immediate                     : false
 rtcp_mux                           : true
 rtp_engine                         : asterisk
 rtp_ipv6                           : false
 rtp_keepalive                      : 0
 rtp_symmetric                      : false
 rtp_timeout                        : 0
 rtp_timeout_hold                   : 0
 sdp_owner                          : -
 sdp_session                        : Asterisk
 send_connected_line                : yes
 send_diversion                     : true
 send_history_info                  : false
 send_pai                           : false
 send_rpid                          : false
 set_var                            : 
 srtp_tag_32                        : false
 stir_shaken                        : false
 sub_min_expiry                     : 0
 subscribe_context                  : subscribe
 suppress_q850_reason_headers       : false
 t38_udptl                          : false
 t38_udptl_ec                       : none
 t38_udptl_ipv6                     : false
 t38_udptl_maxdatagram              : 0
 t38_udptl_nat                      : false
 timers                             : yes
 timers_min_se                      : 90
 timers_sess_expires                : 1800
 tone_zone                          : 
 tos_audio                          : 0
 tos_video                          : 0
 transport                          : transport-wss
 trust_connected_line               : yes
 trust_id_inbound                   : false
 trust_id_outbound                  : false
 use_avpf                           : true
 use_ptime                          : false
 user_eq_phone                      : false
 voicemail_extension                : 
 webrtc                             : yes


      Aor:  <Aor..............................................>  <MaxContact>
    Contact:  <Aor/ContactUri............................> <Hash....> <Status> <RTT(ms)..>
==========================================================================================

      Aor:  6100                                                 5
    Contact:  6100/sip:6100@10.0.0.13:5060                 7dd855e193 NonQual         nan


 ParameterName        : ParameterValue
 ==============================================
 authenticate_qualify : false
 contact              : sip:6100@10.0.0.13:5060
 default_expiration   : 3600
 mailboxes            : 
 max_contacts         : 5
 maximum_expiration   : 7200
 minimum_expiration   : 60
 outbound_proxy       : 
 qualify_frequency    : 0
 qualify_timeout      : 3.000000
 remove_existing      : true
 support_path         : false
 voicemail_extension  : 



System B Configs (10.0.0.35)


#####Sorcery#####

[res_pjsip] ; Realtime PJSIP configuration wizard
endpoint=realtime,ps_endpoints
endpoint=config,pjsip.conf,criteria=type=endpoint
auth=realtime,ps_auths
auth=config,pjsip.conf,criteria=type=auth
aor=realtime,ps_aors
aor=config,pjsip.conf,criteria=type=aor
domain_alias=realtime,ps_domain_aliases
contact=realtime,ps_contacts

[res_pjsip_endpoint_identifier_ip]
identify=realtime,ps_endpoint_id_ips
identify=config,pjsip.conf,criteria=type=identify

####Extconfig#####

ps_endpoints => odbc,asterisk
ps_auths => odbc,asterisk
ps_aors => odbc,asterisk
ps_domain_aliases => odbc,asterisk
ps_endpoint_id_ips => odbc,asterisk
ps_contacts => odbc,asterisk

#####PJSIP Conf#####

[trunk]
type=aor
contact=sip:10.0.0.13:5060

[trunk]
type=endpoint
context=incoming
disallow=all
allow=ulaw,vp9,vp8,h264
aors=trunk
identify_by=ip

[trunk]
type=identify
endpoint=trunk
match=10.0.0.13


#####Extensions #######
[outgoing]
exten => _61XX,1,Set(header=${PJSIP_HEADER(read,Session-ID)})
same => n,Dial(PJSIP/${EXTEN}@trunk,,b(handler^addheader^1(${header})))
same => n,Hangup()

[incoming]
exten => _62XX,1,Set(header=${PJSIP_HEADER(read,Session-ID)})
same => n,Dial(PJSIP/${EXTEN},,b(handler^addheader^1(${header})))
same => n,Hangup()


[endpoints]
include => outgoing
include => incoming

#####Endpoints (MySQL) #####

 Endpoint:  <Endpoint/CID.....................................>  <State.....>  <Channels.>
    I/OAuth:  <AuthId/UserName...........................................................>
        Aor:  <Aor............................................>  <MaxContact>
      Contact:  <Aor/ContactUri..........................> <Hash....> <Status> <RTT(ms)..>
  Transport:  <TransportId........>  <Type>  <cos>  <tos>  <BindAddress..................>
   Identify:  <Identify/Endpoint.........................................................>
        Match:  <criteria.........................>
    Channel:  <ChannelId......................................>  <State.....>  <Time.....>
        Exten: <DialedExten...........>  CLCID: <ConnectedLineCID.......>
==========================================================================================

 Endpoint:  6200                                                 Not in use    0 of inf
        Aor:  6200                                               5
      Contact:  6200/sip:6200@10.0.0.35:5060               ca52a79a69 NonQual         nan
  Transport:  transport-wss             wss      0      0  0.0.0.0:5060


 ParameterName                      : ParameterValue
 ===================================================================================================
 100rel                             : yes
 accept_multiple_sdp_answers        : false
 accountcode                        : 
 acl                                : 
 aggregate_mwi                      : true
 allow                              : (ulaw|vp9|vp8|h264)
 allow_overlap                      : true
 allow_subscribe                    : true
 allow_transfer                     : true
 allow_unauthenticated_options      : false
 aors                               : 6200
 asymmetric_rtp_codec               : false
 auth                               : 
 bind_rtp_to_media_address          : false
 bundle                             : true
 call_group                         : 
 callerid                           : <unknown>
 callerid_privacy                   : allowed_not_screened
 callerid_tag                       : 
 codec_prefs_incoming_answer        : prefer:pending, operation:intersect, keep:all, transcode:allow
 codec_prefs_incoming_offer         : prefer:pending, operation:intersect, keep:all, transcode:allow
 codec_prefs_outgoing_answer        : prefer:pending, operation:intersect, keep:all, transcode:allow
 codec_prefs_outgoing_offer         : prefer:pending, operation:union, keep:all, transcode:allow
 connected_line_method              : invite
 contact_acl                        : 
 context                            : endpoints
 cos_audio                          : 0
 cos_video                          : 0
 device_state_busy_at               : 0
 direct_media                       : true
 direct_media_glare_mitigation      : none
 direct_media_method                : invite
 disable_direct_media_on_nat        : false
 dtls_auto_generate_cert            : Yes
 dtls_ca_file                       : 
 dtls_ca_path                       : 
 dtls_cert_file                     : 
 dtls_cipher                        : 
 dtls_fingerprint                   : SHA-256
 dtls_private_key                   : 
 dtls_rekey                         : 0
 dtls_setup                         : actpass
 dtls_verify                        : Yes
 dtmf_mode                          : rfc4733
 fax_detect                         : false
 fax_detect_timeout                 : 0
 follow_early_media_fork            : true
 force_avp                          : false
 force_rport                        : true
 from_domain                        : 
 from_user                          : 
 g726_non_standard                  : false
 ice_support                        : true
 identify_by                        : username,ip
 ignore_183_without_sdp             : false
 inband_progress                    : false
 incoming_call_offer_pref           : local
 incoming_mwi_mailbox               : 
 language                           : 
 mailboxes                          : 
 max_audio_streams                  : 1
 max_video_streams                  : 1
 media_address                      : 
 media_encryption                   : dtls
 media_encryption_optimistic        : false
 media_use_received_transport       : true
 message_context                    : 
 moh_passthrough                    : false
 moh_suggest                        : default
 mwi_from_user                      : 
 mwi_subscribe_replaces_unsolicited : no
 named_call_group                   : 
 named_pickup_group                 : 
 notify_early_inuse_ringing         : false
 one_touch_recording                : false
 outbound_auth                      : 
 outbound_proxy                     : 
 outgoing_call_offer_pref           : remote_merge
 pickup_group                       : 
 preferred_codec_only               : false
 record_off_feature                 : automixmon
 record_on_feature                  : automixmon
 refer_blind_progress               : true
 rewrite_contact                    : false
 rpid_immediate                     : false
 rtcp_mux                           : true
 rtp_engine                         : asterisk
 rtp_ipv6                           : false
 rtp_keepalive                      : 0
 rtp_symmetric                      : false
 rtp_timeout                        : 0
 rtp_timeout_hold                   : 0
 sdp_owner                          : -
 sdp_session                        : Asterisk
 send_connected_line                : yes
 send_diversion                     : true
 send_history_info                  : false
 send_pai                           : false
 send_rpid                          : false
 set_var                            : 
 srtp_tag_32                        : false
 stir_shaken                        : false
 sub_min_expiry                     : 0
 subscribe_context                  : subscribe
 suppress_q850_reason_headers       : false
 t38_udptl                          : false
 t38_udptl_ec                       : none
 t38_udptl_ipv6                     : false
 t38_udptl_maxdatagram              : 0
 t38_udptl_nat                      : false
 timers                             : yes
 timers_min_se                      : 90
 timers_sess_expires                : 1800
 tone_zone                          : 
 tos_audio                          : 0
 tos_video                          : 0
 transport                          : transport-wss
 trust_connected_line               : yes
 trust_id_inbound                   : false
 trust_id_outbound                  : false
 use_avpf                           : true
 use_ptime                          : false
 user_eq_phone                      : false
 voicemail_extension                : 
 webrtc                             : yes



      Aor:  <Aor..............................................>  <MaxContact>
    Contact:  <Aor/ContactUri............................> <Hash....> <Status> <RTT(ms)..>
==========================================================================================

      Aor:  6200                                                 5
    Contact:  6200/sip:6200@10.0.0.35:5060                 ca52a79a69 NonQual         nan


 ParameterName        : ParameterValue
 ==============================================
 authenticate_qualify : false
 contact              : sip:6200@10.0.0.35:5060
 default_expiration   : 3600
 mailboxes            : 
 max_contacts         : 5
 maximum_expiration   : 7200
 minimum_expiration   : 60
 outbound_proxy       : 
 qualify_frequency    : 0
 qualify_timeout      : 3.000000
 remove_existing      : true
 support_path         : false
 voicemail_extension  : 

Call Trace SYSTEM A

<--- Received SIP request (4068 bytes) from UDP:10.0.0.13:5060 --->
INVITE sip:6200@10.0.0.13 SIP/2.0
Record-Route: <sip:10.0.0.13;r2=on;lr=on;ftag=4i37qoi8n8;did=67a.0642;nat=yes>
Record-Route: <sip:10.0.0.13:8060;transport=ws;r2=on;lr=on;ftag=4i37qoi8n8;did=67a.0642;nat=yes>
Via: SIP/2.0/UDP 10.0.0.13;branch=z9hG4bK3b4a.26045bc8c6d19aab2606efed09d1b83f.0
Via: SIP/2.0/WSS 192.0.2.1;rport=39230;received=10.0.0.42;branch=z9hG4bK9653858
Max-Forwards: 69
To: <sip:6200@10.0.0.13>
From: "6100" <sip:6100@10.0.0.13>;tag=4i37qoi8n8
Call-ID: eumj8i4kffdha10khvou
CSeq: 3100 INVITE
WG67-Version: phone.02
Subject: da/ida call
Priority: normal
WG67-CallType: phone.02;da/ida call
Contact: <sip:o118b42l@192.0.2.1;transport=wss;ob;alias=10.0.0.42~39230~6;alias=10.0.0.42~39230~6>
Allow: ACK,CANCEL,INVITE,MESSAGE,BYE,OPTIONS,INFO,NOTIFY,REFER
Supported: outbound
User-Agent: Raspberry Phone (SipJS - 0.11.6)
Content-Type: application/sdp
Content-Length: 3126
Session-ID: eumj8i4kffdha10khvou 

v=0
o=- 6687993059057468517 2 IN IP4 127.0.0.1
s=-
t=0 0
a=group:BUNDLE 0
a=extmap-allow-mixed
a=msid-semantic: WMS u1Em3BkI5EyIHeVmV4k7Zu3d7p9ji1ds43G5
m=audio 57101 UDP/TLS/RTP/SAVPF 111 63 103 104 9 0 8 106 105 13 110 112 113 126
c=IN IP4 78.189.148.208
a=rtcp:58771 IN IP4 78.189.148.208
a=candidate:3795201729 1 udp 2122260223 10.0.0.42 57101 typ host generation 0 network-id 1
a=candidate:4035150127 1 udp 2122194687 10.0.0.41 42102 typ host generation 0 network-id 2 network-cost 10
a=candidate:3795201729 2 udp 2122260222 10.0.0.42 55515 typ host generation 0 network-id 1
a=candidate:4035150127 2 udp 2122194686 10.0.0.41 58771 typ host generation 0 network-id 2 network-cost 10
a=candidate:2696663113 1 udp 1686052607 78.189.148.208 57101 typ srflx raddr 10.0.0.42 rport 57101 generation 0 network-id 1
a=candidate:2987294631 1 udp 1685987071 78.189.148.208 42102 typ srflx raddr 10.0.0.41 rport 42102 generation 0 network-id 2 network-cost 10
a=candidate:2987294631 2 udp 1685987070 78.189.148.208 58771 typ srflx raddr 10.0.0.41 rport 58771 generation 0 network-id 2 network-cost 10
a=candidate:2696663113 2 udp 1686052606 78.189.148.208 55515 typ srflx raddr 10.0.0.42 rport 55515 generation 0 network-id 1
a=candidate:2897596977 1 tcp 1518280447 10.0.0.42 9 typ host tcptype active generation 0 network-id 1
a=candidate:3187703263 1 tcp 1518214911 10.0.0.41 9 typ host tcptype active generation 0 network-id 2 network-cost 10
a=candidate:2897596977 2 tcp 1518280446 10.0.0.42 9 typ host tcptype active generation 0 network-id 1
a=candidate:3187703263 2 tcp 1518214910 10.0.0.41 9 typ host tcptype active generation 0 network-id 2 network-cost 10
a=ice-ufrag:5mIT
a=ice-pwd:CKxqHXHYjL9T+e+qvT5d/wrN
a=ice-options:trickle
a=fingerprint:sha-256 85:BA:52:9E:19:FB:99:72:C8:70:11:E4:D1:35:91:A0:29:8E:B6:E9:62:D1:FA:DE:0C:C2:EE:D2:C3:5D:FD:B3
a=setup:actpass
a=mid:0
a=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level
a=extmap:2 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time
a=extmap:3 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01
a=extmap:4 urn:ietf:params:rtp-hdrext:sdes:mid
a=extmap:5 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id
a=extmap:6 urn:ietf:params:rtp-hdrext:sdes:repaired-rtp-stream-id
a=sendrecv
a=msid:u1Em3BkI5EyIHeVmV4k7Zu3d7p9ji1ds43G5 053cef2e-3fac-438a-996a-a849e3b2450f
a=rtcp-mux
a=rtpmap:111 opus/48000/2
a=rtcp-fb:111 transport-cc
a=fmtp:111 minptime=10;useinbandfec=1
a=rtpmap:63 red/48000/2
a=fmtp:63 111/111
a=rtpmap:103 ISAC/16000
a=rtpmap:104 ISAC/32000
a=rtpmap:9 G722/8000
a=rtpmap:0 PCMU/8000
a=rtpmap:8 PCMA/8000
a=rtpmap:106 CN/32000
a=rtpmap:105 CN/16000
a=rtpmap:13 CN/8000
a=rtpmap:110 telephone-event/48000
a=rtpmap:112 telephone-event/32000
a=rtpmap:113 telephone-event/16000
a=rtpmap:126 telephone-event/8000
a=ssrc:2603391953 cname:NYMgdfG1cW0+7PTA
a=ssrc:2603391953 msid:u1Em3BkI5EyIHeVmV4k7Zu3d7p9ji1ds43G5 053cef2e-3fac-438a-996a-a849e3b2450f
a=ssrc:2603391953 mslabel:u1Em3BkI5EyIHeVmV4k7Zu3d7p9ji1ds43G5
a=ssrc:2603391953 label:053cef2e-3fac-438a-996a-a849e3b2450f

<--- Transmitting SIP response (562 bytes) to UDP:10.0.0.13:5060 --->
SIP/2.0 100 Trying
Via: SIP/2.0/UDP 10.0.0.13;rport=5060;received=10.0.0.13;branch=z9hG4bK3b4a.26045bc8c6d19aab2606efed09d1b83f.0
Via: SIP/2.0/WSS 192.0.2.1;rport=39230;received=10.0.0.42;branch=z9hG4bK9653858
Record-Route: <sip:10.0.0.13;lr;r2=on;ftag=4i37qoi8n8;did=67a.0642;nat=yes>
Record-Route: <sip:10.0.0.13:8060;transport=ws;lr;r2=on;ftag=4i37qoi8n8;did=67a.0642;nat=yes>
Call-ID: eumj8i4kffdha10khvou
From: "6100" <sip:6100@10.0.0.13>;tag=4i37qoi8n8
To: <sip:6200@10.0.0.13>
CSeq: 3100 INVITE
Server: Asterisk PBX 18.4.0
Content-Length:  0


    -- Executing [6200@endpoints:1] Set("PJSIP/6100-0000004e", "header=eumj8i4kffdha10khvou") in new stack
    -- Executing [6200@endpoints:2] Dial("PJSIP/6100-0000004e", "PJSIP/6200@trunk,,b(handler^addheader^1(eumj8i4kffdha10khvou))") in new stack
    -- PJSIP/trunk-0000004f Internal Gosub(handler,addheader,1(eumj8i4kffdha10khvou)) start
    -- Executing [addheader@handler:1] Set("PJSIP/trunk-0000004f", "PJSIP_HEADER(add,Session-ID)=eumj8i4kffdha10khvou") in new stack
    -- Executing [addheader@handler:2] Return("PJSIP/trunk-0000004f", "") in new stack
  == Spawn extension (incoming, 6200, 1) exited non-zero on 'PJSIP/trunk-0000004f'
    -- PJSIP/trunk-0000004f Internal Gosub(handler,addheader,1(eumj8i4kffdha10khvou)) complete GOSUB_RETVAL=
    -- Called PJSIP/6200@trunk
<--- Transmitting SIP request (915 bytes) to UDP:10.0.0.35:5060 --->
INVITE sip:6200@10.0.0.35:5060 SIP/2.0
Via: SIP/2.0/UDP 10.0.0.13:5080;rport;branch=z9hG4bKPjdca3c357-e07c-41ba-b24f-b1a4b417a129
From: "6100" <sip:6100@10.0.0.13>;tag=36f3347e-1764-4eb7-b086-9b99ac96a582
To: <sip:6200@10.0.0.35>
Contact: <sip:asterisk@10.0.0.13:5080>
Call-ID: 745dd941-8817-4f4e-aa1a-ac71190505df
CSeq: 21227 INVITE
Allow: OPTIONS, REGISTER, SUBSCRIBE, NOTIFY, PUBLISH, INVITE, ACK, BYE, CANCEL, UPDATE, PRACK, MESSAGE, REFER
Supported: 100rel, timer, replaces, norefersub, histinfo
Session-Expires: 1800
Min-SE: 90
Session-ID: eumj8i4kffdha10khvou
Max-Forwards: 70
User-Agent: Asterisk PBX 18.4.0
Content-Type: application/sdp
Content-Length:   229

v=0
o=- 650042477 650042477 IN IP4 10.0.0.13
s=Asterisk
c=IN IP4 10.0.0.13
t=0 0
m=audio 10166 RTP/AVP 0 101
a=rtpmap:0 PCMU/8000
a=rtpmap:101 telephone-event/8000
a=fmtp:101 0-16
a=ptime:20
a=maxptime:150
a=sendrecv

<--- Received SIP response (399 bytes) from UDP:10.0.0.35:5060 --->
SIP/2.0 100 trying -- your call is important to us
Via: SIP/2.0/UDP 10.0.0.13:5080;rport=5080;branch=z9hG4bKPjdca3c357-e07c-41ba-b24f-b1a4b417a129;received=10.0.0.13
From: "6100" <sip:6100@10.0.0.13>;tag=36f3347e-1764-4eb7-b086-9b99ac96a582
To: <sip:6200@10.0.0.35>
Call-ID: 745dd941-8817-4f4e-aa1a-ac71190505df
CSeq: 21227 INVITE
Server: kamailio (5.5.0 (x86_64/linux))
Content-Length: 0


<--- Received SIP response (738 bytes) from UDP:10.0.0.35:5060 --->
SIP/2.0 401 Unauthorized
Via: SIP/2.0/UDP 10.0.0.13:5080;rport=5080;received=10.0.0.13;branch=z9hG4bKPjdca3c357-e07c-41ba-b24f-b1a4b417a129
Record-Route: <sip:10.0.0.35;lr;ftag=36f3347e-1764-4eb7-b086-9b99ac96a582;did=f6.37f;nat=yes>
Call-ID: 745dd941-8817-4f4e-aa1a-ac71190505df
From: "6100" <sip:6100@10.0.0.13>;tag=36f3347e-1764-4eb7-b086-9b99ac96a582
To: <sip:6200@10.0.0.35>;tag=z9hG4bK9185.af341f88de92414533a59c68ddb4b0b4.0
CSeq: 21227 INVITE
WWW-Authenticate: Digest realm="asterisk",nonce="1642075252/5e3bac7369c451e68a073285ad44931f",opaque="68271d8d089a5e8d",algorithm=md5,qop="auth"
Server: Asterisk PBX 18.4.0
Content-Length:  0
WG67-Version:0 
Subject:0 
Priority:0 
WG67-CallType:0 
P-Asserted-Identity:0 


<--- Transmitting SIP request (419 bytes) to UDP:10.0.0.35:5060 --->
ACK sip:6200@10.0.0.35:5060 SIP/2.0
Via: SIP/2.0/UDP 10.0.0.13:5080;rport;branch=z9hG4bKPjdca3c357-e07c-41ba-b24f-b1a4b417a129
From: "6100" <sip:6100@10.0.0.13>;tag=36f3347e-1764-4eb7-b086-9b99ac96a582
To: <sip:6200@10.0.0.35>;tag=z9hG4bK9185.af341f88de92414533a59c68ddb4b0b4.0
Call-ID: 745dd941-8817-4f4e-aa1a-ac71190505df
CSeq: 21227 ACK
Max-Forwards: 70
User-Agent: Asterisk PBX 18.4.0
Content-Length:  0


[Jan 13 12:00:52] WARNING[6332]: res_pjsip_outbound_authenticator_digest.c:181 digest_create_request_with_auth: Endpoint: 'trunk': Unable to create request with auth. No auth credentials for realm(s) 'asterisk' in challenge.
  == Everyone is busy/congested at this time (1:0/0/1)
    -- Executing [6200@endpoints:3] Hangup("PJSIP/6100-0000004e", "") in new stack
  == Spawn extension (endpoints, 6200, 3) exited non-zero on 'PJSIP/6100-0000004e'
<--- Transmitting SIP response (630 bytes) to UDP:10.0.0.13:5060 --->
SIP/2.0 403 Forbidden
Via: SIP/2.0/UDP 10.0.0.13;rport=5060;received=10.0.0.13;branch=z9hG4bK3b4a.26045bc8c6d19aab2606efed09d1b83f.0
Via: SIP/2.0/WSS 192.0.2.1;rport=39230;received=10.0.0.42;branch=z9hG4bK9653858
Record-Route: <sip:10.0.0.13;lr;r2=on;ftag=4i37qoi8n8;did=67a.0642;nat=yes>
Record-Route: <sip:10.0.0.13:8060;transport=ws;lr;r2=on;ftag=4i37qoi8n8;did=67a.0642;nat=yes>
Call-ID: eumj8i4kffdha10khvou
From: "6100" <sip:6100@10.0.0.13>;tag=4i37qoi8n8
To: <sip:6200@10.0.0.13>;tag=6cadb8a9-70d3-4981-b3ee-0a1c6862d4ab
CSeq: 3100 INVITE
Server: Asterisk PBX 18.4.0
Reason: Q.850;cause=21
Content-Length:  0


<--- Received SIP request (317 bytes) from UDP:10.0.0.13:5060 --->
ACK sip:6200@10.0.0.13 SIP/2.0
Via: SIP/2.0/UDP 10.0.0.13;branch=z9hG4bK3b4a.26045bc8c6d19aab2606efed09d1b83f.0
Max-Forwards: 69
To: <sip:6200@10.0.0.13>;tag=6cadb8a9-70d3-4981-b3ee-0a1c6862d4ab
From: "6100" <sip:6100@10.0.0.13>;tag=4i37qoi8n8
Call-ID: eumj8i4kffdha10khvou
CSeq: 3100 ACK
Content-Length: 0


Call Trace SYSTEM B

<--- Received SIP request (1193 bytes) from UDP:10.0.0.35:5060 --->
INVITE sip:6200@10.0.0.35:5060 SIP/2.0
Record-Route: <sip:10.0.0.35;lr=on;ftag=36f3347e-1764-4eb7-b086-9b99ac96a582;did=f6.37f;nat=yes>
Via: SIP/2.0/UDP 10.0.0.35;branch=z9hG4bK9185.af341f88de92414533a59c68ddb4b0b4.0
Via: SIP/2.0/UDP 10.0.0.13:5080;received=10.0.0.13;rport=5080;branch=z9hG4bKPjdca3c357-e07c-41ba-b24f-b1a4b417a129
From: "6100" <sip:6100@10.0.0.13>;tag=36f3347e-1764-4eb7-b086-9b99ac96a582
To: <sip:6200@10.0.0.35>
Contact: <sip:asterisk@10.0.0.13:5080;alias=10.0.0.13~5080~1>
Call-ID: 745dd941-8817-4f4e-aa1a-ac71190505df
CSeq: 21227 INVITE
Allow: OPTIONS, REGISTER, SUBSCRIBE, NOTIFY, PUBLISH, INVITE, ACK, BYE, CANCEL, UPDATE, PRACK, MESSAGE, REFER
Supported: 100rel, timer, replaces, norefersub, histinfo
Session-Expires: 1800
Min-SE: 90
Session-ID: eumj8i4kffdha10khvou
Max-Forwards: 69
User-Agent: Asterisk PBX 18.4.0
Content-Type: application/sdp
Content-Length:   229
Session-ID: 745dd941-8817-4f4e-aa1a-ac71190505df 

v=0
o=- 650042477 650042477 IN IP4 10.0.0.13
s=Asterisk
c=IN IP4 10.0.0.13
t=0 0
m=audio 10166 RTP/AVP 0 101
a=rtpmap:0 PCMU/8000
a=rtpmap:101 telephone-event/8000
a=fmtp:101 0-16
a=ptime:20
a=maxptime:150
a=sendrecv

[Jan 13 12:00:52] NOTICE[6134]: res_pjsip/pjsip_distributor.c:676 log_failed_request: Request 'INVITE' from '"6100" <sip:6100@10.0.0.13>' failed for '10.0.0.35:5060' (callid: 745dd941-8817-4f4e-aa1a-ac71190505df) - No matching endpoint found
<--- Transmitting SIP response (766 bytes) to UDP:10.0.0.35:5060 --->
SIP/2.0 401 Unauthorized
Via: SIP/2.0/UDP 10.0.0.35;rport=5060;received=10.0.0.35;branch=z9hG4bK9185.af341f88de92414533a59c68ddb4b0b4.0
Via: SIP/2.0/UDP 10.0.0.13:5080;rport=5080;received=10.0.0.13;branch=z9hG4bKPjdca3c357-e07c-41ba-b24f-b1a4b417a129
Record-Route: <sip:10.0.0.35;lr;ftag=36f3347e-1764-4eb7-b086-9b99ac96a582;did=f6.37f;nat=yes>
Call-ID: 745dd941-8817-4f4e-aa1a-ac71190505df
From: "6100" <sip:6100@10.0.0.13>;tag=36f3347e-1764-4eb7-b086-9b99ac96a582
To: <sip:6200@10.0.0.35>;tag=z9hG4bK9185.af341f88de92414533a59c68ddb4b0b4.0
CSeq: 21227 INVITE
WWW-Authenticate: Digest realm="asterisk",nonce="1642075252/5e3bac7369c451e68a073285ad44931f",opaque="68271d8d089a5e8d",algorithm=md5,qop="auth"
Server: Asterisk PBX 18.4.0
Content-Length:  0


<--- Received SIP request (375 bytes) from UDP:10.0.0.35:5060 --->
ACK sip:6200@10.0.0.35:5060 SIP/2.0
Via: SIP/2.0/UDP 10.0.0.35;branch=z9hG4bK9185.af341f88de92414533a59c68ddb4b0b4.0
From: "6100" <sip:6100@10.0.0.13>;tag=36f3347e-1764-4eb7-b086-9b99ac96a582
To: <sip:6200@10.0.0.35>;tag=z9hG4bK9185.af341f88de92414533a59c68ddb4b0b4.0
Call-ID: 745dd941-8817-4f4e-aa1a-ac71190505df
CSeq: 21227 ACK
Max-Forwards: 69
Content-Length: 0

Thank you all in advance

Is the res_pjsip_endpoint_identifier_ip module loaded in the first place?

1 Like

Hello @jcolp , thanks again for the swift response.

When I try to load the module manually I get below warning in both systems:

WARNING[6259]: loader.c:1771 load_resource: Module ‘res_pjsip_endpoint_identifier_ip’ already loaded and running.

And does your identify show in “pjsip show identifies”?

SYSTEM A Identify

 Identify:  <Identify/Endpoint...........................................................>
      Match:  <criteria...........................>
==========================================================================================

 Identify:  trunk/trunk
      Match: 10.0.0.35/32


 ParameterName : ParameterValue
 =========================================
 endpoint      : trunk
 match         : 10.0.0.35/255.255.255.255
 match_header  : 
 srv_lookups   : true

SYSTEM B Identify

 Identify:  <Identify/Endpoint...........................................................>
      Match:  <criteria...........................>
==========================================================================================

 Identify:  trunk/trunk
      Match: 10.0.0.13/32


 ParameterName : ParameterValue
 =========================================
 endpoint      : trunk
 match         : 10.0.0.13/255.255.255.255
 match_header  : 
 srv_lookups   : true

It looks like you have them reversed. The traffic shown in your debug on “SYSTEM B” has a source of “10.0.0.35” so the identify would have to match that. It’s not currently. Oh, your proxy usage would also be changing things such that if it went through the proxy on 10.0.0.35 to Asterisk on the same system, then it would appear as coming from 10.0.0.35 (itself).

This is great thanks, @jcolp! I think that is the issue however I can’t figure what the solution would be,

Please bear with me:
Below is the initial invite that my System B received:
As you mentioned, it clearly says “Received SIP request (1193 bytes) from UDP:10.0.0.35:5060”
As that is indeed my Kamailio routing the invite to my Asterisk.

<--- Received SIP request (1193 bytes) from UDP:10.0.0.35:5060 --->
INVITE sip:6200@10.0.0.35:5060 SIP/2.0
Record-Route: <sip:10.0.0.35;lr=on;ftag=36f3347e-1764-4eb7-b086-9b99ac96a582;did=f6.37f;nat=yes>
Via: SIP/2.0/UDP 10.0.0.35;branch=z9hG4bK9185.af341f88de92414533a59c68ddb4b0b4.0
Via: SIP/2.0/UDP 10.0.0.13:5080;received=10.0.0.13;rport=5080;branch=z9hG4bKPjdca3c357-e07c-41ba-b24f-b1a4b417a129
From: "6100" <sip:6100@10.0.0.13>;tag=36f3347e-1764-4eb7-b086-9b99ac96a582
To: <sip:6200@10.0.0.35>
Contact: <sip:asterisk@10.0.0.13:5080;alias=10.0.0.13~5080~1>

However if I change the identifier to match my Kamailio ip then I forward the invite to myself

[trunk]
type=identify
endpoint=trunk
match=10.0.0.13:5060

<--- Received SIP request (4070 bytes) from UDP:10.0.0.13:5060 --->
INVITE sip:6200@10.0.0.13 SIP/2.0
Record-Route: <sip:10.0.0.13;r2=on;lr=on;ftag=p8suaav6d6;did=12b.1961;nat=yes>
Record-Route: <sip:10.0.0.13:8060;transport=ws;r2=on;lr=on;ftag=p8suaav6d6;did=12b.1961;nat=yes>
Via: SIP/2.0/UDP 10.0.0.13;branch=z9hG4bKbdd9.737084857fa45f97c65629a5ee8d4065.0
Via: SIP/2.0/WSS 192.0.2.31;rport=39694;received=10.0.0.42;branch=z9hG4bK8977891
Max-Forwards: 69
To: <sip:6200@10.0.0.13>
From: "6100" <sip:6100@10.0.0.13>;tag=p8suaav6d6
Call-ID: 7a5rqiinf115slneghuj
CSeq: 5141 INVITE
WG67-Version: phone.02
Subject: da/ida call
Priority: normal
WG67-CallType: phone.02;da/ida call
Contact: <sip:ff4a097u@192.0.2.31;transport=wss;ob;alias=10.0.0.42~39694~6;alias=10.0.0.42~39694~6>
Allow: ACK,CANCEL,INVITE,MESSAGE,BYE,OPTIONS,INFO,NOTIFY,REFER
Supported: outbound
User-Agent: Raspberry Phone (SipJS - 0.11.6)
Content-Type: application/sdp
Content-Length: 3126
Session-ID: 7a5rqiinf115slneghuj 

v=0
o=- 6022758634066066003 2 IN IP4 127.0.0.1
s=-
t=0 0
a=group:BUNDLE 0
a=extmap-allow-mixed
a=msid-semantic: WMS Aj2xL2SagxDAOhgmIgzB2soPmZeGRhXExvkN
m=audio 58371 UDP/TLS/RTP/SAVPF 111 63 103 104 9 0 8 106 105 13 110 112 113 126
c=IN IP4 78.189.148.208
a=rtcp:39265 IN IP4 78.189.148.208
a=candidate:3795201729 1 udp 2122260223 10.0.0.42 58371 typ host generation 0 network-id 1
a=candidate:4035150127 1 udp 2122194687 10.0.0.41 45853 typ host generation 0 network-id 2 network-cost 10
a=candidate:3795201729 2 udp 2122260222 10.0.0.42 39265 typ host generation 0 network-id 1
a=candidate:4035150127 2 udp 2122194686 10.0.0.41 40864 typ host generation 0 network-id 2 network-cost 10
a=candidate:2696663113 1 udp 1686052607 78.189.148.208 58371 typ srflx raddr 10.0.0.42 rport 58371 generation 0 network-id 1
a=candidate:2696663113 2 udp 1686052606 78.189.148.208 39265 typ srflx raddr 10.0.0.42 rport 39265 generation 0 network-id 1
a=candidate:2987294631 2 udp 1685987070 78.189.148.208 40864 typ srflx raddr 10.0.0.41 rport 40864 generation 0 network-id 2 network-cost 10
a=candidate:2987294631 1 udp 1685987071 78.189.148.208 45853 typ srflx raddr 10.0.0.41 rport 45853 generation 0 network-id 2 network-cost 10
a=candidate:2897596977 1 tcp 1518280447 10.0.0.42 9 typ host tcptype active generation 0 network-id 1
a=candidate:3187703263 1 tcp 1518214911 10.0.0.41 9 typ host tcptype active generation 0 network-id 2 network-cost 10
a=candidate:2897596977 2 tcp 1518280446 10.0.0.42 9 typ host tcptype active generation 0 network-id 1
a=candidate:3187703263 2 tcp 1518214910 10.0.0.41 9 typ host tcptype active generation 0 network-id 2 network-cost 10
a=ice-ufrag:gydf
a=ice-pwd:IoxUzODpyb+FYFrQwf/M5jht
a=ice-options:trickle
a=fingerprint:sha-256 9B:B5:25:A6:D3:ED:EA:97:64:E5:12:3D:7F:2C:C1:73:B0:2E:66:C0:53:C3:9A:2E:F5:03:65:F4:23:69:4B:AF
a=setup:actpass
a=mid:0
a=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level
a=extmap:2 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time
a=extmap:3 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01
a=extmap:4 urn:ietf:params:rtp-hdrext:sdes:mid
a=extmap:5 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id
a=extmap:6 urn:ietf:params:rtp-hdrext:sdes:repaired-rtp-stream-id
a=sendrecv
a=msid:Aj2xL2SagxDAOhgmIgzB2soPmZeGRhXExvkN 743678e6-d78e-4dc8-bae5-999d01ec80e5
a=rtcp-mux
a=rtpmap:111 opus/48000/2
a=rtcp-fb:111 transport-cc
a=fmtp:111 minptime=10;useinbandfec=1
a=rtpmap:63 red/48000/2
a=fmtp:63 111/111
a=rtpmap:103 ISAC/16000
a=rtpmap:104 ISAC/32000
a=rtpmap:9 G722/8000
a=rtpmap:0 PCMU/8000
a=rtpmap:8 PCMA/8000
a=rtpmap:106 CN/32000
a=rtpmap:105 CN/16000
a=rtpmap:13 CN/8000
a=rtpmap:110 telephone-event/48000
a=rtpmap:112 telephone-event/32000
a=rtpmap:113 telephone-event/16000
a=rtpmap:126 telephone-event/8000
a=ssrc:1948709300 cname:WduLuED85oM2AHh0
a=ssrc:1948709300 msid:Aj2xL2SagxDAOhgmIgzB2soPmZeGRhXExvkN 743678e6-d78e-4dc8-bae5-999d01ec80e5
a=ssrc:1948709300 mslabel:Aj2xL2SagxDAOhgmIgzB2soPmZeGRhXExvkN
a=ssrc:1948709300 label:743678e6-d78e-4dc8-bae5-999d01ec80e5

[Jan 13 12:45:36] NOTICE[7130]: res_pjsip_session.c:3964 new_invite:  trunk: Call (UDP:10.0.0.13:5060) to extension '6200' rejected because extension not found in context 'incoming'.

I don’t know what “forward the invite to myself” really means. It would be matched as the “trunk” endpoint and directed to the dialplan of the context configured for “trunk”. Whatever that dialplan context does is up to you. If you have it configured to dial back out to your proxy, then that’s what it would do.

If you can’t do IP based matching for whatever reason then you could switch to user based, if that doesn’t work you’d need to show what is going on with that.

And if all traffic goes through the proxy you COULD try changing the endpoint identifier order[1] to prefer username first instead of IP address and see if matching based on IP still works. I don’t know if that would work for your setup.

Your use of proxies is really what complicates all of this, because you have to be specific with matching and understand precisely what is going on and define how it should all work together. Asterisk to Asterisk is easier.

[1] asterisk/pjsip.conf.sample at master · asterisk/asterisk · GitHub

I don’t know what “forward the invite to myself” really means.

Sorry for the confusion. I make the below change to my identifier on Asterisk @ System A (10.0.0.13:5080) so that it would identify all packets coming (incoming) from the Kamailio which is running on the same system on port 5060

[trunk]
type=identify
endpoint=trunk
match=10.0.0.13:5060

That proved to be problematic as both incoming and outgoing connections are coming from Kamailio while my trunk was configured to use “incoming” context only.

If you can’t do IP based matching for whatever reason then you could switch to user based

I feel like user based authentication is exactly what I need to do as all the sip uris in my local network is actually known to me. Once I successfully connect my boxes, I will post an update here.

And if all traffic goes through the proxy you COULD try changing the endpoint identifier order[1]

And this was very useful to know too!

What you pointed out was indeed the problem, thanks for your kind and speedy help @jcolp.

Here is my solution to this problem:

-I created a trunk just for Kamailio so that Asterisk did not challenge any connections coming from it.
-I created another trunk for inter-asterisk connections.
-I have changed the endpoint_identifier_order=username,ip,anonymous so that my local endpoints could identify with their own aors and not with the Kamailio trunk (since all packets are arriving through a single trunk now).

I don’t know if this is the best way to do it but with these changes I was able to make a call.

Thanks again @jcolp