PJSIP shows all lists as empty from CLI

eepstein · October 7, 2016, 12:03am

I’m using Asterisk on OpenBSD 5.9 from packages
_> Asterisk 13.7.1 built by pbuild @ i386.ports.openbsd.org on a i386 running OpenBSD on 2016-02-24 18:34:10 UTC

I’ve had no luck with PostgreSQL and PJSIP being stable with AORs and Contacts so I installed the basic version without PostgreSQL support. The configurations I started from were the stock configs. extconfig.conf only has the ‘[settings]’ line it, and I’ve tried with and without a sorcery file. Stock configs don’t provide a sorcery file configured for configuration file storage.

What’s strange is that the command ‘pjsip show endpoints’ always returns ‘no objects found’, but I have two endpoints registered and able to setup and complete a call. The CLI also shows me messages about Contacts being removed and/or created. Running the command ‘pjsip show endpoint 300’ returns information about the Endpoint. Aor and Auth work in the same fashion, but Contacts returns ‘no objects found’ as well.

While it seems like its working just fine, there has to be something wrong if the CLI is unable to enumerate and show the Endpoints.

Any ideas are appreciated. Any working configuration files that use config storage would be helpful too.

The sorcery file:

[res_pjsip]
endpoint=config,pjsip.endpoint.conf,criteria=type=endpoint
aor=config,pjsip.conf,criteria=type=aor
auth=config,pjsip.conf,criteria=type=auth
;contact=config,pjsip.conf,criteria=type=contact

[res_pjsip_endpoint_identifier_ip]
identify=config,pjsip.conf,criteria=type=identify

The pjsip conf files:

;#include pjsip.transports.conf
#include pjsip.endpoint.conf
;#include pjsip.aor.conf
;#include pjsip.auth.conf
;#include pjsip.registration.conf
;#include pjsip.identify.conf

[general]
tos=cs3
tos_audio=ef
tos_video=af41
cos=3
cos_audio=5
cos_video=3

[global]
type=global
user_agent=SystemOne v1.0
default_outbound_endpoint=dpma_endpoint

[updbindlocal]
type=transport
protocol=udp
bind=10.88.33.3:5060
external_media_address=216.218.241.203
external_signaling_address=216.218.241.203
local_net=10.0.0.0/8

auth-userpass
type=auth
auth_type=userpass

aor-single-reg
type=aor
max_contacts=2
remove_existing=yes
maximum_expiration=7200
minimum_expiration=60
qualify_frequency=60

;; End Points

[300-identify]
type=identify
endpoint=300
match=10.88.33.71/32

auth300
password=devpassone
username=300

300
mailboxes=300@imapvm

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

[301-identify]
type=identify
endpoint=301
match=10.88.33.62/32

auth301
password=devpassone
username=301

301
mailboxes=301@imapvm

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

[dpma_endpoint]
type=endpoint
context=dpma-invalid

endpoint-basic
type=endpoint
transport=updbindlocal
context=internal
disallow=all
allow=g729,ulaw
dtmf_mode=rfc4733
aggregate_mwi=yes
use_avpf=no
ice_support=no
media_use_received_transport=no
trust_id_inbound=yes
media_encryption=no
media_encryption_optimistic=no
rtp_symmetric=yes
rewrite_contact=yes
force_rport=yes
language=en

300
auth=auth300
aors=300
accountcode=300

301
auth=auth301
aors=301
accountcode=301

gjoseph · October 7, 2016, 2:06pm

I’m not sure whether you’re having the CLI issue with the database or config files but when using realtime for PJSIP, the CLI commands won’t search the database for performance reasons. We had complaints that someone accidentally hit the key after typing “pjsip show endpoint” and we happily went off and retrieved 2000 endpoints from the database to show as possible completions.

In later versions of Asterisk (13.8 i think), additional options were added to the mappings in sorcery.conf that allow you to add a cache and set whether you’ll allow unqualified searches.

`

eepstein · October 7, 2016, 7:01pm

@gjoseph

Interesting. The CLI issue is with config files only. When connected to the database the CLI will show everything. That’s with or without caching. I understand the performance reasons, but I was expecting to verify database connectivity by asking Asterisk to show me via the CLI. I believe it even showed Contacts. I get it, but I wasn’t expecting it

This specific setup though is without database support at all, and I was expecting under that scenario to have Asterisk directly read the file and enumerate my Endpoints for me. The fact it doesn’t makes me feel that it is broken, and the fact the Endpoints are working just confuses me. The fact I can’t get a working Asterisk/PJSIP under any circumstances is trying my patience to put it mildly. I like PJSIP from an architecture point of view, but I’ve yet to see it work unfortunately

My expectations after reading documentation were that everything would be stored and updated in the database, but I could also cache things as necessary. For some things I was going to attempt to integrate a Redis cluster as well. I haven’t seen the database work yet though. When I expect Asterisk to update Aors and Contacts, it doesn’t. Looking at PJSIP debugging I can now see my problem is the 10.0.0.0:5060 as my defaults. However, I can’t possibly predict the correct Contact when populating it initially can I? Two Contacts exist for each Endpoint. One is the initially provisioned one to get it to work, and the 2nd is the one added from registration. The first is not expired with remove_existing=yes, and Asterisk attempts to use the bad URI to make contact. Time doesn’t help either as the bad Contact is never removed or updated.

This wasn’t the behavior with ODBC, Firebird, and Asterisk 1.6.2. The sip table must be populated with a SIP URI initially for provisioning, but after the first registration from the Endpoint, it was kept updated by Asterisk and functioned without incident.

I would really love to use a local database, even to the extent of storing configuration files too, but I’m having serious issues with PostgreSQL and Asterisk 13. Tables are not being written to, and certain records are not being updated. SQL logging confirms it.

A working configuration using flat files or realtime is what I need. Either one of them, I can still work with.

sorcery caching (direct from the Wiki and it looked suitable for my testing)

endpoint/cache = memory_cache,object_lifetime_stale=600,object_lifetime_maximum=1800,expire_on_reload=yes
auth/cache=memory_cache,expire_on_reload=yes
aor/cache = memory_cache,object_lifetime_stale=1500,object_lifetime_maximum=1800,expire_on_reload=yes

gjoseph · October 7, 2016, 7:25pm

Let’s take 1 thing at a time…

Get asterisk 13.11.2. 13.7 is ancient and there have been many realtime/database/sorcery fixes since then.
Rename extconfig.conf and sorcery.conf and just use config files and make sure you’re stable.
Set up extconfig for endpoints, aors and auths ONLY and use ODBC to get to postgres. Do NOT use realtime for contacts. Since contacts are written frequently, it’ll make performance worse. endpoints, aors and auths are never written to. The ODBC driver is the only officially supported one for use with sorcery/pjsip.
Set up sorcery.conf to use only the basic config for endpoints, aors and auths. I.E.: endpoint=realtime,ps_endpoints.
Once that’s working you can play with the cache settings. Do NOT mix config files and realtime for the same object type in sorcery.conf.

eepstein · October 8, 2016, 10:02pm

@gjoseph

Thank you for the instructions. #1 is a bit problematic as it required me to compile from source on OpenBSD 5.9. The current package for 5.9 is 13.7 instead. With OpenBSD it’s often better to use the packages.

I’m unable to get a working Asterisk from source at the moment, but I’ll keep trying. Using safe_asterisk I get a response code of 128, and running asterisk I receive

/usr/local/sbin/asterisk: can’t load library ‘libasteriskssl.so.0.0’

gjoseph · October 8, 2016, 10:36pm

First, make sure you’ve uninstalled the packaged version of asterisk.
Make sure there are no libasterisk* libraries in /usr/lib or /usr/local/lib. If there are, delete them.
Re run the asterisk make install
Run ldconfig

If that doesn’t help…
If you did NOT specify --prefix when you ran configure then the install will be to /usr/local.
If you did, the install will be to wherever you specified.
Look for libasteriskssl.so* in /usr/local/lib or wherever.
If it’s not there, the install didn’t work.

eepstein · October 11, 2016, 12:21am

@gjoseph

Yes, I uninstalled the package first. I did remove the libraries as suggested and recompiled. While I ran ldconfig, it had no output. I also ran gmake basic-pbx at the end to get a basic configuration for Asterisk to start with.

The files are there, but no script exist inside /etc/rc.d/ to start Asterisk anymore. gmake config has no init scripts for OpenBSD either. Running ‘/usr/local/sbin/asterisk -c’ does attempt to start Asterisk. I immediately receive errors:

# /usr/local/sbin/asterisk -rvvvvvvvvvvvvvvvvvvvvvvv
/usr/local/sbin/asterisk: can’t load library ‘liburiparser.so.0.1’
# /usr/local/sbin/asterisk -rvvvvvvvvvvvvvvvvvvvvvvv
/usr/local/sbin/asterisk: can’t load library ‘libiconv.so.6.0’
# /usr/local/sbin/asterisk -rvvvvvvvvvvvvvvvvvvvvvvv
/usr/local/sbin/asterisk: can’t load library ‘liburiparser.so.0.1’
# /usr/local/sbin/asterisk -rvvvvvvvvvvvvvvvvvvvvvvv
/usr/local/sbin/asterisk: can’t load library ‘libjansson.so.1.0’
# /usr/local/sbin/asterisk -rvvvvvvvvvvvvvvvvvvvvvvv
/usr/local/sbin/asterisk: can’t load library ‘libxml2.so.15.1’
# /usr/local/sbin/asterisk -rvvvvvvvvvvvvvvvvvvvvvvv
/usr/local/sbin/asterisk: can’t load library ‘liburiparser.so.0.1’
\ # /usr/local/sbin/asterisk -rvvvvvvvvvvvvvvvvvvvvvvv
/usr/local/sbin/asterisk: can’t load library ‘libjansson.so.1.0’
# /usr/local/sbin/asterisk -rvvvvvvvvvvvvvvvvvvvvvvv
/usr/local/sbin/asterisk: can’t load library ‘libxslt.so.3.8’
# /usr/local/sbin/asterisk -rvvvvvvvvvvvvvvvvvvvvvvv
/usr/local/sbin/asterisk: can’t load library ‘libiconv.so.6.0’
# /usr/local/sbin/asterisk -rvvvvvvvvvvvvvvvvvvvvvvv
/usr/local/sbin/asterisk: can’t load library ‘libasteriskssl.so’
# /usr/local/sbin/asterisk -rvvvvvvvvvvvvvvvvvvvvvvv
/usr/local/sbin/asterisk: can’t load library ‘libxml2.so.15.1’

They all exist under /usr/local/lib.

There may be a reason why OpenBSD usually insists you install something from its package. Any ideas on how to proceed here?

gjoseph · October 11, 2016, 4:26am

I have an OpenBSD virtual machine set up. Let me try to get asterisk running in the morning.

eepstein · October 11, 2016, 7:39pm

@gjoseph

Thank you very much.

gjoseph · October 11, 2016, 8:54pm

Don’t thank me yet.
I was at least able to duplicate the original issue with the packages… No objects returned.
Unfortunately, I wasn’t able to even compile asterisk on 5.9. There were too many missing dependencies and since I have limited Open/FreeBSD knowledge I wasn’t able to get much further. If I’ve got some free time over the next few days I can look again but we’re pretty swamped right now. Sorry.

eepstein · October 11, 2016, 9:05pm

@gjoseph

We’re on OpenBSD 6.0 too , but I really appreciate you working on it. Sometimes OpenBSD can be a bit difficult. Thanks for verifying the issue with the package as well. I’ll keep working on it over here myself with the source to see if I can get it stable.

eepstein · October 12, 2016, 6:12pm

I just checked into 6.0 again in the ports trying to find the maintainer. I think I have something wrong with my environment since I can see version 13.9.1 at http://mirrors.sonic.net/pub/OpenBSD/6.0/packages/i386/, but my own pkg_info -Q asterisk shows 13.7.1.

I’m going to attempt to reproduce the errors on 13.9.1 too.

eepstein · October 19, 2016, 8:07pm

I never was able to get the current version to work after compiling. Suspect there is quite a bit of work with the dependencies and I’ll continue to try to locate the port maintainer to get their view on the matter. I do find it odd that none of the Asterisk packages for PostgreSQL on OpenBSD ever provision or even look at PostgreSQL. I needed to locate the source for Asterisk 13.9.1 to get the correct Alembic version of SQL to provision the database.

Installing the Asterisk 13.9.1 package w/ PostgreSQL support on OpenBSD 6.0 seems to be stable though. CLI commands are working, and even enumerating all the Endpoints stored in the database. PJSIP is stable as well with two-way audio working and the contacts are being updated appropriately.

As you suggested I limited the realtime support, but I’ve added the Endpoint IP matches which are working as well. Here are the configurations if anyone is curious:

res_pgsql.conf

; The value of dbhost may be either a hostname or an IP address.
; If dbhost is commented out or the string "localhost", a connection
; to the local host is assumed and dbsock is used instead of TCP/IP
; to connect to the server.
;
[general]
dbhost=127.0.0.1
dbport=5432
dbname=asterisk
dbuser=postgres
dbpass=**************************
;dbappname=asterisk    ; Postgres application_name support (optional). Whitespace not allowed.
;
; dbsock is specified as the directory where the socket file may be found. The
; actual socket is constructed as a combination of dbsock and dbport.  For
; example, the values of '/tmp' and '5432', respectively, will specify a socket
; file of '/tmp/.s.PGSQL.5432'.
; 
;dbsock=/tmp
;
; requirements - At startup, each realtime family will make requirements
; on the backend.  There are several strategies for handling requirements:
; warn        - Warn if the required column does not exist.
; createclose - Create columns as close to the requirements as possible.
; createchar  - Create char columns only
;
requirements=createclose

sorcery.conf

[res_pjsip]
;endpoint/cache = memory_cache,object_lifetime_stale=600,object_lifetime_maximum=1800,expire_on_reload=yes,full_backend_cache=yes
;auth/cache = memory_cache,expire_on_reload=yes,full_backend_cache=yes
;aor/cache = memory_cache,object_lifetime_stale=1500,object_lifetime_maximum=1800,expire_on_reload=yes,full_backend_cache=yes

endpoint=realtime,ps_endpoints
auth=realtime,ps_auths
aor=realtime,ps_aors
;domain_alias=realtime,ps_domain_aliases
;contacts=realtime,ps_contacts
;transports=realtime,ps_transports

[res_pjsip_endpoint_identifier_ip]
identify=realtime,ps_endpoint_id_ips

extconfig.conf

;
; Static and realtime external configuration
; engine configuration
;
; See https://wiki.asterisk.org/wiki/display/AST/Realtime+Database+Configuration
; for basic table formatting information.
;
[settings]
ps_endpoints => pgsql,asterisk
ps_auths => pgsql,asterisk
ps_aors => pgsql,asterisk
ps_endpoint_id_ips => pgsql,asterisk

gjoseph · October 20, 2016, 12:28pm

Thanks for the update!