Asterisk stops accepting new TLS connections

[lemongrass]

Hello

We have a pretty big Asterisk 18.15.1 deployment (~1500 registered clients). The transport is TLS only and we just use the PJSIP channel driver.

Somehow, we already entered the Situation where the Server stopped accepting new TLS connections. This was proven both by Uptimerobot reporting an Offline Status and also SIP clients unable to register.

This Situation seems happening approx. every month (uptime). Not yet sure if this is a hard fact or just coincidence.

Already initiated connections seem to remain open.

The log doesn’t show any error or warning.

Any idea how this can be diagnosed further if it happens again?

[meightee]

Hello,

“The log” doesnt show any error or warning, which log are you talking about?

I have experienced this issue with Kamailio, where it was the tcp_conn tracker not having enough memory…

The syslog clearly showed that…

[lemongrass]

Hello

I have checked the Syslog. There is no entry in the Syslog at the time when the Server stopped accepting TLS Sockets.

Interestingly, the same issue now reoccurred after just about 4 days runtime.

Is there also a way to configure TCP/TLS connections/buffers in Asterisk for large deployments?

[lemongrass]

Ok, maybe this is just an out of FD issue. Will increase them now and check if it happens again…

$ netstat -tn | grep ESTABLISHED | wc -l
734
$ ulimit -n
1024

[lemongrass]

Happened again after a few days uptime, even with increased FD.

[ldo]

You can check the number of connections currently open with “ss -nt” or (pre-iproute2) “netstat -nt”. If you see an ever-increasing list of entries here, then somehow you are opening connections but not closing them. Then you can use the lsof command to find out which processes are holding those connections open.

Just a guess.

[lemongrass]

I think I was able to fix it with adding this to the systemd service file. Somehow it sometimes spikes to over 1024 used fd.

LimitNOFILE=4096