Calls just stop coming in?

I am having a strange issue with our Asterisk server. After it has been running for about a day, sometimes two, calling our number results in “this number has been changed or disconnected” and the call never even reaches our server. I cannot figure out why. I do not see anything in the logs, so I am going to post my firewall and my log. I just cannot figure it out and the fix is to SSH in and reboot Asterisk, which works EVERY TIME.

~# cat ./asterisk.log
[Oct  1 11:38:05] NOTICE[32583] cdr.c: CDR simple logging enabled.
[Oct  1 11:38:05] NOTICE[32583] loader.c: 221 modules will be loaded.
[Oct  1 11:38:05] NOTICE[32583] res_odbc.c: res_odbc loaded.
[Oct  1 11:38:05] NOTICE[32583] res_smdi.c: No SMDI interfaces are available to listen on, not starting SMDI listener.
[Oct  1 11:38:05] NOTICE[32583] config.c: Registered Config Engine sqlite
[Oct  1 11:38:05] NOTICE[32583] res_config_ldap.c: No directory user found, anonymous binding as default.
[Oct  1 11:38:05] ERROR[32583] res_config_ldap.c: No directory URL or host found.
[Oct  1 11:38:05] ERROR[32583] res_config_ldap.c: Cannot load LDAP RealTime driver.
[Oct  1 11:38:05] NOTICE[32583] config.c: Registered Config Engine curl
[Oct  1 11:38:05] NOTICE[32583] chan_sip.c: The 'username' field for sip peers has been deprecated in favor of the term 'defaultuser'
[Oct  1 11:38:06] NOTICE[32583] chan_skinny.c: Configuring skinny from skinny.conf
[Oct  1 11:38:06] NOTICE[32583] cdr_pgsql.c: cdr_pgsql configuration contains no global section, skipping module load.
[Oct  1 11:38:06] WARNING[32583] cel_pgsql.c: CEL pgsql config file missing global section.
[Oct  1 11:38:06] NOTICE[32583] cel_tds.c: cel_tds has no global category, nothing to configure.
[Oct  1 11:38:06] WARNING[32583] cel_tds.c: cel_tds module had config problems; declining load
[Oct  1 11:38:06] ERROR[32583] chan_vpb.cc: No Voicetronix cards detected
[Oct  1 11:38:06] NOTICE[32583] pbx_ael.c: Starting AEL load process.
[Oct  1 11:38:06] NOTICE[32583] pbx_ael.c: AEL load process: parsed config file name '/etc/asterisk/extensions.ael'.
[Oct  1 11:38:06] NOTICE[32583] pbx_ael.c: AEL load process: checked config file name '/etc/asterisk/extensions.ael'.
[Oct  1 11:38:06] NOTICE[32583] pbx_ael.c: AEL load process: compiled config file name '/etc/asterisk/extensions.ael'.
[Oct  1 11:38:06] NOTICE[32583] pbx_ael.c: AEL load process: merged config file name '/etc/asterisk/extensions.ael'.
[Oct  1 11:38:06] NOTICE[32583] pbx_ael.c: AEL load process: verified config file name '/etc/asterisk/extensions.ael'.
[Oct  1 11:38:06] NOTICE[32583] chan_ooh323.c: Unable to load config ooh323.conf, OOH323 disabled
[Oct  1 11:38:06] ERROR[32583] ais/clm.c: Could not initialize cluster membership service: Try Again
[Oct  1 11:39:06] NOTICE[32694] Ext. s:  Incoming call from "Travis Juan D " <910yyyxxxx>
[Oct  1 11:41:24] NOTICE[346] Ext. s:  Incoming call from "Travis Juan D " <910yyyxxxx>
[Oct  1 11:45:51] NOTICE[637] Ext. 910yyyxxxx:  Dialing out from "Front Desk" <101> to 1910yyyxxxx through Nextiva
[Oct  1 12:00:25] NOTICE[1308] Ext. 910yyyxxxx:  Dialing out from "Front Desk" <101> to 1910yyyxxxx through Nextiva
[Oct  1 12:01:03] NOTICE[1363] Ext. 910yyyxxxx:  Dialing out from "Front Desk" <101> to 1910yyyxxxx through Nextiva
[Oct  1 12:03:40] NOTICE[1447] Ext. 910yyyxxxx:  Dialing out from "Front Desk" <101> to 1910yyyxxxx through Nextiva
[Oct  1 12:32:15] NOTICE[2825] Ext. 910yyyxxxx:  Dialing out from "Front Desk" <101> to 1910yyyxxxx through Nextiva
[Oct  1 12:39:35] NOTICE[3197] Ext. 910yyyxxxx:  Dialing out from "Front Desk" <101> to 1910yyyxxxx through Nextiva
[Oct  1 12:59:27] NOTICE[4206] Ext. 910yyyxxxx:  Dialing out from "Front Desk" <101> to 1910yyyxxxx through Nextiva
[Oct  1 13:00:20] NOTICE[4252] Ext. 910yyyxxxx:  Dialing out from "Front Desk" <101> to 1910yyyxxxx through Nextiva
[Oct  2 09:09:18] NOTICE[31688] Ext. 910yyyxxxx:  Dialing out from "Front Desk" <101> to 1910yyyxxxx through Nextiva
[Oct  2 09:25:37] NOTICE[32349] Ext. s:  Incoming call from "DANIELS RICHARD" <910yyyxxxx>
[Oct  2 09:26:06] WARNING[32349] file.c: File are-you-still-there does not exist in any format
[Oct  2 09:26:06] WARNING[32349] file.c: Unable to open are-you-still-there (format 0x4 (ulaw)): No such file or directory
[Oct  2 09:26:06] WARNING[32349] app_playback.c: ast_streamfile failed on SIP/line1-00000016 for are-you-still-there
[Oct  2 09:26:09] WARNING[32600] chan_sip.c: Retransmission timeout reached on transmission BW062505678021015246040749@192.168.145.101 for seqno 341647112 (Critical Response) -- See https://wiki.asterisk.org/wiki/display/AST/SIP+Retransmissions
Packet timed out after 31999ms with no response
[Oct  2 09:26:09] WARNING[32600] chan_sip.c: Hanging up call BW062505678021015246040749@192.168.145.101 - no reply to our critical packet (see https://wiki.asterisk.org/wiki/display/AST/SIP+Retransmissions).
[Oct  2 09:27:51] NOTICE[32471] Ext. 910yyyxxxx:  Dialing out from "Front Desk" <101> to 1910yyyxxxx through Nextiva
[Oct  2 09:28:13] NOTICE[32513] Ext. s:  Incoming call from "DANIELS RICHARD" <910yyyxxxx>
[Oct  2 09:28:59] NOTICE[32530] Ext. 910yyyxxxx:  Dialing out from "Front Desk" <101> to 1910yyyxxxx through Nextiva
[Oct  2 09:29:10] NOTICE[32581] Ext. 910yyyxxxx:  Dialing out from "Front Desk" <101> to 1910yyyxxxx through Nextiva
[Oct  2 09:31:36] NOTICE[32754] Ext. s:  Incoming call from "Travis Juan D " <910yyyxxxx>
[Oct  2 09:39:04] NOTICE[753] Ext. 910yyyxxxx:  Dialing out from "Front Desk" <101> to 1910yyyxxxx through Nextiva
[Oct  2 09:44:46] NOTICE[1011] Ext. s:  Incoming call from "Unavailable" <8889137654>
[Oct  2 09:45:16] WARNING[1011] file.c: File are-you-still-there does not exist in any format
[Oct  2 09:45:16] WARNING[1011] file.c: Unable to open are-you-still-there (format 0x4 (ulaw)): No such file or directory
[Oct  2 09:45:16] WARNING[1011] app_playback.c: ast_streamfile failed on SIP/line1-00000024 for are-you-still-there
[Oct  2 09:45:29] NOTICE[1060] Ext. 910yyyxxxx:  Dialing out from "Front Desk" <101> to 1910yyyxxxx through Nextiva
[Oct  2 10:24:46] NOTICE[2992] Ext. 910yyyxxxx:  Dialing out from "Front Desk" <101> to 1910yyyxxxx through Nextiva
[Oct  2 10:41:31] NOTICE[3705] Ext. s:  Incoming call from "Blaylock F " <910yyyxxxx>
[Oct  2 10:41:42] NOTICE[3706] Ext. s:  Incoming call from "Blaylock F " <910yyyxxxx>
[Oct  2 10:42:04] NOTICE[3707] Ext. s:  Incoming call from "Blaylock F " <910yyyxxxx>
[Oct  2 10:42:11] WARNING[3706] file.c: File are-you-still-there does not exist in any format
[Oct  2 10:42:11] WARNING[3706] file.c: Unable to open are-you-still-there (format 0x4 (ulaw)): No such file or directory
[Oct  2 10:42:11] WARNING[3706] app_playback.c: ast_streamfile failed on SIP/line1-0000002b for are-you-still-there
[Oct  2 10:42:14] WARNING[32600] chan_sip.c: Retransmission timeout reached on transmission BW0741109730210151087382925@192.168.145.101 for seqno 343929759 (Critical Response) -- See https://wiki.asterisk.org/wiki/display/AST/SIP+Retransmissions
Packet timed out after 32000ms with no response
[Oct  2 10:42:14] WARNING[32600] chan_sip.c: Hanging up call BW0741109730210151087382925@192.168.145.101 - no reply to our critical packet (see https://wiki.asterisk.org/wiki/display/AST/SIP+Retransmissions).
[Oct  2 11:00:31] NOTICE[4490] Ext. s:  Incoming call from "Cell Phone NC" <910yyyxxxx>
[Oct  2 11:00:31] NOTICE[4491] Ext. s:  Incoming call from "Cell Phone NC" <910yyyxxxx>
[Oct  2 11:00:33] NOTICE[4492] Ext. s:  Incoming call from "Cell Phone NC" <910yyyxxxx>
[Oct  2 11:00:38] NOTICE[4493] Ext. s:  Incoming call from "Cell Phone NC" <910yyyxxxx>
[Oct  2 11:00:40] NOTICE[4494] Ext. s:  Incoming call from "Cell Phone NC" <910yyyxxxx>
[Oct  2 11:01:02] WARNING[4492] file.c: File are-you-still-there does not exist in any format
[Oct  2 11:01:02] WARNING[4492] file.c: Unable to open are-you-still-there (format 0x4 (ulaw)): No such file or directory
[Oct  2 11:01:02] WARNING[4492] app_playback.c: ast_streamfile failed on SIP/line1-00000030 for are-you-still-there
[Oct  2 11:01:05] WARNING[32600] chan_sip.c: Retransmission timeout reached on transmission BW080002024021015-1382902351@192.168.145.101 for seqno 344495285 (Critical Response) -- See https://wiki.asterisk.org/wiki/display/AST/SIP+Retransmissions
Packet timed out after 32000ms with no response
[Oct  2 11:01:05] WARNING[32600] chan_sip.c: Hanging up call BW080002024021015-1382902351@192.168.145.101 - no reply to our critical packet (see https://wiki.asterisk.org/wiki/display/AST/SIP+Retransmissions).
[Oct  2 11:01:09] WARNING[4494] file.c: File are-you-still-there does not exist in any format
[Oct  2 11:01:09] WARNING[4494] file.c: Unable to open are-you-still-there (format 0x4 (ulaw)): No such file or directory
[Oct  2 11:01:09] WARNING[4494] app_playback.c: ast_streamfile failed on SIP/line1-00000033 for are-you-still-there
[Oct  2 11:01:12] WARNING[32600] chan_sip.c: Retransmission timeout reached on transmission BW080009052021015-861003138@192.168.145.101 for seqno 344498799 (Critical Response) -- See https://wiki.asterisk.org/wiki/display/AST/SIP+Retransmissions
Packet timed out after 32000ms with no response
[Oct  2 11:01:12] WARNING[32600] chan_sip.c: Hanging up call BW080009052021015-861003138@192.168.145.101 - no reply to our critical packet (see https://wiki.asterisk.org/wiki/display/AST/SIP+Retransmissions).
[Oct  2 11:23:32] NOTICE[5737] Ext. 910yyyxxxx:  Dialing out from "Front Desk" <101> to 1910yyyxxxx through Nextiva
[Oct  2 11:26:56] NOTICE[5857] Ext. s:  Incoming call from "ELLIS VERNON " <910yyyxxxx>
[Oct  2 11:27:05] WARNING[5857] file.c: File option-is-invalid does not exist in any format
[Oct  2 11:27:05] WARNING[5857] file.c: Unable to open option-is-invalid (format 0x4 (ulaw)): No such file or directory
[Oct  2 11:27:05] WARNING[5857] app_playback.c: ast_streamfile failed on SIP/line1-00000036 for option-is-invalid
[Oct  2 11:27:06] WARNING[5857] file.c: File option-is-invalid does not exist in any format
[Oct  2 11:27:06] WARNING[5857] file.c: Unable to open option-is-invalid (format 0x4 (ulaw)): No such file or directory
[Oct  2 11:27:06] WARNING[5857] app_playback.c: ast_streamfile failed on SIP/line1-00000036 for option-is-invalid
[Oct  2 11:27:09] WARNING[5857] file.c: File option-is-invalid does not exist in any format
[Oct  2 11:27:09] WARNING[5857] file.c: Unable to open option-is-invalid (format 0x4 (ulaw)): No such file or directory
[Oct  2 11:27:09] WARNING[5857] app_playback.c: ast_streamfile failed on SIP/line1-00000036 for option-is-invalid

My firewall is as follows.

~# iptables -S
-P INPUT ACCEPT
-P FORWARD DROP
-P OUTPUT ACCEPT
-N ASIP
-N DPTS
-N ICMPALL
-N IPSPF
-N RLMSET
-A INPUT -p tcp -m tcp --dport 5060:5082 -m conntrack --ctstate RELATED,ESTABLISHED -m recent ! --rcheck --name MYSIP --rsource -j DROP
-A INPUT -m conntrack --ctstate INVALID -j DROP
-A INPUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A INPUT -i lo -j ACCEPT
-A INPUT -m recent --update --seconds 600 --hitcount 1 --name RLM --rsource -j DROP
-A INPUT -p icmp -m icmp --icmp-type any -j ICMPALL
-A INPUT -p udp -m udp --sport 67:68 --dport 67:68 -j ACCEPT
-A INPUT -i eth+ -j IPSPF
-A INPUT -p tcp -m tcp --dport 22 -j ACCEPT
-A INPUT -j ASIP
-A INPUT -j DPTS
-A INPUT -m limit --limit 10/min -j LOG
-A INPUT -j DROP
-A ASIP -p tcp -m tcp --dport 5060:5082 -j ACCEPT
-A ASIP -p udp -m udp --dport 5060:5082 -m recent --update --name MYSIP --rsource -j ACCEPT
-A ASIP -p udp -m udp --dport 5060:5082 -j DROP
-A ASIP -p udp -m udp --dport 10000:20000 -j ACCEPT
-A ASIP -j RETURN
-A DPTS -p tcp -m tcp --dport 21 -j DROP
-A DPTS -p tcp -m tcp --dport 22 -j ACCEPT
-A DPTS -p tcp -m tcp --dport 23 -j RLMSET
-A DPTS -p tcp -m tcp --dport 25 -j RLMSET
-A DPTS -p tcp -m tcp --dport 80 -j DROP
-A DPTS -p tcp -m tcp --dport 443 -j DROP
-A DPTS -p tcp -m tcp --dport 1433 -j RLMSET
-A DPTS -p tcp -m tcp --dport 3128 -j RLMSET
-A DPTS -p tcp -m tcp --dport 3306 -j RLMSET
-A DPTS -p tcp -m tcp --dport 3389 -j RLMSET
-A DPTS -p tcp -m tcp --dport 4899 -j RLMSET
-A DPTS -p tcp -m tcp --dport 5900 -j RLMSET
-A DPTS -j RETURN
-A ICMPALL -p icmp -f -j DROP
-A ICMPALL -p icmp -m icmp --icmp-type 0 -j ACCEPT
-A ICMPALL -p icmp -m icmp --icmp-type 3 -j ACCEPT
-A ICMPALL -p icmp -m icmp --icmp-type 4 -j ACCEPT
-A ICMPALL -p icmp -m icmp --icmp-type 8 -j ACCEPT
-A ICMPALL -p icmp -m icmp --icmp-type 11 -j ACCEPT
-A ICMPALL -p icmp -j DROP
-A IPSPF -s 10.0.0.0/8 -j DROP
-A IPSPF -s 172.16.0.0/12 -j DROP
-A IPSPF -s 192.168.0.0/16 -j DROP
-A IPSPF -s 0.0.0.0/8 -j DROP
-A IPSPF -s 100.64.0.0/10 -j DROP
-A IPSPF -s 127.0.0.0/8 -j DROP
-A IPSPF -s 169.254.0.0/16 -j DROP
-A IPSPF -s 192.0.0.0/24 -j DROP
-A IPSPF -s 192.0.2.0/24 -j DROP
-A IPSPF -s 198.18.0.0/15 -j DROP
-A IPSPF -s 198.51.100.0/24 -j DROP
-A IPSPF -s 203.0.113.0/24 -j DROP
-A IPSPF -s 224.0.0.0/4 -j DROP
-A IPSPF -s 240.0.0.0/4 -j DROP
-A IPSPF -s 255.255.255.255/32 -j DROP
-A IPSPF -d 0.0.0.0/8 -j DROP
-A IPSPF -d 127.0.0.0/8 -j DROP
-A IPSPF -d 224.0.0.0/4 -j DROP
-A IPSPF -d 255.255.255.255/32 -j DROP
-A IPSPF -p tcp -m tcp --tcp-flags FIN,SYN,RST,PSH,ACK,URG NONE -j DROP
-A IPSPF -p tcp -m tcp --tcp-flags FIN,SYN,RST,PSH,ACK,URG FIN,SYN,RST,PSH,ACK,URG -j DROP
-A IPSPF -p tcp -m tcp --tcp-flags FIN,SYN,RST,PSH,ACK,URG FIN,PSH,URG -j DROP
-A IPSPF -p tcp -m tcp --tcp-flags FIN,SYN,RST,PSH,ACK,URG FIN,SYN,RST,ACK,URG -j DROP
-A IPSPF -p tcp -m tcp --tcp-flags SYN,RST SYN,RST -j DROP
-A IPSPF -p tcp -m tcp --tcp-flags FIN,SYN FIN,SYN -j DROP
-A IPSPF -p tcp -m tcp --tcp-flags FIN,SYN,RST,PSH,ACK,URG FIN -j DROP
-A IPSPF -p tcp -m tcp --tcp-flags SYN,ACK SYN,ACK -m conntrack --ctstate NEW -j REJECT --reject-with tcp-reset
-A IPSPF -p tcp -m tcp ! --tcp-flags FIN,SYN,RST,ACK SYN -m conntrack --ctstate NEW -j DROP
-A IPSPF -p udp -m length --length 0:28 -j DROP
-A IPSPF -p tcp -m tcp --tcp-flags FIN,SYN,RST,ACK SYN -m recent --update --seconds 1 --hitcount 11 --name INSYN --rsource -j DROP
-A IPSPF -p tcp -m tcp --tcp-flags FIN,SYN,RST,ACK SYN -m recent --set --name INSYN --rsource -j RETURN
-A IPSPF -j RETURN
-A RLMSET -m recent --set --name RLM --rsource -j DROP

Disabling the firewall does not help, but then I start getting a bunch of random addresses trying to login and make calls.

Alright, I have been working with our trunk provider and we are stumped here. In an effort to figure this out, I have simplified my firewall to only allow port 5060 UDP connections from out office address, which is static. Will my provider also need access to port 5060? This UDP stuff seriously complicates things when you want security. Removing the firewall results in a flood of attacks, but it seems like any firewall breaks SIP. This has to be possible somehow.

Also, our provider and phones all support H.323. Since SIP seems to be hastily thrown together and messy, we are looking at switching to H.323. How can we go about that?

EDIT

Here is the current firewall.

-P INPUT DROP
-P FORWARD DROP
-P OUTPUT ACCEPT
-A INPUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A INPUT -p icmp -m icmp --icmp-type 3 -j ACCEPT
-A INPUT -p icmp -m icmp --icmp-type 11 -j ACCEPT
-A INPUT -p icmp -m icmp --icmp-type 12 -j ACCEPT
-A INPUT -p tcp -m tcp --dport 113 --tcp-flags FIN,SYN,RST,ACK SYN -j REJECT --reject-with tcp-reset
-A INPUT -p tcp -m state --state NEW -m tcp --dport 22 -j ACCEPT
-A INPUT -s 111.222.333.444/32 -p udp -m state --state NEW -m multiport --dports 5060,10000:20000 -j ACCEPT
-A INPUT -i lo -j ACCEPT

I only changed our main office IP address for security reasons. It is correct on the server. Our server also has a static address and is not protected by NAT or anything other than iptables. This is why having a working firewall is so important. This thing sits on the Internet with no other protection.

I figured it out. When Asterisk connects to our trunk a rule is added to allow communication with that IP address. However, our provider uses not one, but TWO clouds (clusters) of servers. So I am connected to one, but a call could come in from a load of others, and the firewall blocks it. I discovered this while working with a level 2 technician at our provider. He gave me the networks and masks of both clouds and I added both networks to the firewall for port 5060. All has been fine since.

I guess this goes to show that you need every last detail of information about your provider if you want to secure your server. Oh well, live and learn!

H.323 is much less used than SIP, and has a similar structure in terms of having a signalling channel and media over RTP. As such I would think it is more likely to cause problems with firewalls.

Actually, it uses TCP for transport by default, making it much more firewall friendly. It scales better and works seamlessly with PSTN equipment. A decent comparison is linked below. However, since the change to our firewall we have had no issues, so I may just leave this alone for now.

packetizer.com/ipmc/h323_vs_sip/

It uses UDP for media, but the real issue is that hardly anyone uses it any longer, and, in the case of Asterisk, there has been no real development on the code for many years.

My impression is that, when it was used, it was used for large company internally use, so NAT was not a real issue. There is an assumption that you have a gatekeeper, so individual devices never have to signal across boundaries.

I note that Windows switched from H.323.to SIP for their messenger type offerings.

Maybe Asterisk dropped support, but I can find as many H.323 products as SIP. You did pique my interest however. What Microsoft products use SIP? I know Skype does NOT use SIP. In fact, I am not sure what the heck Skype uses, but I dislike it immensely since MS bought it anyway. I am still waiting for a friggin’ 64bit version for Linux…

I get confused about the names of the Microsoft offerings, but the first audio conferencing system used H.323 and MSN Messenger, the one that was replaced by Skype, used SIP. If you had the office version, it could initiate normal SIP calls, but even the home version used SIP for voice call setup, but in a complicated way. The party originating the call made an MSN Messenger protocol request to the called party, which then did a SIP INVITE back to the initiator.