isdn30 trunk going down couple of times a week

I installed an Asterisk based solution that is giving problems with the incoming isdn30 trunk.

isdn30 -> Asterisk -> Tenovis pbx -> 300 tenovis phones and 2 faxes

This is connected, registered and handled on the Asterisk box.
-60 Snom320
-2 faxes connected
by Grandstream ATA’s

In the server there is a Sangoma 2 * E1 card without hardware echo cancelation. One of the ports is used for the incoming isdn30 trunk of the telco and the other to connect the asterisk box with the Tenovis PBX

Once in a while (a couple of times a week) the incoming isdn30 is losing incoming and outgoing connection. Lights on the card stay green and there is no alarm and no trace in the loggings. When making an outgoing call during an incident there is a busy tone (that can’t be provided by the Asterisk box I suppose). To get back in business one has to plug out and plug in the cable and, if this doesn’t help, restart the server.

During an incident all channels of the incoming isdn-trunk gives the following message:

Zaptel Disabled echo canceler because of tone (rx) on channel …
Zaptel Disabled echo canceler because of tone (tx) on channel …

last two lines on screen
Zaptel: Master changed to WPE/1
Zaptel: Master changed to WPE/0

For the same channel the message appears several times.

The telco has done tests and is monitoring the trunk. They are really doing the best they can and they discovered some what they call “slips” but they don’t match with the incidents. We changed the cable for a new one of the best quality available and updated the system but the problem is still there.

This is what makes the server:
-asterisk 1.4.14
-asterisk-addons 1.4.4
-wanpipe 3.2.1
-zaptel 1.4.6
-Scopserv is installed as the gui.

I asked questions on the irc channel and one suggested that the cause could be faxes. Could there really be any relation between the problem and incoming or outgoing faxes and what should be the probem. Does the message indicate that the Asterisk server is to blame or could there be a problem on the telco side. Any suggestion or pointer to resolve this problem is very much appreciated.

This is additional info from the /var/log/asterisk/messages (I used grep chan_zap.c) and the wanpipe logging.

The strange thing is that it skips the first 5 channels and channel 26 (and channel 16 but that is the signaling channel). All the alarms are released within seconds but the problem is that the connection doesn’t come back. Any suggestion is very welcome.

Dec 30 15:43:25] NOTICE[32766] chan_zap.c: Fax detected, but no fax extension
[Dec 31 09:23:02] NOTICE[12657] chan_zap.c: Fax detected, but no fax extension
[Dec 31 09:42:27] NOTICE[17688] chan_zap.c: Fax detected, but no fax extension
[Dec 31 10:13:48] WARNING[5964] chan_zap.c: Detected alarm on channel 6: Red Alarm
[Dec 31 10:13:48] WARNING[5964] chan_zap.c: Unable to disable echo cancellation on channel 6
[Dec 31 10:13:48] WARNING[5964] chan_zap.c: Detected alarm on channel 7: Red Alarm
[Dec 31 10:13:48] WARNING[5964] chan_zap.c: Unable to disable echo cancellation on channel 7
[Dec 31 10:13:48] WARNING[5964] chan_zap.c: Detected alarm on channel 8: Red Alarm
[Dec 31 10:13:48] WARNING[5964] chan_zap.c: Unable to disable echo cancellation on channel 8
[Dec 31 10:13:48] WARNING[5964] chan_zap.c: Detected alarm on channel 9: Red Alarm
[Dec 31 10:13:48] WARNING[5964] chan_zap.c: Unable to disable echo cancellation on channel 9
[Dec 31 10:13:48] WARNING[5964] chan_zap.c: Detected alarm on channel 10: Red Alarm
[Dec 31 10:13:48] WARNING[5964] chan_zap.c: Unable to disable echo cancellation on channel 10
[Dec 31 10:13:48] WARNING[5964] chan_zap.c: Detected alarm on channel 11: Red Alarm
[Dec 31 10:13:48] WARNING[5964] chan_zap.c: Unable to disable echo cancellation on channel 11
[Dec 31 10:13:48] WARNING[5964] chan_zap.c: Detected alarm on channel 12: Red Alarm
[Dec 31 10:13:48] WARNING[5964] chan_zap.c: Unable to disable echo cancellation on channel 12
[Dec 31 10:13:48] WARNING[5964] chan_zap.c: Detected alarm on channel 13: Red Alarm
[Dec 31 10:13:48] WARNING[5964] chan_zap.c: Unable to disable echo cancellation on channel 13
[Dec 31 10:13:48] WARNING[5964] chan_zap.c: Detected alarm on channel 14: Red Alarm
[Dec 31 10:13:48] WARNING[5964] chan_zap.c: Unable to disable echo cancellation on channel 14
[Dec 31 10:13:48] WARNING[5964] chan_zap.c: Detected alarm on channel 15: Red Alarm
[Dec 31 10:13:48] WARNING[5964] chan_zap.c: Unable to disable echo cancellation on channel 15
[Dec 31 10:13:48] WARNING[5964] chan_zap.c: Detected alarm on channel 17: Red Alarm
[Dec 31 10:13:48] WARNING[5964] chan_zap.c: Unable to disable echo cancellation on channel 17
[Dec 31 10:13:48] WARNING[5964] chan_zap.c: Detected alarm on channel 18: Red Alarm
[Dec 31 10:13:48] WARNING[5964] chan_zap.c: Unable to disable echo cancellation on channel 18
[Dec 31 10:13:48] WARNING[5964] chan_zap.c: Detected alarm on channel 19: Red Alarm
[Dec 31 10:13:48] WARNING[5964] chan_zap.c: Unable to disable echo cancellation on channel 19
[Dec 31 10:13:48] WARNING[5964] chan_zap.c: Detected alarm on channel 20: Red Alarm
[Dec 31 10:13:48] WARNING[5964] chan_zap.c: Unable to disable echo cancellation on channel 20
[Dec 31 10:13:48] WARNING[5964] chan_zap.c: Detected alarm on channel 21: Red Alarm
[Dec 31 10:13:48] WARNING[5964] chan_zap.c: Unable to disable echo cancellation on channel 21
[Dec 31 10:13:48] WARNING[5964] chan_zap.c: Detected alarm on channel 22: Red Alarm
[Dec 31 10:13:48] WARNING[5964] chan_zap.c: Unable to disable echo cancellation on channel 22
[Dec 31 10:13:48] WARNING[5964] chan_zap.c: Detected alarm on channel 23: Red Alarm
[Dec 31 10:13:48] WARNING[5964] chan_zap.c: Detected alarm on channel 24: Red Alarm
[Dec 31 10:13:48] WARNING[5964] chan_zap.c: Unable to disable echo cancellation on channel 24
[Dec 31 10:13:48] WARNING[5964] chan_zap.c: Detected alarm on channel 25: Red Alarm
[Dec 31 10:13:48] WARNING[5964] chan_zap.c: Unable to disable echo cancellation on channel 25
[Dec 31 10:13:48] WARNING[5964] chan_zap.c: Detected alarm on channel 27: Red Alarm
[Dec 31 10:13:48] WARNING[5964] chan_zap.c: Unable to disable echo cancellation on channel 27
[Dec 31 10:13:48] WARNING[5964] chan_zap.c: Detected alarm on channel 28: Red Alarm
[Dec 31 10:13:48] WARNING[5964] chan_zap.c: Unable to disable echo cancellation on channel 28
[Dec 31 10:13:48] WARNING[5964] chan_zap.c: Detected alarm on channel 29: Red Alarm
[Dec 31 10:13:48] WARNING[5964] chan_zap.c: Unable to disable echo cancellation on channel 29
[Dec 31 10:13:48] WARNING[5964] chan_zap.c: Detected alarm on channel 30: Red Alarm
[Dec 31 10:13:48] WARNING[5964] chan_zap.c: Unable to disable echo cancellation on channel 30
[Dec 31 10:13:48] WARNING[5964] chan_zap.c: Detected alarm on channel 31: Red Alarm
[Dec 31 10:13:48] WARNING[5964] chan_zap.c: Unable to disable echo cancellation on channel 31
[Dec 31 10:14:05] NOTICE[5964] chan_zap.c: Alarm cleared on channel 6
[Dec 31 10:14:05] NOTICE[5964] chan_zap.c: Alarm cleared on channel 7
[Dec 31 10:14:05] NOTICE[5964] chan_zap.c: Alarm cleared on channel 8
[Dec 31 10:14:05] NOTICE[5964] chan_zap.c: Alarm cleared on channel 9
[Dec 31 10:14:05] NOTICE[5964] chan_zap.c: Alarm cleared on channel 10
[Dec 31 10:14:05] NOTICE[5964] chan_zap.c: Alarm cleared on channel 11
[Dec 31 10:14:05] NOTICE[5964] chan_zap.c: Alarm cleared on channel 12
[Dec 31 10:14:05] NOTICE[5964] chan_zap.c: Alarm cleared on channel 13
[Dec 31 10:14:05] NOTICE[5964] chan_zap.c: Alarm cleared on channel 14
[Dec 31 10:14:05] NOTICE[5964] chan_zap.c: Alarm cleared on channel 15
[Dec 31 10:14:05] NOTICE[5964] chan_zap.c: Alarm cleared on channel 17
[Dec 31 10:14:05] NOTICE[5964] chan_zap.c: Alarm cleared on channel 18
[Dec 31 10:14:05] NOTICE[5964] chan_zap.c: Alarm cleared on channel 19
[Dec 31 10:14:05] NOTICE[5964] chan_zap.c: Alarm cleared on channel 20
[Dec 31 10:14:05] NOTICE[5964] chan_zap.c: Alarm cleared on channel 21
[Dec 31 10:14:05] NOTICE[5964] chan_zap.c: Alarm cleared on channel 22
[Dec 31 10:14:05] NOTICE[5964] chan_zap.c: Alarm cleared on channel 23
[Dec 31 10:14:05] NOTICE[5964] chan_zap.c: Alarm cleared on channel 24
[Dec 31 10:14:05] NOTICE[5964] chan_zap.c: Alarm cleared on channel 25
[Dec 31 10:14:05] NOTICE[5964] chan_zap.c: Alarm cleared on channel 27
[Dec 31 10:14:05] NOTICE[5964] chan_zap.c: Alarm cleared on channel 28
[Dec 31 10:14:05] NOTICE[5964] chan_zap.c: Alarm cleared on channel 29
[Dec 31 10:14:05] NOTICE[5964] chan_zap.c: Alarm cleared on channel 30
[Dec 31 10:14:05] NOTICE[5964] chan_zap.c: Alarm cleared on channel 31

Logging uit wanpipe messages (the driver of the sangoma 2 * E1 card)

Dec 31 10:13:48 asterisk kernel: wanpipe1: AIS alarm is ON
Dec 31 10:13:48 asterisk kernel: wanpipe1: RAI alarm is ON
Dec 31 10:13:48 asterisk kernel: wanpipe1: OOF alarm is ON
Dec 31 10:13:48 asterisk kernel: wanpipe1: E1 disconnected!
Dec 31 10:13:48 asterisk kernel: Zaptel: Master changed to WPE1/1
Dec 31 10:13:48 asterisk kernel: wanpipe1: AFT communications disabled!
Dec 31 10:13:48 asterisk kernel: wanpipe1: Starting TDMV 1ms Timer
Dec 31 10:13:48 asterisk kernel: wanpipe1: RED alarm is ON
Dec 31 10:13:59 asterisk kernel: wanpipe1: RAI alarm is OFF
Dec 31 10:13:59 asterisk kernel: wanpipe1: AIS alarm is OFF
Dec 31 10:13:59 asterisk kernel: wanpipe1: OOF alarm is OFF
Dec 31 10:13:59 asterisk kernel: wanpipe1: RED alarm is OFF
Dec 31 10:14:05 asterisk kernel: wanpipe1: E1 connected!
Dec 31 10:14:05 asterisk kernel: Zaptel: Master changed to WPE1/0
Dec 31 10:14:05 asterisk kernel: wanpipe1: AFT communications enabled!
Dec 31 10:14:05 asterisk kernel: wanpipe1: AFT Global TDM Intr
Dec 31 10:14:05 asterisk kernel: ADDRCONF(NETDEV_CHANGE): w1g1: link becomes ready
Dec 31 10:14:05 asterisk kernel: wanpipe1: Global TDM Ring Resync
Dec 31 10:14:05 asterisk kernel: wanpipe1: Card TDM Rsync Rx=2 Tx=2
Dec 31 10:14:05 asterisk kernel: wanpipe2: Card TDM Rsync Rx=1 Tx=1
Dec 31 10:14:06 asterisk kernel: wanpipe1: RAI alarm is OFF
Dec 31 10:14:06 asterisk kernel: wanpipe1: AIS alarm is OFF
Dec 31 10:14:06 asterisk kernel: wanpipe1: OOF alarm is OFF

see: atis.org/tg2k/_alarm_indication_signal.html
alarm indication signal (AIS)

info about the alarms:

alarm indication signal (AIS): 1. A signal transmitted in lieu of the normal signal to maintain transmission continuity and to indicate to the receiving equipment that there is a transmission interruption located either at the equipment originating the AIS signal or upstream of that equipment. [T1.231-1997] 2. A signal transmitted in lieu of the normal signal to maintain transmission continuity and indicate to the receiving terminal that there is a transmission fault located either at the transmitting terminal or upstream of the transmitting terminal. [T1.Rpt25-1993]

see: atis.org/tg2k/_remote_alarm_indication.html

remote alarm indication (RAI)

remote alarm indication (RAI): A signal transmitted in the outgoing direction when a terminal determines that it has lost the incoming signal. RAI is commonly called the yellow alarm signal. For superframe (SF) operation, an RAI is transmitted by setting the bit-2 position of every channel time slot to zero. For extended superframe (ESF) operation, an RAI is transmitted by introducing a priority message codeword 1 1 1 1 1 1 1 1 00000000 (LtR) in the ESF datalink. [T1.Rpt25-1993]
On 31-d