[HELP] Asterisk Design Question - SMB / 2 Offices / Failover


#1

Hello everyone! This is my first time posting to the forums and I thank you all in advance for any help you can provide in regards to this.

My main question is in regards to survivability of an Asterisk system. Some background on the project is probably in order.

My employer currently has two offices located about 1 mile apart. They are connected with a 100mbps fiber link. At each office, there are approximately 40 users and 4 PSTN lines (so total 80 users + 8 PSTN lines). Currently, they have a Toshiba Strata CTX670 at one office and an ancient Comdial PBX at the other. The systems do not speak any VoIP protocols (although the Toshiba will support Toshiba VoIP phones with a costly upgrade and licensing fee). Additionally, the systems are not integrated at all so to dial between offices you need to dial the PSTN number of the other office and come in just like any other caller.

I am actually the IT manager so this is my first foray into the wonderful world of telecom. I’ve been working with our resident telecom administrator to come up with a good plan on getting an integrated system in place. After looking at several alternatives and options, we happened upon Asterisk and will probably end up at least doing a proof of concept system to test if this design will work.

Basically, what we are thinking is buying 80 Grandstream GXP-2000 phones (40 for each office), 2 FXO-to-VoIP gateways, and 2 Dell Poweredge boxes. We’ll have 1 gateway and 1 Poweredge at each office. What we’d like to accomplish is that if we lost any single component, the system will still work. Since there is high-speed connectivity between the offices, it should be possible.

My idea was having the GXP-2000 phones all register to both Asterisk servers as well as having the FXO/VoIP gateways register to both servers. As long as we can get Asterisk out of the media-path by keeping the audio codecs the same, the actual voice data should be going directly from the gateway to the end-user’s handset. Obviously, if we lose a gateway, then all calls going through that gateway would be cut off and if we lose a server, all calls currently being signaled by that server would also be cut off. But the end-user should be able to hang up and pick the other “Line” (actually the other Asterisk server) on the GXP-2000 and use that to dial out.

Has anyone been able to get something similiar to the above working in a production environment or know if it is possible with Asterisk? Any suggestions on the VoIP gateways we should look at for this kind of functionality (the D-Link ones look alright but it seems they’re vaporware at the moment)? I’ve scoured the wiki looking for information on load-balancing and fail-over but none of the things I was able to find address a distributed fail-over system like I’ve described above.

The only other question I have to pose to some of the guru’s here is with regards to voicemail. In any fail-over scenario, how do you handle voicemail during that time? Obviously, you can have the configuration for the voicemail in a database but then you need a couple more servers for redundancy of your database (unless you throw MySQL on the Asterisk box). But how do you handle the actual stored voicemails? I’m thinking some kind of sync process that shuffles VM files between the two so if you lose a server, the user can still access voicemail and incoming callers can still leave the person voicemail.

Thanks in advance for any help, advice, insights you can give on the above questions. If you need more info, I’ve got this thread tagged to notify me so I’ll post back right away whatever you ask for.


#2

Given your high speed link between the offices, I would be tempted to go with a single high availability server at one site. * and Linux have proven to be highly reliable, and I would invest my money in making the server robust, (raid, redundent PSU’s etc) 80 users is not a lot for * to handle.

With regard to the phones, buy a few to trial with first. There are a number of issues with regard to business use with these phones, and in particular reported failure rates of ~ 15%. You might like to have a look at the Aastra and snom range. I have found the GP’s very sensitive on the microphone, where, in a busy office, users have complained of too much background conversation being picked up by the phones.

rgds

P.S. Welcome to the wonderful world of * where the learning curve is steep but rewarding


#3

davidarcher,

What a great topic!

You suggested an interesting outbound fail-over. The problem I see is inbound fail-over, or did I miss that?

I suspect you will want ONE production server, serving all the buildings, with a spare ready for action.

You might look into a good VoIP IAX2 provider to augment your solution, I have had good service with teliax.com. Their PAYGO plan with IAX2 allows multiple incoming calls with one DID. Try outbound service first, with little service risk.

You might be able to eliminate half of your POTS lines, replace them with VoIP for outbound. Of course this requires a rock-solid IP connection.

Have you decided how to handle any FAX requirements?

Lonnie


#4

middletn,

Although we have looked at the single HA server route (with remote extensions at the other building), we still have the issue that if the link were to go down, all phone service at the other building would be down until it was restored. Having a couple * servers seems like it would help since I’d at least have intra-office and PSTN access at each office if the link went down.

We are planning on getting a couple of GXP-2000’s and doing some testing before proceeding with a full purchase. I’ll have to look into those other models too if we run into issues with the Grandstreams.

lonnie,

Actually, the plan for inbound fail-over would be to have a spare FXO gateway available and configured so we could just plug it in and go in case we did have a failure of one of the gateways.

See my above note to middletn regarding the single server solution as it gives less than optimal failover in case the link goes down.

As far as using an internet VoIP provider, we have an issue with security (as we’re not really a SMB having been recently bought out by a large corporation last year. We’re just a branch office with a lot of autonomy). I doubt any internet VoIP traffic is going to be doable in line with our current corporate security policy (we’re a military R&D contractor and security is a pretty high priority). We may be looking at at least having an ITSP (is that the correct acronym? Internet Telephone Service Provider?) for an outgoing PSTN backup in case all 8 lines are down/busy. The only issue with that is our only internet connection is actually over a T-1 WAN link to our corporate HQ and out their internet connection, which is shared with 9 other branch offices. Incoming VoIP, however, is definately out as we’d have to have holes poked in the firewall and then forwarded back to us or else co-locate hardware up at the corporate HQ.

Fax requirements are going to be handled how they currently are. There is a direct POTS line out for the fax machine which also gives us an emergency back-up in case we lose the PBX.

So it seems you both recommend a single-server approach. Is it not doable in * to have users/gateways register to dual servers? I would think you just need your dial-plan to ring both servers whenever calling a user’s extension (i.e. Dial(SIP/UserA@server1&SIP/UserA@server2) - I know the syntax is wrong but hopefully you get the idea). Sharing the outgoing gateways should also be fairly easy as long as you have some way of knowing whether a certain port on the gateway is in use or not (I saw an outgoing hunt group macro on the wiki that might do the trick).

The other consideration is that I’d like to minimize the traffic over the fiber link if at all possible. We already have all our servers at one building so those 40 people are already sharing a 100mbps link just to hit the servers for files, e-mail, internet, etc. I would think by having PSTN gateways at each office, you can prioritize the servers such that they’ll dial out the gateway that is physically located closest to the user (hard-coding is fine since the users won’t be moving their phones too much and with only 80 total, it’s not that much maintenance) which should cause the voice data to just go over the local office LAN instead of the fiber link.

Great replies, thanks for the tips… Keep em coming! :smile:


#5

[quote=“davidarcher”]As far as using an internet VoIP provider, we have an issue with security (as we’re not really a SMB having been recently bought out by a large corporation last year. We’re just a branch office with a lot of autonomy). I doubt any internet VoIP traffic is going to be doable in line with our current corporate security policy (we’re a military R&D contractor and security is a pretty high priority). We may be looking at at least having an ITSP (is that the correct acronym? Internet Telephone Service Provider?) for an outgoing PSTN backup in case all 8 lines are down/busy. The only issue with that is our only internet connection is actually over a T-1 WAN link to our corporate HQ and out their internet connection, which is shared with 9 other branch offices. Incoming VoIP, however, is definately out as we’d have to have holes poked in the firewall and then forwarded back to us or else co-locate hardware up at the corporate HQ.[/quote]Is POTS more “secure” than VoIP? I would think POTS would be easier to ‘listen’ to, but that is another topic. As far as firewall concerns, if you use IAX2 only UDP port 4569 needs to be forwarded, very easy compared to SIP. QoS would be an issue if you only have a T1 for all your IP internet traffic.

This might be a case for VLANs and a secondary (local) IP internet connection for VoIP services.

[quote=“davidarcher”]So it seems you both recommend a single-server approach. …
The other consideration is that I’d like to minimize the traffic over the fiber link if at all possible. …
which should cause the voice data to just go over the local office LAN instead of the fiber link.[/quote]If your SIP phones are set to “canreinvite=yes” in sip.conf the fiber traffic would be almost the same if you had one or two servers. Voicemail transactions would be an exception. One server would be much easier to manage.

If you get this all to work, you should write a book.

Lonnie


#6

[quote]Is POTS more “secure” than VoIP? I would think POTS would be easier to ‘listen’ to, but that is another topic. As far as firewall concerns, if you use IAX2 only UDP port 4569 needs to be forwarded, very easy compared to SIP. QoS would be an issue if you only have a T1 for all your IP internet traffic.

This might be a case for VLANs and a secondary (local) IP internet connection for VoIP services. [/quote]

Well, I don’t think it’s more secure but I may have some trouble convincing the powers-that-be to allow us to move to an IP-based PSTN trunk instead of POTS lines. In the meantime, I’m just concentrating on getting our internal phone system overhauled.

Alright, I get what you’re saying. If I have one server and 2 gateways, I can still mostly keep the voice data going over the local office LAN. Then it all comes down to the survivability factor.

I totally agree that a single server would be easier to manage but my interest is if * is flexible enough for us to have a system that would allow a server to completely die (or a gateway or even the fiber link between the offices) and still have phone service available to the users.

I’ve been doing some more research tonight and this paper seems interesting: http://www.cs.columbia.edu/techreports/cucs-011-04.pdf

They discuss some SIP failover scenarios that may work. The GXP-2000’s do support “SIP Server Failover” and “DNS SRV lookups” which may do everything I need for a client failover configuration (of course, i’d have to tell the clients to register with both * servers). The problem I can see here is that I may add some FXS-to-VoIP gateways (i.e. Analog Telephone Adapters) in the future and would be reliant on getting ATA’s that support dual server registration, which doesn’t seem likely.

The other option, which may be better, is to use a MySQL replication scheme to allow the * servers to share a common set of SIP client registration records. Then I can just tell the phones to register once with “mydomain.com” and set my DNS SRV records up so they contain both * servers. My DNS is running on Win2k3 so it should be possible to make it give the local * server priority so clients will attempt to register with the local * server first.

The only issue with the MySQL replication is the added complexity (which is to be expected with a redundant solution) as well as the possibility of additional hardware needed. Is it alright to throw MySQL on the same box as Asterisk or are there issues with doing that? I imagine that with 40 to 80 users, the load on a modern server should be fine for running both * and MySQL (given that I’ll be doing as little codec conversion at possible and trying to keep * out of the audio stream as much as possible).

Believe me, if I get all this to work, I’m going to take a vacation. Then, I’ll write the book.

:smile:


#7

I’ve been eating, sleeping and even dreaming Asterisk failover and redundancy for the last several weeks. That’s an intesting PDF document, at least for me, because I’ve been over all those suggested HA implementations. It’s a shame that just about everything it sugests for HA, Asterisk doesn’t natively support.

Your in for one HELL of a struggle to get this to work!


#8

Well, I just had an epiphany after reading that PDF and thinking some more about it. Basically, the only thing I really need to share between the two * servers is the SIP registrations so I can find extension 123 when I dial it into my phone. Then I got to thinking about the problem and realized I could just give all my phones static IP addresses and not worry about registering them with Asterisk. * would already know how to contact any of the extensions. So then really I just need Asterisk to do some proxying for my SIP phones to talk to my SIP-to-FXO gateways.

Voicemail replication/failover is still going to be tricky but I may be able to get away with just having it on one of the servers (the one at our “main” office where the receptionist answers all the incoming lines). The remote * server will probably need to be set up with voicemail but since it’ll only be used in case of a failure, we can just shoot that through e-mail and wipe it off the server. No need for the user to ever access it through the phone.

When/If we start getting dynamic clients, then it’s going to be fun. I believe the expectations (from the end-user’s perspective) of using the PBX remotely using a soft-phone or even a hard-phone over our VPN is going to be a little lower than the expectations of a user picking up their handset in the office. So having those just register with a single * server should be fine (as long as I can get the “remote” * server to be able to find them).

So what exactly have you found out in your several weeks of failover/redundancy? I’ve seen some crazy postings on having gobs and gobs of servers (failover SIP proxies, load balanced * servers, replicated databases, etc.) but nothing really for a SMB that just wants some redundancy in their system.

Also, I really wonder why there isn’t more information out there on FXO gateway boxes. I know Digium would like you to buy their cards but it really seems stupid to “put all your eggs in one basket”. So I either have to buy cards for each server or else just buy external gateways that will register with * to handle the PSTN calls. It seems like a no-brainer at least for FXO lines (for T-1/E-1 lines, the external gateways seem to be a BIT more expensive than the Digium cards - $5k versus $500 for a single T-1). Any recommendations on external FXO gateways that work well with *?

Well, I love a challenge. :smile: I’m comfortable with Linux and * seems to be a fairly robust system. I just wonder why there isn’t more documentation on real HA solutions or simple, elegant fail-over solutions that don’t take 8 servers or some esoteric hardware to implement.