Scalability question on potential large scale ACD deployment

We have 850+ call center agents spread over 6 offices nationwide. 2 of those offices are actually 3office campuses, all tied back to the ‘main’ office on the campus.

We’re on Nortel TDM PBX systems at each site. We’re following an initiative from our parent company to deploy Cisco IP Telephony and basically forklift the Nortel systems. I’ve deployeed a testbed of IP phones for several months now and everything seems to be running fine. I haven’t gotten the Unity servers online yet (not because I’m unable to get them working, haven’t even unboxed them yet).

I’m not at all happy with Cisco’s IPCC ACD platform. Not happy with the performance. Not happy with the hefty price tag. I’ve been trying to find an alternative ACD system with little luck. It has to integrate with Cisco Call Managers. It has to support SIP and SCCP. SIP because I have a gut we’ll want to use SIP proxies for the redundant servers. SCCP because I know that the powers that be will want the same firmware for the non-ACD Cisco phones as for the Cisco agent phones. I’d like to be prepared either way. By “integrate with Cisco” I mean we’ll want to connect both systems with a combination of H.323 and PRI gateways. The agent phones will register to the * servers.

So I’m looking for some feedback on Asterisk’s scalability. How many production agents can I put on each * server? I understand that it’s a vague question. I’ve seen quotes of 25-50 agents depending on call volume. It’s got to be more than that. Server hardware platform plays a big part of that as well, and we’ll have solid servers.

I’ve also read that some * users are having to restart * nightly as well as restart the servers weekly. Is * that unstable?

I haven’t read through all of the case studies on Digium’s site yet, but I’ll be spending the next 30mins or so reading through those. * is the only system that I could find that meets the criteria given to me though, and I’ll be pitching it to the powers that be this afternoon to get a couple of test servers going in an ACD test environment.

I’d appreciate any feedback on the number of agents an * server can reliably handle (assuming kicka$$ server hardware), as well as any success stories of large call-centers on *.

thanks,
Will

I have never used the Asterisk implementation of SCCP. You will want to test that out before you get too far down the path on that particular requirements, as I have not seen many success stories on that particular point. If SIP on the Cisco phones is viable, I would definately go this direction.

A question with lots of variables as this will be highly dependent on your contact center needs. There are various ways to create scale within the Asterisk platform, depending on what you require. Some design principles to keep in mind:

  • Specialize your Asterisk boxes, for example one for voicemail, one for IVR, recording, more for ACDs, etc
  • Keep transcoding to a minimum, for example keep all of your audio formats in one codec such as GSM, or whatever codec you may interconnect with to the outside world (ie - G711/G729/etc)
  • etc

One consideration you will need to consider is how you maintain a global visibility of your ACD. As best I know, each Asterisk box has its own instance of ACD queues, and the statistics within these are not necessarily capable of being viewed across all instance of Asterisk. Therefore you may need to write/acquire your own ACD mechansim that sits on top of Asterisk if you want enterprise wide routing based on ACD statistics. There are various ways to architect this, so you must take this into consideration.

This seems to be a problem that is more acute when Asterisk is running hardware for PSTN interconnection and not so much when it is a pure softswitch. It is possible to use Asterisk in conjunction with channel banks to minimize some of these issues. But highly dependent on what your network infrastructure will look like.

Thank you very much for the quick response.

I’m sure that skinny would be preferred by the big wigs for standardization, but if the SIP functionality is more stable/reliable then that’s definitely a viable option.

I was planning on testing the * vmail as well but under the radar initially. We already have the Unity hardware and software but our parent company purchased it all and it all can be redeployed. I don’t even want to think about that yet though.

We don’t have remote or vitual agents yet. Our call center groups are used to standard good old fashioned local agents logging into their queues. We introduced skills-based routing to them a couple of years ago and they’re still working on fully-understanding and using that functionality. Each office will be separate from that standpoint, but it might all be hosted here depending on how that testing goes.

There is a desire on a management level to have unified reporting – pull one report that accumulates stats from all offices (i.e. all call center apps). That’s something that our db guys and programmers can handle though.

Brilliant.

thanks again,
Will

Sometimes the best way to start with Asterisk is not to make it the hub of your contact center but an adjunct. So a voicemail server, or a recording server, or an enhanced IVR, etc. Then, as confidence gains you may slowly add new functions until it becomes the central point of your voice network.

This will make things simpler for an initial deployment. If each contact center is discreet, then you do not need to worry about enterprise routing. I am presuming that you are doing some type of network allocation of 800#s between the centers without statistical access to the premise (or are they using the old Geotel capability of Cisco for network allocation?).

Your only concern now would be the ability to have one Asterisk per discreet center (not counting failover servers).

You might have a look at this if you have not already:

queuemetrics.loway.it/download.jsp

Not sure about ´multi-site´ reporting, but gives you a good start.

I’m being REALLY proactive this time around, so there’s seriously plenty of time to completely put * through its paces. If it doesn’t perform in the test environment, then I won’t get the sign off to put it into production.

Tentatively speaking, after a successful test I’ll put our internal help desk on it. It’s a small call center group and the agents are reasonably technical folk so it should be an excellent test bed. From that point each additional group will be migrated one at a time. When I’ve finished with our main office, I’ll roll it out to the remote sites.

Our toll-free carrier has a platform that allows for the creation of multiple alt. routes for quick re-routing for DRP scenarios. We periodically will transfer calls (via the carrier…completely off-net so to speak) to other offices in the event of a bldg evac, weather closing, etc. Other than they’re all treated as stand-alone offices which definitely makes things easier.

Thanks for the link. I’ll take a look.

Well I’ve got the green light to proceed with setting up the test environment which is great. I’ve submitted the open-source application usage request to legal and with any luck will be setting up the server within a few days.

I have one last question before I get started. There’s a ‘supported hardware’ page on asterisk.org, but there’s really not a lot of hardware listed. From browsing through this forum, I see mention of BrookTrout and Dialogic boards as well but with no mention on the asterisk website.

Is there a definitive list of all known compatible hardware out there somewhere??

thanks,
Will

If you want to stay with GPL Asterisk then you will want to use a Digium or Sangoma T1 cad. If you choose Dialogic you will have to get a Asterisk-Business-Edition license and if you want to alter that code you will need another license from Digium to do that.

As for your call center, we are currently using VICIDIAL(astguiclient.sourceforge.net/vicidial.html) for inbound and outbound call center functions in our 200 seats across 4 locations. It works great for us, and the development version we are using has load-balancing call distribution across multiple servers so we have 100 agents across 4 servers able to take calls in turn from any campaign. There are lots of stats available and it works on top of any version of asterisk(no Skinny support though)

And on that note I would strongly recommend using SIP over Skinny. You will be limited in the future if you stick with Skinny for both interoperability and features, as well as having limited choice when you decide to do things like remote users and external call load balancing in the future.