Upgrading to Asterisk 13 - Strange Music On Hold Issue - Banging my head on this one

I posted this to asterisk-users last night also (but it’s currently awaiting moderation evidently).

So I’ve been banging my head against the rack on this one and am now turning to the group for help.

I’m in the process of bringing five Asterisk servers (all originally built from source code by myself) from various versions (1.6.2.x,11.6-cert13, and 13.1-cert2) up to the latest Asterisk 13.8-cert3. This has happened across multiple operating systems and compiler versions: an old CentOS 5.9 server (GCC 4.1.2) that is running 11.6-cert13 just fine and wanting to do an in place upgrade to 13.8-cert3, an Ubuntu 14.04.5 LTS (GCC 4.8.4) that was built from scratch to replace an old server, and multiple Ubuntu 16.04.1 LTS (GCC 4.8, 4.9 and 5.4.0) built from scratch to replace old servers. The servers themselves range from old to new, fast and faster, Dell and Supermicro, production and lab. With one exception, I have had the same Music On Hold problem on each one of the servers.

The problem is this: when a caller is placed on hold or parked, initially the music on hold 100% of the time is garbled. It sounds like what it did back in Asterisk 1.2 when zaptel/dahdi was screwed up and timing was off. Within 5-20 seconds it will clear up. Once its cleared up, you can take that caller off hold and put them back on hold, or un-park and re-park them as many times as you want and the Music On Hold will not glitch again.

I’ve tried timing with res_timing_dahdi and res_timing_timerfd. I’ve tried quietmp3 and files (wav, gsm, ulaw, sln, mp3). I’ve tried different codec: ULAW and G729. There is absolutely no load on any of these machines when I’m testing… just the one phone call that I’m testing on. I’ve tested with various SIP endpoints (Polycom SoundPointIP 550, Polycom VVX 500 and 600, Zoiper, Linphone). It doesn’t matter if its two SIP endpoints or a SIP endpoint to PSTN (via SIP of course), the behavior is always the same. Absolutely nothing I’ve done has made a dent in the problem.

The only thing that is consistent is I don’t have the problem on 11.6-cert13 or -cert15 (regardless of OS or processor). I have not tried Asterisk 12 to see if the issue is present. I have tried compiling 13.8-cert3 with GCC 4.8 and 4.9 and no change there either. I’ve also tested Asterisk 14.1.1 and the behavior is the same This behavior exists already on the in production 13.1-cert2 server, but doesn’t impact anything because nothing ever uses Music On Hold on that box.

The two exceptions are a new Supermicro server and a Dell server that are not having the problem at all running Asterisk 13.8-cert3. The Supermicro is running Ubuntu 16.04.1 LTS (GCC 5.4.0) and the Dell is running Ubuntu 14.04.5 LTS (GCC 4.8.4). Using res_timing_timerfd and no dahdi kernel modules loaded, Music On Hold is pristine. The only thing I can see is that both of these machines have Xeon E5 processors.

For anyone interested, my build instructions/script is posted at the link below.

Below are the processors running in the five servers. This is the only difference I can really see in the machines. If it’s a CPU issue, if I knew what specific feature was causing it work I’d just replace the three problem children with servers that knowingly fixed the problem.

Working fine on these processors:
Intel Xeon CPU E5-2650 @ 2.00Ghz (DELL PowerEdge R620, Currently Ubuntu 14.04.5 LTS)
Intel Xeon CPU E5-2637 v4 @ 3.50Ghz (Supermicro, Currently Ubuntu 16.04.1 LTS)

Not working on these processors:
Intel Atom CPU 330 @ 1.60Ghz (Supermicro, Currently CentOS 5.9, retiring but not until I figure this out!)
Intel Xeon CPU X3470 @ 2.93Ghz (Supermicro Currently Ubuntu 16.04.1 LTS)
Intel Xeon CPU E3-1230 v2 @ 3.30Ghz (DELL PowerEdge R310, Currently Ubuntu 16.04.1 LTS)

I feel like I’m spinning my wheels on this one. Any help would be GREATLY appreciated!!!

– Mitch Sharp (bluecrow76)

That’s a pretty unusual problem, did you install all of the moh files for each codec?

Most of the customers are using custom provided on hold music that was originally delivered as MP3. I’m using mpg123 on all the existing servers. As part of testing I have converted the MP3 files to all the various formats with the results being the same. Garbled MOH on all but the Xeon E5 processors.

Unusual is right… especially when I’ve scoured the Internet looking for others that have had similar problems with Asterisk 13 and finding none. I’ve got myself wondering what I’m doing wrong and can’t figure it out. I feel a bit derpy not being able to figure this out after working with Asterisk for over 12 years. :slight_smile: