Audio Choppiness Both I/O

Hello,
For a while now I have been getting choppy audio both during recording and playback. Im looking for any ideas or suggestions. I cant for the life of me figure this out. Ill edit the post with more as I think of it.

Heres all the information ive collected:

  • Quad Core CPU with 64GB RAM with 1GB ETH (Debian GNU/Linux 9.5 (stretch))
  • Calls peak around 100.
  • Load average is about 1.5 with ~100 calls and < 1 normally as calls average around ~60.
  • I have tried 4 different versions of asterisk now with the same problems
  • Current version: Asterisk certified/13.21-cert2
  • I have performed hours of RAM checks to see if any errors occured. There were none.
  • I have been monitoring the disk (8TB Helium) and have had no errors.
  • I have 16GB of Swap assigned but its not being used.
  • This issue happens on both VoiP and PRI (using both in my system)
  • Ive tried all 3 different timing modules and am currently using res_timing_timerfd.so
  • I have dramatically reduced the number of modules loaded (see below for list)
  • Using ulaw codec.
# vmstat
procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
 2  0      0 590212 1744836 27748772    0    0   213   255    2    0  9  9 81  1  0

Current Timing Module

Module                         Description                              Use Count  Status      Support Level
res_timing_timerfd.so          Timerfd Timing Interface                 161        Running              core
1 modules loaded

Currently loaded modules:

Module                         Description                              Use Count  Status      Support Level
app_chanspy.so                 Listen to the audio of an active channel 0          Running              core
app_confbridge.so              Conference Bridge Application            40         Running              core
app_controlplayback.so         Control Playback Application             0          Running              core
app_dial.so                    Dialing Application                      0          Running              core
app_directory.so               Extension Directory                      0          Running              core
app_echo.so                    Simple Echo Application                  0          Running              core
app_macro.so                   Extension Macros                         0          Running              core
app_mixmonitor.so              Mixed Audio Monitoring Application       13         Running              core
app_page.so                    Page Multiple Phones                     0          Running              core
app_playback.so                Sound File Playback Application          4          Running              core
app_senddtmf.so                Send DTMF digits Application             0          Running              core
app_stack.so                   Dialplan subroutines (Gosub, Return, etc 0          Running              core
app_transfer.so                Transfers a caller to another extension  0          Running              core
app_verbose.so                 Send verbose output                      0          Running              core
bridge_builtin_features.so     Built in bridging features               1          Running              core
bridge_simple.so               Simple two channel bridging module       0          Running              core
bridge_softmix.so              Multi-party software based channel mixin 18         Running              core
chan_dahdi.so                  DAHDI Telephony w/PRI                    6          Running              core
chan_sip.so                    Session Initiation Protocol (SIP)        51         Running              core
codec_gsm.so                   GSM Coder/Decoder                        0          Running              core
codec_ulaw.so                  mu-Law Coder/Decoder                     316        Running              core
format_gsm.so                  Raw GSM data                             0          Running              core
format_pcm.so                  Raw/Sun uLaw/ALaw 8KHz (PCM,PCMA,AU), G. 0          Running              core
format_wav.so                  Microsoft WAV/WAV16 format (8kHz/16kHz S 40         Running              core
func_callerid.so               Party ID related dialplan functions (Cal 0          Running              core
func_channel.so                Channel information dialplan functions   0          Running              core
func_dialplan.so               Dialplan Context/Extension/Priority Chec 0          Running              core
func_periodic_hook.so          Periodic dialplan hooks.                 1          Running              core
func_volume.so                 Technology independent volume control    0          Running              core
pbx_config.so                  Text Extension Configuration             0          Running              core
pbx_spool.so                   Outgoing Spool Support                   0          Running              core
res_agi.so                     Asterisk Gateway Interface (AGI)         59         Running              core
res_ari.so                     Asterisk RESTful Interface               0          Running              core
res_crypto.so                  Cryptographic Digital Signatures         1          Running              core
res_fax.so                     Generic FAX Applications                 1          Running              core
res_fax_spandsp.so             Spandsp G.711 and T.38 FAX Technologies  0          Running          extended
res_monitor.so                 Call Monitoring Resource                 1          Running              core
res_musiconhold.so             Music On Hold Resource                   12         Running              core
res_rtp_asterisk.so            Asterisk RTP Stack                       52         Running              core
res_smdi.so                    Simplified Message Desk Interface (SMDI) 1          Running              core
res_sorcery_config.so          Sorcery Configuration File Object Wizard 0          Running              core
res_speech.so                  Generic Speech Recognition API           0          Running              core
res_stasis.so                  Stasis application support               0          Running              core
res_timing_timerfd.so          Timerfd Timing Interface                 173        Running              core

SIP.CONF

[general]
disallow=all
allow=ulaw
jblog=no
jbenable=yes
jbforce=yes
jbimpl=adaptive
context=inbound
dtmfmode=auto
canreinvite=yes
faxdetect=yes
rtpkeepalive=60
srvlookup=yes
directmedia=no
videosupport=no
defaultexpiry=1800
registerattempts=0
session-timers=refuse
nat=force_rport,comedia
insecure=port,invite

DAHDI.CONF

[channels]
context=inbound
switchtype=national
usecallerid=yes
hidecallerid=no
callwaiting=yes
usecallingpres=yes
callwaitingcallerid=yes
threewaycalling=yes
transfer=yes
canpark=yes
cancallforward=yes
callreturn=yes
echocancel=no
echocancelwhenbridged=no
rxgain=0.0
txgain=0.0
group=1
callgroup=1
pickupgroup=1
immediate=no
relaxdtmf=yes
;dtmfmode=auto
;toneduration=300
switchtype=national
signalling=em_w
group=1
channel => 1-24

Something else I noticed:

After restarting Asterisk things seem OK for ~72 hours and then degradation starts to happen and I have the audio problems.

Not sure if that helps generate some ideas or advice.

It suggests some sort of resource leak, so look at memory, in particular, to see if something is hogging RAM. Choppiness might be the result of thrashing the page file.

I’m not seeing any RAM issues. Plenty available and swap is hardly ever used.

Can I bring this theme up again? I have same issue word-to-word. Can’t find any solution.
My specifications are:
Linux 3.10.0-693.17.1.el7.x86_64
Asterisk 13.19.0
Lenovo server, 48 cores, 125G RAM, fast storage. asterisk starts to chop and chew any audio (even while calling IVR from local phones). If I fwconsole restart (FreePBX is my distro for now) - all’s great till some time passes by and for no visible cause it starts to chop and chew again.

Current state: chopping, chewing, dropping some seconds
Current overhead: 176 threads and 56% CPU (according to atop), 20.7 G. RAM

Network is not an issue. For now I have direct copper link between my server and my Yealink. For testing purposes.

Asterisk 13 is no longer supported, and 13.19.0 is not the end of life version of Asterisk 13.

fwconsole is part of FreePBX, which is not supported here. IVRs are a FreePBX abstraction on top of Asterisk.

chan_sip was deprecated from Asterisk 17, isn’t supported by anyone, and will be removed next year.

Are you really dedicating all of this machine just to Asterisk?

As you said your situation was exactly the same, in spite of being a different OS and Asterisk version, I’ll add the following paragraphs, however, basically you should start a new thread, probably on the FreePBX forum, and provide all your evidence. You can then link to this thread, but your new thread should stand without it.

Setting insecure=invite in the general section is a very bad idea. Both nat= and insecure=port are heavily overused.

Using jitter buffers are SIP Is unusual. The phone or PSTN gateway should do this.

Just a note for this old issue of mine (if it matters)…

The choppiness was due to the networking at the provider. Since moving to OVH Cloud I have not had any issues.