We have finally been able to run RTP debug logging during an affected call, and identified the issue.
The issue is that both Asterisk and our provider’s media server appear to switch codecs when they receive a packet using a different codec from the other side. Because this is happening on both ends, some calls get into an infinite loop of switching codecs.
As a temporary/interim measure, we have changed our allowed codec list to only PCMA, so that there is no room for confusion. This has successfully resolved the extreme audio quality issues described in this post. The ast_dsp_silence_noise_with_energy log described in the original post was also a symptom of the same problem, we aren’t seeing it any more.
However, going forward, we would still want to use the preferred codec indicated by our provider for a given call where possible. I have opened a separate topic asking for advice on the best way to implement this while still avoiding the infinite codec switching issue: Infinite codec switching
Hopefully this can help someone else experiencing similar issues in the future