The key word here is “sometimes”. If your issue is intermittent, it looks like you have problems on a network level. Please check IP connectivity first.
Apart from that, it is not clear whether all these issues (unreachable trunks, one-way audio, total loss of audio) appear simultaneously or they look independent?
In any case, I recommend you providing packet trace from Asterisk server next time when this issue arises:
tcpdump -w <filename.pcap>.
It may reveal a lot.
If the issue is permanent, in might be caused by incorrect NAT settings on your router. Are you using NAT/PAT? How have you configured it in Asterisk / FreePBX? Have you enabled ALG on the router?