Each of your sub-systems will have it’s own challenges.
For your SIP to SIP calls, I would suggest something more along the lines of OpenSIPs or Kamailio (both a fork of OpenSER). Any of these running on decent hardware will be able to register 10s of thousands of users and setup hundreds (or thousands) of calls per second. They both do a much better job of NAT traversal than asterisk. The only hiccup would be if you needed asterisk to stay in the media path for call recording (or network issues). There is a media proxy component, but that would defeat the purpose as you would no longer have such a scalable SIP proxy if it was forced to process media.
Generally most people can get away with running a single OpenSER server with LinuxHA (heartbeat) for high-availability and failover.
OpenSER and asterisk are not mutually exclusive. You would still want to forward calls off to an asterisk server for voicemail purposes, and obviously your asterisk systems would still handle the FAX processing.
Once you insert OpenSER into your network, you have MUCH more control over where your calls terminate and how they reach their destination.