Asterisk and faxes is kind of permanent topic. I suppose if you provide a more detailed explanation of your setup people here might be able to help you. I’m sure anyone would need to know how do you connect those fax machines, what routes aren’t working (for example telco->pbx1->pbx2->LAN->ATA->fax or pbx1->pbx2->FXS->fax).
Yes, you can choose codecs with IAX pretty much the same way as with SIP. I suppose in PBX, Asterisk, IAX, Asterisk, PBX you mean FAX, not PBX, as asterisk itself is a PBX. Correct me if I’m wrong. Fax over VoIP, which is not fax over IP, is a very challenging and not so reliable. I have personally tried a number of times to get this working - in best case scenario it works “most of the times”. For a reliable fax transmission you’re gonna need T.38 support. I’m not sure if IAX has it.
On a second thought re-reading your answer I suppose I got it wrong. Sorry about this. I guess you’re just using the asterisk boxes as a bridge between your PBXs, replacing the leased line. Try using SIP and enable t38passthrough.