Direct client communication with 2 Asterisk server

Hello and sorry for my bad english. I hope you understand me.

I have two asterisk server in a local network. Each of them have a sip-client. If i make a call from one phone to the other the communication is as follow: the sip clients build a sip and rtp connection to the asterisk-server which they are connected. The asterisk-server communicate via iax2.
But i want that the asterisk server initialize the call and then i want to have a direct rtp-connection. If i have only one asterisk server then it is ok. But how is that possible with two servers? I tried canreinvite, but nothing happens.

Thank you for your help