No two way audio SIP

When making outbound calls thru my SIP provider audio is only available for one second after that it is silient.

My SIP provider said he sees two invites coming from my 1.4 server. The first with the public address of the server and the second with my private address.
My server is setup as follows

1.4
Fedora 8
Two ethernet interfaces one with a pub-IP the other with a private-IP
The FW is configured to accept all traffic from my SIP provider.
NAT is not enabled. All my phones are on the same network as the private address.

What possiblely can be wrong?

this is my sip.conf file

[general]
port=5060
bindaddr=0.0.0.0
externip=96.57.13.230
checkmwi=10
disallow=all
allow=ulaw

[VCentrix] ;SIP Trunk
type=friend
host=ip-phone.vcentrix.net

port=5060
context=from-vcentrix
dtmfmode=rfc2833
canreinvite=no
allow=ulaw
qualify=yes