Bindaddr in sip.conf not working

I have these in the ‘general’ section of
my box.

port = 5060 ; Port to bind to (SIP is 5060)
bindaddr = ; Address to bind to (all addresses on machine)

I have 3 IP Addresses configured before
I start asterisk. I am only able to make
sip connections to the ‘primary’ address
of the box. Connections to the other
IP Addresses are refused.

If I use sipsak to test, I do:
sipsak -T -s sip:nobody@pbx1
and it connects and tells me that that is an incorrect user.
If I use:
sipsak -T -s sip:nobody@pbx2
I get a timeout even though pbx2
has been setup and I can ssh pbx2
and login… it seems to be only SIP
and Asterisk that have the problem
connecting to the 2nd and 3rd IP

Thanks - jack