Packet lost before reaching the interface

I have a weird issue - My asterisks are loosing packets between asterisk and the interface of the host it is running on.
In details, I see answers to invite (100 trying) being generated in the sip set debug, yet, a sngrep/ngrep/tcpdump does not show these packets. and the remote eventually issues a 408.
While this happens, the console log (that runs crazy) is fine and uninterrupted, And even weirder, the audio is fine ! I
The twist is that those asterisk are running on VMWare instances. t does not seem related to load on the vms.
This happens with Asterisk 11.13, 13.14, chan_sip and pjsip !!!

Any suggestions will be gladly welcomed !