Iaxclient no voice problem

when my iaxclients connect voice channel through asterisk server, it works fine.but several mins later all the clients will hear no voice. this lasts about 5 seconds then it become ok now. this strange will happen every several minutes. what 's wrong? it is a client issue or a server issue?
any reply will be greate appreciated!