is there any way through which I can remove all the agents from queue
Just edit queues.conf ?
Or if they are added dynamicly you can use either AMI or RemoveQueueMember cli command.
I wrote a little subroutine that does this using the QUEUE_MEMBER_LIST function, the SHIFT function and the RemoveQueueMember Application.
It works pretty well, I pass in a queue name as an argument and my routine logs each agent out one at a time and gives me audible feedback as to what agent was logged out.