This is just an idea. Does it make sence to use Asterisk to start and stop services running on a server or scripts performing certain tasks like starting the backup procedure or start and stop the ssh deamon.
exten => start_ssh,1,System(/etc/init.d/sshd start)
exten => start_ssh,1,System(/etc/init.d/sshd stop)
You can start the sshd when you need it and stop it when you are finished. What is more save then a server that only gives remote access when this is enabled by someone with the authority to do this based on numbermatching of the inbound call and disable it when done. This is just an example.
The idea is to add an asterisk server with number matching and a proper voicemenu and an iax2 trunk to control the behaviour of the server and the services running by calling in with a number that is recognized by the Asterisk server.