Random error with AGI "GET DATA"

What could cause the AGI “GET DATA” to return “200 result=-1”? Is there a way to find the cause?

I have ran into an issue with a server that has been in production for about 3.5 years and has no configuration changes in the last 2 years.

Here is the section that is now randomly failing:

  -- <SIP/172.24.177.131-088bb4c8> Playing 'dialtone' (language 'en')
  router.php: GET DATA dialtone 10000 returned -1
  router.php: User entered phonenumber: -1
  router.php: Got Dial String, -1
  router.php: Prompting For PIN now
    -- <SIP/172.24.177.131-088bb4c8> Playing 'bong' (language 'en')
  router.php: GET DATA bong 10000 returned -1
  router.php: User entered -1 as PIN in pin loop

Within minutes after that, the user redials and everything works as normal (I exxed out the phone number and pincode):

  -- <SIP/172.24.177.131-088bb4c8> Playing 'dialtone' (language 'en')
  router.php: User entered phonenumber: 1xxxxxx5555
  router.php: Got Dial String, 1xxxxxx5555
  router.php: Prompting For PIN now
    -- <SIP/172.24.177.131-088bb4c8> Playing 'bong' (language 'en')
  router.php: User entered xxxx as PIN in pin loop

Are there any resource related issues that could be causing this? The hard drive space, memory, and cpu look good. The system is close to idling most of the time, averaging only around 4 concurrent calls.

Asterisk 1.4.22
CentOS release 5.2 (Final)

At this point, it looks like the Hard drive read times have gone up and the overall system wait time is high.

From top:
Cpu(s): 1.3%us, 1.3%sy, 0.0%ni, 73.1%id, 24.3%wa, 0.0%hi, 0.0%si, 0.0%st

From iostat -d -x 60 3:
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await svctm %util
sda 0.22 24.77 0.44 7.91 20.22 268.19 34.56 8.87 1060.29 18.07 15.08
sdb 0.01 0.00 0.00 0.00 0.05 0.00 13.78 0.00 2.66 2.04 0.00
dm-0 0.00 0.00 0.64 33.52 20.17 268.19 8.44 43.14 1262.54 4.41 15.08
dm-1 0.00 0.00 0.00 0.00 0.01 0.00 8.00 0.00 7.54 0.73 0.00

Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await svctm %util
sda 0.00 28.12 0.00 4.15 0.00 241.87 58.28 129.34 24798.16 240.97 100.00
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
dm-0 0.00 0.00 0.00 34.12 0.00 272.93 8.00 579.05 15205.19 29.31 100.00
dm-1 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00

Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await svctm %util
sda 0.00 8.98 0.00 5.47 0.00 103.47 18.93 105.19 18345.76 182.93 100.00
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
dm-0 0.00 0.00 0.00 15.05 0.00 120.40 8.00 251.35 14176.87 66.45 100.00
dm-1 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00

AGI Rx << GET DATA dialtone 10000
AGI Tx >> 200 result=-1
AGI Rx << VERBOSE “GET DATA dialtone 10000 returned -1” 1
AGI Tx >> 200 result=1
AGI Rx << VERBOSE “User entered phonenumber: -1” 1
AGI Tx >> 200 result=1
AGI Rx << VERBOSE “Got Dial String, -1” 1
AGI Tx >> 200 result=1n
AGI Rx << VERBOSE “Prompting For PIN now” 1
AGI Tx >> 200 result=1
AGI Rx << GET DATA bong 10000
AGI Tx >> 200 result=-1
AGI Rx << VERBOSE “GET DATA bong 10000 returned -1” 1
AGI Tx >> 200 result=1