Lots of short sound files kills asterisk

Is there a reason that lots of short sound files played consecutively would cause asterisk to crawl and ultimately stop answering calls? If I set a 5 second file, followed by a 10 second silence file and repeat, the cpu for about 150 concurrent calls sits at around 99%. If I play both once and then leave the call empty, then CPU sits at about 20%. Longer audio files also use less CPU. But why? It’s not exactly a lot of calls