Hi, @lardconcepts and anyone else that is interested, I got what I need with EAGI using python script.
For every incoming call the (E)AGI app executes a python script in a different process from which you can control the sequence of the call with AGI commands.
See this for what it is capable of:
https://wiki.asterisk.org/wiki/display/AST/Asterisk+15+AGI+Commands
The E in EAGI enable you to read the incoming audio data of the call from a file descriptor which you can do with infinite loop as long as the call goes on and do whatever you like with the audio data. I am planning to feed it to our system.
In addition, its pretty easy (available also in AGI) to stream back to the call your own audio - either from file/Asterisk Playback/audio returned from another system/etc.
I used https://github.com/rdegges/asterisk-python package as Asterisk python API and got some inspiration for audio streaming processing code from both of those:
GitHub - ederwander/Asterisk-Google-Speech-Recognition
GitHub - phatjmo/eagi_lex: simple EAGI script that interacts with AWS Lex.
Although I am doing something else with the audio data itself.
I hope that helps
Danny