Synchronising Voicemail Folders

Asterisk Asterisk APIs
1

I’m on developing a web based voicemail reader to use with asterisk.

I have a custom FAGI script, that’s triggered from an inbound call extension when a message is left in a persons mailbox, and that all works perfectly, but I can’t seem to find a way to synchronise the folder counts/operations when a user is in the mailbox manipulating messages.

I had considered scanning the spool folders directly, as the app will likely run on the same server as the VM folders are on, but if possible, I’d like to find a way using AMI/ARI/FAGI if at all possible.

If the user deletes a message, or moves a message to another folder, or basically does anything is there a way to get a notification of the operation.

The only think I can see in the AMI is the “MessageWaiting” event, and that’s basically just a standard WMI call with a new/old folder total.

Does anyone know of any ways I’ve missed, while researching this.

2

Although I haven’t used it myself, I suspect the IMAP option would be better for this.

1 Like
3

@david551 you know, I never thought of that actually… good call, worth of some investigation that’s for sure.

The client app is in .NET and I have a very good C# library for accessing IMAP servers.

Nice…

That’s one further avenue to explore.

Any others folks, please do let me know.

4

Hey all, just a quick update:

I suggested the IMAP approach, and didn’t get the approval to do things that way, so using my own resources I started writing a set of libraries to work directly on the filesystem of the server running asterisk.

The first of which you can grab a copy of here:

C# code to parse an asterisk voicemail.conf configuration file in the asterisk telephony toolkit.

The 2 classes in that public gist can be added to any .NET core/5/6/+ code running under linux on the same server as asterisk, and will give you a managed and logical way to read the system voicemail.conf file.

It uses the IniParse library from : https://github.com/rickyah/ini-parser to do the RAW file parsing as my comments in the file explain.

usage is dead simple:

  VoicemailConfig = new VoicemailConfig();
  VoicemailConfig.LoadVoicemailConf(Path.Combine(_ConfigPath, "voicemail.conf"));

then you can access the VM options using

  var myoptionvalue = VoicemailConfig.Options["optionname"];

where “optionname” is any of the keys to be found in the general section.

Note though, only keys present in the conf file will be in the collection, there are no defaults or anything like that.

To get the details of a mail box, simply use the following:

  var myMailbox = VoicemailConfig.FindMailbox("100"); // defaults to "default"
  var myOtherMailbox = VoicemailConfig.FindMailbox("100", "mySection"); // 100@mySection

if the return from “FindMailbox” is null, then the requested mailbox doesn’t exist.

Once I finish the code to deal with the physical recordings and their info, I will also post that.

5

but I can’t seem to find a way to synchronise the folder counts/operations when a user is in the mailbox manipulating messages.

Why do you need to synchronize the call at all? You can use AMI to just get the mailbox count in realtime.

Personally, I think it would be easier to add AMI events for these actions to app_voicemail then to try to jump through all these hoops.

Suppose events like MessageMoved, MessageDeleted, etc. existed. Would that solve your issue?

6

@InterLinked I need to sync it because I’ve been asked to list Voicemails in a mailbox on a web page along with date/time and number the message was left from.

That list also has to match what the user of the mailbox may have changed via the telephone.

So EG: If I phone in, move a message from INBOX to Old, then refresh the web page, the inbox list should show one less and the page should list that there are also message’s in Old and also list those.

Ideally, I’d love to be able to just say “AMI tell me everything I need to know about 100@default”, and get a list of messages, callers, counts etc etc that way I would never have to touch the files or the configs on the file system, but the most I can get from AMI is the MWI indicator along with a new and old count. I can name folders using a FAGI call, so that at least allows me to ask if there are folders other than Old or INBOX, but I have no way of getting a list of folders to know what names to ask for.

Suppose events like MessageMoved, MessageDeleted, etc. existed. Would that solve your issue?

Yup for sure. I would much prefer to do this over the AMI interface, I am building from source, but the scope of the project (IE: my client) won’t allow for me to make custom Asterisk changes, in the long run, I may actually undertake this approach, but not in time to complete the current task.

I was also exploring switching from comedian to the MiniVM system, but the documentation on that is not as abundant beyond a few dialplan pages.

Cheers
Shawty

7

@InterLinked I need to sync it because I’ve been asked to list Voicemails in a mailbox on a web page along with date/time and number the message was left from.

That list also has to match what the user of the mailbox may have changed via the telephone.

So EG: If I phone in, move a message from INBOX to Old, then refresh the web page, the inbox list should show one less and the page should list that there are also message’s in Old and also list those.

That doesn’t explain why you need to sync it.

I also list voicemails on webpages, and I don’t sync any data. When the webpage loads, your server can use AMI to obtain the mailbox account. You don’t need to locally cache and sync the data.

Ideally, I’d love to be able to just say “AMI tell me everything I need to know about 100@default”, and get a list of messages, callers, counts etc etc that way I would never have to touch the files or the configs on the file system, but the most I can get from AMI is the MWI indicator along with a new and old count. I can name folders using a FAGI call, so that at least allows me to ask if there are folders other than Old or INBOX, but I have no way of getting a list of folders to know what names to ask for.

Is there some reason you can’t use these AMI actions for that task?

https://wiki.asterisk.org/wiki/display/AST/Asterisk+18+ManagerAction_MailboxCount

https://wiki.asterisk.org/wiki/display/AST/Asterisk+18+ManagerAction_MailboxStatus

Suppose events like MessageMoved, MessageDeleted, etc. existed. Would that solve your issue?

Yup for sure. I would much prefer to do this over the AMI interface, I am building from source, but the scope of the project (IE: my client) won’t allow for me to make custom Asterisk changes, in the long run, I may actually undertake this approach, but not in time to complete the current task.

If it were useful, I could add them for you, but I’m still not understanding why you need to sync data, rather than use AMI to retrieve on demand, per above.

8

When I open the webpage, it is expected that a LIST of all the voice mails currently in the specified folder be displayed, NOT just the counts.

If the mailbox being queried has 2 folders called “INBOX” and “Saved” with 3 msgs in 1 and 1 in the other then something like:

-------------------------------------------------------------
| Folder: INBOX                                             |
-------------------------------------------------------------
| 1 | From Shawty | 123 | 04/08/22 | 13:00 | [play][delete] |
-------------------------------------------------------------
| 2 | From Sam    | 321 | 04/08/22 | 13:00 | [play][delete] |
-------------------------------------------------------------
| 3 | From Paul   | 400 | 04/08/22 | 13:00 | [play][delete] |
-------------------------------------------------------------
| Folder: Saved                                             |
-------------------------------------------------------------
| 1 | From Shawty | 123 | 04/08/22 | 13:00 | [play][delete] |
-------------------------------------------------------------

If another user has 4 folders EG: INBOX, Old,Important,Tasks

Then the list should show those 4 folders and the messages listed in them.

If a user is not at their desk, but phones in and moves a message in their VM to another folder, then the list should be updated to reflect that move, as their may be more than one person monitoring a specific mailbox.

Now I could hit the server every time a page is rendered, but that could conceivably be several users, and quite a few hits per hour depending on how busy they are.

THUS: Poling that data once, saving it locally on the app server, then getting asterisk to tell me when the changes occur so I can make local updates, is in my opinion the best way to do this.

I cannot however do things that way, so I have to resort to reading files off of the server, which if I’m careful I can keep the number of hits on the server down to a minimum poling say every 5 or 10 mins and mirroring that data on the application server, for fast access by the web application.

If it were useful, I could add them for you, but I’m still not understanding why you need to sync data, rather than use AMI to retrieve on demand, per above.

As I commented previously, you’ve given me food for thought there, I’m not beyond adding them myself, I just don’t have the scope and bandwidth now and in time for this project to do things this way, I may however have time in the future to look at at asterisk code base, add this feature, contribute it back to the community, then in a future update change the project code to use the new functionality.

Thank you for the offer though.

9

Now I could hit the server every time a page is rendered, but that could conceivably be several users, and quite a few hits per hour depending on how busy they are.

THUS: Poling that data once, saving it locally on the app server, then getting asterisk to tell me when the changes occur so I can make local updates, is in my opinion the best way to do this.

Okay, that’s what I was looking for - wasn’t clear from the original question that caching locally was actually a design requirement. That’s not something I do.

The additional context also suggests an updated mailbox status event with new folder counts, as opposed to simply “MessageDeleted” would be more useful for you - do you really care that a message was deleted, or just that the state is different and you want the updated counts?

For such an event, you might not always necessarily need to poll the server again, though certainly to get message timestamps and info still.

I think I understand the use case now, though how that connects with you manually parsing the voicemail config, I’m not sure I see the connection to that… what are you trying to glean with that exactly?

10

Additionally, another thought:

I’m assuming all voicemail access to your system is through the VoiceMailMain application?

You could make your own AMI event by using the h extension or hangup handler in the dialplan, that would emit a “VoiceMailAccessed” custom (user) AMI event when a call access VMB and hangs up.

Then you could sync the data manually upon receiving that. You can do that today with no modifications to anything.

As long as you’re fine with a delay between when a voicemail might be deleted/moved and the user hanging up (which I assume would be fine), this is 99.8% realtime and would do what you wanted, I think?

11

98.9% Realtime is fine, I was discussing earlier with my client what an acceptable delay would be, between data updates, and they stated no more than 15 minutes.

To be fair I really wasn’t expecting to get above 80% :slight_smile:

The use case is quite simply that mailboxes are monitored from a central location, where someone can step in and deal with a call that’s potentially life threatening, it’s basically a set of voice mail boxes that belong to a team of community out reach workers, who are out on the road visiting at risk people in the community, and who carry a VoIP enabled smart phone with them, that’s capable of dialling back into the internal system. Case workers can dial into their associated VM boxes, and listen to/deal with message left for them via their phones.

There are cases however where subjects of the outreach program are more at risk than others, and some names are flagged from a database. If such a subject phones their case worker, but their caseworker is engaged with another subject and may not be able to take the call, someone back at base can spot the VM appearing on the system via the central web system, and they can issue an emergency page via a separate channel to the case worker in question.

In extreme circumstances, the support teams back at base, may need to listen to the message themselves, for instance … to alert emergency services, or in cases of a high workload the team may take over managing the messages, while the case works deal with the boots on the ground scenario.

So things don’t have to be split second real time, but they have to be acceptable enough that things are mostly kept in sync on a reasonably regular basis.

I was parsing out the voicemail.conf to get the definitive list of voicemail boxes, as I know that looking at the files in the spool directory doesn’t always tell the full story, and that it’s possible for a mailbox to be defined, but not appear on the file system if no messages have yet been left. My reasoning is that voicemail.conf is the source of all truth as to exactly what boxes are created in the system.

12

I’m sort of doing that now, in a roundabout way.

I have a custom FAGI script that sends the VMSTATE, extension dialed and cid to the app server and runs right after any call to app_voicemail, it’s not perfect but it does allow me to know when a VM was left and then use the extension to find the associated VM box number.

13

PS: forgot to mention, I’m also experimenting with “minivm” too, but there’s not a great deal of documentation on it.

14

98.9% Realtime is fine, I was discussing earlier with my client what an acceptable delay would be, between data updates, and they stated no more than 15 minutes.

To be fair I really wasn’t expecting to get above 80% :slight_smile:

Got it - in that case, I think what I described might work well for this use case.

The use case is quite simply that mailboxes are monitored from a central location, where someone can step in and deal with a call that’s potentially life threatening, it’s basically a set of voice mail boxes that belong to a team of community out reach workers, who are out on the road visiting at risk people in the community, and who carry a VoIP enabled smart phone with them, that’s capable of dialling back into the internal system. Case workers can dial into their associated VM boxes, and listen to/deal with message left for them via their phones.

There are cases however where subjects of the outreach program are more at risk than others, and some names are flagged from a database. If such a subject phones their case worker, but their caseworker is engaged with another subject and may not be able to take the call, someone back at base can spot the VM appearing on the system via the central web system, and they can issue an emergency page via a separate channel to the case worker in question.

To me, it sounds like this is too important for there to be a manual component involved at all. Your system should be able to detect such calls and automatically issue the page.

In extreme circumstances, the support teams back at base, may need to listen to the message themselves, for instance … to alert emergency services, or in cases of a high workload the team may take over managing the messages, while the case works deal with the boots on the ground scenario.

So things don’t have to be split second real time, but they have to be acceptable enough that things are mostly kept in sync on a reasonably regular basis.

The delay I mentioned would only be when voicemails that have already been left are deleted are moved; it would still be realtime for new messages, since there’s already an AMI event for that which is realtime.

I was parsing out the voicemail.conf to get the definitive list of voicemail boxes, as I know that looking at the files in the spool directory doesn’t always tell the full story, and that it’s possible for a mailbox to be defined, but not appear on the file system if no messages have yet been left. My reasoning is that voicemail.conf is the source of all truth as to exactly what boxes are created in the system.

Could you not use the VoiceMailUsersList action for this?

https://wiki.asterisk.org/wiki/display/AST/Asterisk+18+ManagerAction_VoicemailUsersList

Parsing the configs yourself definitely sounds very unnecessary to me.

I have a custom FAGI script that sends the VMSTATE, extension dialed and cid to the app server and runs right after any call to app_voicemail, it’s not perfect but it does allow me to know when a VM was left and then use the extension to find the associated VM box number.

Not when a voicemail is left - you already have an AMI event for that which will be more reliable.

When voicemail boxes are managed, messages might be moved or disappear, and a custom event for that.

15

YESSSSSSS!!!

Holy heck, why have I not seen this one yet???

This is not listed in the docs I’m using for reference, I don’t recall seeing it before now anyway.

That absolutely get’s me my master mailbox list, and along with the experiments I’ve just been doing using AMI user events, i think I’m in much better shape now than I was previously.

Of course, I’ll still have to manually check files in the spool directory, but I’m thinking of changing to use mini VM rather than comedian as that seems to give me better control of the recorded files.

question: Is it possible to “exec” something on the asterisk server (IE: a shell script) from a manager action?

I’m digging through docs looking now, but if you know off hand, then with your event ideas, switching to mini VPS and getting the mailbox users, I might actually be able to drive this entire thing via AMI

Cheers
shawty

16

That is part of the plan too, but I’ve gotta get the VM part working first :slight_smile:

17

This is not listed in the docs I’m using for reference, I don’t recall seeing it before now anyway.

What docs are you using? It’s in the official wiki documentation for all the recent/supported versions.

That absolutely get’s me my master mailbox list, and along with the experiments I’ve just been doing using AMI user events, i think I’m in much better shape now than I was previously.

Of course, I’ll still have to manually check files in the spool directory, but I’m thinking of changing to use mini VM rather than comedian as that seems to give me better control of the recorded files.

Yeah, I imagine you would need to check the directories if you want individual timestamps. Though if the market is big enough, hey, maybe we should add an AMI event to get all the messages’ metadata in a particular mailbox. Then these kinds of systems calls could be avoided altogether.

What else did you want to do with app_minivm that you don’t think app_voicemail can do?

question: Is it possible to “exec” something on the asterisk server (IE: a shell script) from a manager action?

Like an Asterisk CLI command, or an actual system command?
There is a Command action for the former.
For the latter, I’m quite certain “no”. You could use an SSH library to SSH into the server from your webserver if you needed to do that. But again, I have to ask - what is the use case? Maybe there is a better way.

I’m digging through docs looking now, but if you know off hand, then with your event ideas, switching to mini VPS and getting the mailbox users, I might actually be able to drive this entire thing via AMI

You should be able to (apart from getting voicemail timestamps as of right this second, I think), but if there are other barriers, please do bring them up.

18

System command… I’m thinking. Shell script on the asterisk server, call it from AMI and it echo’s back in AMI Event format a list of folders in the users mailbox, or one that sends back a list of files in the folders EG:

Event: UserEvent
UserEvent: FolderList
Folders: INBOX,Old,Saved,Client1,Client2
Box:103

or

Event:UserEvent
UserEvent:MessageFileList
Files:msg0000,msg0001,msg0002
Box:103;

Looking at that VoicemailList call you just pointed out to me, something like

Response: Success
EventList: start
Message: Voicemail messge files will follow

Event: VoicemailMessageFile
VMContext: default
VoiceMailbox: 103
Filename: Msg00001.txt
From: Shawty
Number: 12345
Duration: 9s
.....

Event: VoicemailMessageFile
VMContext: default
VoiceMailbox: 103
Filename: Msg00002.txt
From: Paul
Number: 67890
Duration: 30s
...

Event: VoicemailFileEntryComplete
EventList: Complete
ListItems: xxx

could be generated from the txt files in the VPS spool directory, I’m sure you can see where I’m going with this idea.

I was siding more with minivps, not beacuse of feature set, but it just doesn’t seem to get in the way as much as comedian does, but I still have experiments to finish, first so we’ll see :slight_smile:

right now, users of mailboxes can add/delete/rename folders in thier mailbox, from the phone and short of monitoring the file system for changes, we have no idea what those folders are called, in the system I’m working on, we have one worker who stores VM’s in “monthxx” folders, for messages they need to keep.

Longterm too, there will be a need to say, you know what folks… messages older than XXXX are getting backed up into long term storage and accessed only from web interface, with the proviso that case workers can dial into the office and ask support to retrieve a given message and put it back into their mailbox to listen too should the need arise (EG: to double check on something a subject previously said)

So lot’s and lot’s of stuff in the future, this is just the beginning of the project :slight_smile:

19

So it turns out is is/was in front of my eyes all along, I see it now that I have my docs open like this:

image
image1653×945 97.9 KB

I previously had them like this:

image
image1078×962 70.4 KB

It’s been a most productive evening @InterLinked thank you, I have a lot more ideas and things to look at now, it however time to finish for the night …

20

System command… I’m thinking. Shell script on the asterisk server, call it from AMI and it echo’s back in AMI Event format a list of folders in the users mailbox, or one that sends back a list of files in the folders EG:

I mean, you could certainly run an Originate command to be a predefined dialplan extension that invokes SHELL to call a shell script to get the directory information and then calls UserEvent in the dialplan. Not as elegant as a built-in, but it would work.

I think it’s very possible to do this all with just app_voicemail and AMI.