Many outbound calls not being logged in master.csv (cdr-csv)

We’re a Fonality customer and have an Asterisk PBX setup with them. We need to make call and queue stats available to a client via a web interface and have written our own interface.

However, when we compare the Master.csv data to Fonality’s reports, we find that Master.csv is missing many outgoing calls that Fonality is logging. Fonality swears that their script is reading the same logs ours are, but I don’t see where their calls are coming from. The queue_log doesn’t store outbound call information.

To give you an idea, the Master.csv file contains about 30 outbound calls for April while Fonality reports about 530. Most of the missing outbound calls are from extension 7204, but that extension is represented in some of the Master.csv entries… so it’s hit or miss.

Is there an asterisk config option that would cause some outbound calls to be logged but not others?

Please help! (Fonality isn’t helping)

Does anyone have any ideas?

Hi

You need to take this up with fonality. And if you are not happy with the outcome and are confident that calls arnt from you, Then consult a lawyer.

First thing would be to see their raw cdr files as this should show the calls coming from your IP address.

Ian

I don’t think you understand.

We make about 530 outgoing calls/mo according to fonality’s reports. However, when parsing through the master.csv.* cdr log files for that month, only about 30 outgoing calls appear. There does not appear to be any pattern to the missing calls – some extensions have calls logged some times and not others.

The question is… where are those calls logged if not in master.csv, or why wouldn’t they be logged in master.csv?

Maybe you are looking at the data wrong in Master.csv? Or you have NoCDR in your dialplan? Does 7204 have call forwarding enabled?

I definitely am not looking at the data wrong in the Master.csv file(s) (rotated every 5 minutes). I did a text string search over an entire week of master.csv.* files on an extension by extension basis. The only way the outgoing calls could be in there is if they aren’t given the same “callerid” as fonality reports.

As for the other two… I’m a web programmer assigned to parse the CSV data and am unfamiliar with asterisk setup… which is why I’m here. I’ll dig into your suggestions.

By the way, 7204 gets the bulk of the calls that fonality is reporting, and most of the missing calls are assigned to 7204. However, there are a few 7204 outgoing calls logged in the master.csv.* files.

If there are any other suggestions, I’m all ears.

How are the logs being rotated? Is is possible the rotation script is allowing cdr’s to be missed? 5 Minutes is extremely short especially for a server doing 500 or so external calls a month.

In addition to forwarding, check and see if the person at that extension transfers calls out as well.

You may also want to go through the log files manually to reconcile each record to see what may be happening. (Maybe the extension number is not in the record for some reason therefore not being picked up by your text search)

The server does 500 outgoing calls/mo, but it handles approximately 150,000 incoming calls/mo. The incoming calls match up nearly perfectly (one or two calls/mo are listed in the master.csv.* files that don’t make it into fonality’s reports) while we are way off on the outgoing calls.

Because of that difference, I’m inclined to doubt the rotation has anything to do with this.

I have gone through the master.csv.* files manually looking for a few of the missing calls. The data simply isn’t there.

Thank you for your suggestion on call transfers.

OK

Couple of things, rotating every 5 mins will/may cause problems. if you want to do this why not write to 2 files. simplist way would be for ALL extensions have the same accountcode varible assigned to them this way you will create a file of all outgoing calls.

Also look to impliment mysql cdr storage and then you can get your data direct from the database.

Could you explain why you rotate ever 5 mins ???

Ian

Sorry, I was in error… the logs rotate approximately every 15 minutes.

Fonality configured a large portion of the server, including the log rotation. I have to defer to them. But I would be very surprised if the log rotation is causing the problem, considering that we have so many more incoming calls, and they are almost 100% perfect.

I have been hoping that the problem is a configuration issue that wouldn’t require recompiling asterisk, as the mysql cdr-csv config requires. Also, we are using asterisk 1.2.17 (again, I don’t know why that version).

Hi OK

Are all the sets SIP ? if so add setvar=CDR(accountcode)=mycdrfile to each sip.conf entry

This way you will also get a file called mycdrfile.csv as well as Master.csv

Ian

Ask Fonality of the specific details and maybe walk you through how they are pulling the data …

Is your company making 30 calls a month or 500 calls a month outbound? You probably should know the answer to that without looking at any logs.

I decided to try some new searches with the data from fonality that is missing in the master.csv files.

What I found is that the calls ARE all there. However, if an extension dials 9 before calling out, then it stores the PBX phone number in the log instead of the extension that calls out. We need to be able to store the extension for all outgoing calls.

How can we solve this? If disabling dialing 9 is our only option, how can we do that?

I have uncovered a bit more detail.
Until noon on April 1, all calls using 9 to dial out were logged to their extension. After 1:30pm on April 1 until the present, all calls using 9 to dial out were logged to the PBX number, not the extension.

I have no idea what Asterisk setting chooses this, but something must have changed on that day. I’ve notified the IT guy who has been working with the PBX.

It would be nice if anyone could offer any ideas so we can resolve this. Thanks.

Hi

round this time i guess someone asked to have the cli set to the outgoing number.

for will find somewhere a line similar to set(${CALLERID(num)}=blahblah)

Ian

As to solving it search on the originating channel not callerid

Ian

Thanks for the advice. Apparently, this was something fonality changed on that date. Perhaps they will give us something helpful to resolve it.

We need the originating extension because our extensions are organized into groups based on the first two numbers of the extension, and we do stats on those groups that match up with different queues. Matching to the channel is a problem because we’d have to maintain a separate table just to match up the MAC address of a given phone to the queue group. Managing that would be rather painful, and a phone could never be assigned to a different queue group, or else the data would never be 100% accurate.

Hi

Hm the fundemental problem is people rely on using field 2 as the source but this is actually the source callerID not the source extension.

This is in column 6 which is the source channel.

“”,“0122123456”,“901242123456”,“international”,""“Snom M3"” <01225123456>",“SIP/2124-09423460”,“SIP/3301-09430e40”,“Dial”,“SIP/801242123456@3001|60|Ww”,“2008-05-15 18:59:10”,“2008-05-15 18:59:23”,“2008-05-15 18:59:40”,30,17,“ANSWERED”,“DOCUMENTATION”

It should be hard to parse the files using column 6 as for example if all the extensions are sip then you ignore SIP/ and then read the next X digits as your extension

why ?

The actual Fields are

  1. accountcode: What account number to use: account, (string, 20 characters)
  2. src: Caller*ID number (string, 80 characters)
  3. dst: Destination extension (string, 80 characters)
  4. dcontext: Destination context (string, 80 characters)
  5. clid: Caller*ID with text (80 characters)
  6. srcchannel: Channel used (80 characters)
  7. dstchannel: Destination channel if appropriate (80 characters)
  8. lastapp: Last application if appropriate (80 characters)
  9. lastdata: Last application data (arguments) (80 characters)
  10. start: Start of call (date/time)
  11. answer: Anwer of call (date/time)
  12. end: End of call (date/time)
  13. duration: Total time in system, in seconds (integer), from dial to hangup
  14. billsec: Total time call is up, in seconds (integer), from answer to hangup
  15. disposition: What happened to the call: ANSWERED, NO ANSWER, BUSY, FAILED (on some CDR backends, e.g. ODBC, these may be integers; note that more detailed info can be found in the dialplan variable $HANGUPCAUSE)
  16. amaflags: What flags to use: see amaflags: DOCUMENTATION, BILLING, IGNORE etc, specified on a per channel basis like accountcode.
  17. user field: A user-defined field, maximum 255 characters [/quote]

You could also do like ianplain suggested above and specify the account code on the sip peer level. Set the code as the group the phone belongs to. Then you’d have 1 file per group with all of the associated records for outbound.

Here’s an example field 6:
SIP/0004F2035277-b3b56270

My PBX guy says that 0004F2035277 is the mac address for the phone, but he doesn’t know what the string after the dash is. I did some searching, and it seems that string is almost always different from entry to entry. In cases where that part is the same, it is a different phone and a different extension making the call – I don’t see any pattern to it.

Also, there are outgoing calls with a field 6 similar to the following:
Local/91xxxxxxxxxx@internal-ee71,2
(where x’s represent the outgoing phone number digits)
I am uncertain as to the case where a call uses the local channel vs. SIP, but we have to account for these calls as well.

That’s why we were sticking to the extension in the log. If the channel log entry could be configured to match Ian’s and/or Dave’s descriptions, that would obviously be useful.