Solution to 1.8 problems on Kirkwood [other?] ARMs

If you are compiling Asterisk on a Kirkwood Arm (SheevaPlug, GuruPlug, Dockstar, etc.), or possibly on any other ARM cpu, this posting is for you.

These issues appeared in 1.8.1.1 and 1.8.2.2 on a SheevaPlug (Kirkwood ARM).

Problems appeared in SIP, such as:

[ul]
[li]“ERROR[23208]: rtp_engine.c:325 ast_rtp_instance_new: No RTP engine was found. Do you have one loaded?
[Nov 9 13:06:32] ERROR[23208]: chan_sip.c :8078 process_sdp: Got SDP but have no RTP session allocated.”[/li]
[li]The inability to use a secret.[/li]
[li]“Peer not found”[/li]
[li]Issues with the deny/permit IP address ranges[/li][/ul]

Problems appeared in IAX:

[ul][li]Failure to validate with RSA keys (could not find the keys)[/li]
[li]Failure to find the context and number when dialing into the pbx from outside.[/li]
[li]Inability to use deny/permit to restrict by IP address[/li][/ul]

In my experience, the various problems would present, or not present, depending on the sequence of actions taken. Because of the ultimate cause, there are probably many other problems which may appear, depending upon any given pbx’s configuration files and the sequence of events.

All of these problem were/are caused because of an ARM alignment problem in the dynamic string allocation code. All of the issues (on my machine) were resolved by fixing that problem.

Technical Cause:

The actual error occurs in a defined macro on line 304 of [Asterisk-sources]/include/asterisk/stringfields.h

#define AST_STRING_FIELD_ALLOCATION(x) *((ast_string_field_allocation *) (x - sizeof(ast_string_field_allocation)))

The purpose of the macro is to calculate the address which is ast_string_field_allocation in front of the string storage address (the passed-in value, x). The definition of ast_string_field_allocation is a uint16_t. Because of the pointer cast, the calculated result will be treated as a uint16_t (that is, a 2-byte half-word)

The problem is that the ARM needs uint16_t’s to be aligned to word, half-word, or byte, depending upon the value being stored [unless cast as (char *)]. So, if (x - sizeof(ast_string_field_allocation)) does not align with the first byte of the memory half-word, attempting to store a half-word into the address will actually store it in the previous byte and the addressed byte.

The result is, for about half of the dynamically-allocated strings, every time a string value is set [using ast_string_field_set()], which calls AST_STRING_FIELD_ALLOCATION(), the string-terminating NULL value in the preceeding string in the memory region will be over-written with some number (the length of the string value being set).

Often the number which gets “concatenated” to the other string will be unprintable; or sometimes in CLI, it will appear as if some string is one character shorter than it should be. On verbose settings, the bogus chars can be seen in the log file.

Resolution:

Actually, this same issue exists for SPARC chips, and the code to dynamically adjust for WORD alignment already exist. It just needs to be included in the ARM compile with a few pre-compiler changes in [Asterisk-Sources]main/utils.c .

A short-term patch for the ARM (at least Kirkwoods) would look like the following.

(Caution: these are unofficial and unsanctioned changes. This is what worked on my machine, but each person has the responsibility for their own understanding and actions.)

main/utils.c
@@ +63,3 -63,0
	#include "asterisk/config.h"
 +	
 +	/* Temporary declaration for ARM cpus ONLY (this belongs in the make environment) */
 +	#define __arm__ 1

@@ +1635,1 -1635,1 @@
 -	#ifdef __sparc__
 +	#if defined(__sparc__) || defined(__arm__)

@@ +1716,1 -1716,1 @@
 -	#ifdef __sparc__
 +	#if defined(__sparc__) || defined(__arm__)

After the changes, the entire system will need to be re-made and re-installed (make clean, make menuselect [save], make, make install). No need to reinstall the samples.

When building my system, I saw a message that PROC type arm was not recognized. That seems to indicate that ARM processors are not supported by Asterisk developers. My initial impression, after making the changes mentioned on this page, is that the code will run reliably on a Kirkwood ARM.

Frankly, I’m somewhat inexperienced with the ins and outs of make, so am not certain where the variable “arm” should be entered. Obviously, it belongs in the environment, not in this individual file. However, this seems to be the only place in the code where the #ifdef sparc is used for the string alignment. So for a temporary patch for ARM users, just adding #define arm to the top of this file should be fine.
(There is some sparc-related code in include/asterisk/unaligned.h, but that appears to be byte swapping for endian issues.)

On a slightly related note, on the SheevaPlug (maybe others in the series), make sure that PROC=arm in makeopts before trying to make codec_gsm. “armv5tel” does not work.

Howdy,

Posting this to the issue tracker (issues.asterisk.org) would be a great step forward. As we can get to it, it’d be a great help for anyone else that runs into the problem.

Cheers.

Where is utils.c? I checked a few versions (from 8.1.2 to 8.2.3) but there’s no utils.c in the utils directory.

Dang!, got me again. It’s copied in from main/utils.c during the make (and erased during make clean). This explains why utils.c “disappeared” while I was tracing the cause (turns out I didn’t rm it in my half-sleep after all).

But, it also means I’ve got to do more testing, and at least another couple of re-builds (underway now).

It’s possible that the dynamic alignment (instead of the extra byte length) can be done. Which would mean the only file to change would be main/utils.c.

I’ll post again as soon as the answers are in. Hopefully tonight, maybe tomorrow.

radael:
Thanks. I made the changes and installed Asterisk 1.8.2.3 on Dockstar with Debian Linux. It appears working. I turned off and on a few times. Everything are working. I have no problem to make call through gtalk.
Your work and discovery is really great. I have been tried to install Asterisk 1.8 with gtalk for a couple of months but never was able to make it work until tried your solution. I only tested it for a few hours but I am sure it has fixed the problem. I will try it more and report if I observe any issues. I hope it, or its refined version, can be included in official release soon. This is indeed significant contribution to Asterisk community.
I believe it will also work in Openwrt installation. However, it will need patch files but I am not familiar with the procedure yet. I will try but I guess others more familiar with the process will beat me on this.
Thanks again. This solved a difficult problem for me and many others on the same boat.

You’re too kind, twinclouds. Just bumbled the way through, . . . and didn’t really discover anything – the Asterisk developers already coded for this, just not for the ARM.

Once I found utils/utils.c, I never suspected that it was being re-written with each make. But, that explains why I wasn’t able to get the dynamic address alignment to work at first ('cause the change was being over-written).

Once you pointed out that utils/utils.c didn’t exist, and the static version was found in main/utils.c, it became clear that was the file where changes should be made.

So, after removing the originally-proposed changes to stringfiles.h, and updating main/utils.c, everything still works as it should. I’ve re-tested the functionality and examined string alignment at the byte-level and the results are as expected (would have been a surprise if it wasn’t).

The initial posting with the instructions has been updated with the preferred method.

If anyone made the change to both main/utils.c and to stringfields.h, there should not be any problems. The executable will be very slightly larger. And each of the dynamic strings will use one extra byte out of the string buffer regions. But, likely there will not even be any additional buffer space allocated.
Still, I’d recommend reverting the stringfields.h changes and using only the changes to main/utils.c.

radael:
Thanks.
A problem is usually quite simple once you discover its cause. The difficult part is to identify it. I feel your work is significant because we all have notice the problem but no one found the cause and solution. (I am an engineer but neither an Asterisk nor C programmer so I can’t do much in this area.) In any case, I think this is why the open source community works so well.
I put a step by step how-to on installing asterisk 1.8 on Dockstar/Debian for gtalk, which incorporated the fix that you discovered (with references of your original post), at forums.plugpbx.org/index.php/topic,247.0.html. Hopefully it will be useful for others.

Asterisk18 (1.8.2.3) has been released for Openwrt trunk. I compiled all of the packages fine. However, it still does not solve the word alignment problem. By applying the patch based on discussion in this thread, it works fine for gtalk. I only tested files needed for gtalk but not all of the files, though.

Did anybody succeed using chan_datacard on Dockstar with an Huawei modem?