RTP packet never makes it to OUTPUT chain

So I have the typical no audio in one direction problem. (Calling from the VoIP phone, I can hear the other side, but they can’t hear me). I have forwarded the relevant ports (UDP: 5004-5082, 19900-20000 (I restricted RTP to that range)). And yet, not much luck.
I have done some tcpdumps and I can see my phone sending RTP data to my asterisk box, and the asterisk box then sending RTP data to an external address.
I have added some LOG targets to iptables on my gateway router, and it would appear the RTP packets are making it to the FORWARD chain, but I don’t see them again on the OUTPUT chain. I tried adding a rule which accepted any UDP packets on the FORWARD chain, and still nothing on the OUTPUT chain.
Any advice? Have I totally diagnosed things incorrectly? Am I misunderstanding iptables. I just don’t see why these packets are not seen again after the FORWARD chain.

Below, the iptables, as well as the log output:

note: 3.141 is the address of the pbx

FILTER:

Chain INPUT (policy DROP)
num  target     prot opt source               destination
1    LOG        udp  --  anywhere             anywhere            limit: avg 5/sec burst 5 udp spts:19900:20000 LOG level debug prefix `--filter-INPUT---'
2    DROP       0    --  anywhere             rrcs-<stringified IP address>.sw.biz.rr.com
3    DROP       0    --  anywhere             anywhere            state INVALID
4    ACCEPT     0    --  anywhere             anywhere            state RELATED,ESTABLISHED
5    ACCEPT     0    --  anywhere             anywhere
6    ACCEPT     0    --  anywhere             anywhere

Chain FORWARD (policy DROP)
num  target     prot opt source               destination
1    LOG        udp  --  anywhere             anywhere            limit: avg 5/sec burst 5 udp spts:19900:20000 LOG level debug prefix `--filter-FORWARD---'
2    ACCEPT     0    --  anywhere             anywhere
3    DROP       0    --  anywhere             anywhere            state INVALID
4    TCPMSS     tcp  --  anywhere             anywhere            tcp flags:SYN,RST/SYN tcpmss match 1461:65535 TCPMSS set 1460
5    L7in       0    --  anywhere             anywhere
6    ACCEPT     0    --  anywhere             anywhere            state RELATED,ESTABLISHED
7    wanin      0    --  anywhere             anywhere
8    wanout     0    --  anywhere             anywhere
9    ACCEPT     0    --  anywhere             anywhere
10   upnp       0    --  anywhere             anywhere

Chain OUTPUT (policy ACCEPT)
num  target     prot opt source               destination
1    LOG        udp  --  anywhere             anywhere            limit: avg 5/sec burst 5 udp spts:19900:20000 LOG level debug prefix `--filter-OUTPUT---'

Chain L7in (1 references)
num  target     prot opt source               destination
1    RETURN     0    --  anywhere             anywhere            LAYER7 l7proto flash
2    RETURN     0    --  anywhere             anywhere            LAYER7 l7proto httpvideo
3    RETURN     0    --  anywhere             anywhere            LAYER7 l7proto shoutcast

Chain triggers (1 references)
num  target     prot opt source               destination
1    TRIGGER    tcp  --  anywhere             anywhere            tcp dpt:55 TRIGGER type:out tcp match:55 relate:55
2    TRIGGER    udp  --  anywhere             anywhere            udp dpt:55 TRIGGER type:out udp match:55 relate:55
3    TRIGGER    tcp  --  anywhere             anywhere            tcp dpt:kerberos TRIGGER type:out tcp match:88 relate:88
4    TRIGGER    udp  --  anywhere             anywhere            udp dpt:kerberos TRIGGER type:out udp match:88 relate:88

Chain upnp (1 references)
num  target     prot opt source               destination

Chain wanin (1 references)
num  target     prot opt source               destination
1    TRIGGER    0    --  anywhere             anywhere            TRIGGER type:in match:0 relate:0
2    ACCEPT     tcp  --  anywhere             offrouter.<my domain name> tcp dpt:8585
3    ACCEPT     udp  --  anywhere             offrouter.<my domain name> udp dpt:8585
4    ACCEPT     udp  --  anywhere             pbx.<my domain name>  udp dpts:19900:20000
5    ACCEPT     udp  --  anywhere             pbx.<my domain name>  udp dpts:5004:5082

Chain wanout (1 references)
num  target     prot opt source               destination
1    triggers   0    --  anywhere             anywhere
#
# clear
# iptables -L --line-numbers
Chain INPUT (policy DROP)
num  target     prot opt source               destination
1    LOG        udp  --  anywhere             anywhere            limit: avg 5/sec burst 5 udp spts:19900:20000 LOG level debug prefix `--filter-INPUT---'
2    DROP       0    --  anywhere             rrcs<stringified IP address>.sw.biz.rr.com
3    DROP       0    --  anywhere             anywhere            state INVALID
4    ACCEPT     0    --  anywhere             anywhere            state RELATED,ESTABLISHED
5    ACCEPT     0    --  anywhere             anywhere
6    ACCEPT     0    --  anywhere             anywhere

Chain FORWARD (policy DROP)
num  target     prot opt source               destination
1    LOG        udp  --  anywhere             anywhere            limit: avg 5/sec burst 5 udp spts:19900:20000 LOG level debug prefix `--filter-FORWARD---'
2    ACCEPT     0    --  anywhere             anywhere
3    DROP       0    --  anywhere             anywhere            state INVALID
4    TCPMSS     tcp  --  anywhere             anywhere            tcp flags:SYN,RST/SYN tcpmss match 1461:65535 TCPMSS set 1460
5    L7in       0    --  anywhere             anywhere
6    ACCEPT     0    --  anywhere             anywhere            state RELATED,ESTABLISHED
7    wanin      0    --  anywhere             anywhere
8    wanout     0    --  anywhere             anywhere
9    ACCEPT     0    --  anywhere             anywhere
10   upnp       0    --  anywhere             anywhere

Chain OUTPUT (policy ACCEPT)
num  target     prot opt source               destination
1    LOG        udp  --  anywhere             anywhere            limit: avg 5/sec burst 5 udp spts:19900:20000 LOG level debug prefix `--filter-OUTPUT---'

Chain L7in (1 references)
num  target     prot opt source               destination
1    RETURN     0    --  anywhere             anywhere            LAYER7 l7proto flash
2    RETURN     0    --  anywhere             anywhere            LAYER7 l7proto httpvideo
3    RETURN     0    --  anywhere             anywhere            LAYER7 l7proto shoutcast

Chain triggers (1 references)
num  target     prot opt source               destination
1    TRIGGER    tcp  --  anywhere             anywhere            tcp dpt:55 TRIGGER type:out tcp match:55 relate:55
2    TRIGGER    udp  --  anywhere             anywhere            udp dpt:55 TRIGGER type:out udp match:55 relate:55
3    TRIGGER    tcp  --  anywhere             anywhere            tcp dpt:kerberos TRIGGER type:out tcp match:88 relate:88
4    TRIGGER    udp  --  anywhere             anywhere            udp dpt:kerberos TRIGGER type:out udp match:88 relate:88

Chain upnp (1 references)
num  target     prot opt source               destination

Chain wanin (1 references)
num  target     prot opt source               destination
1    TRIGGER    0    --  anywhere             anywhere            TRIGGER type:in match:0 relate:0
2    ACCEPT     tcp  --  anywhere             offrouter.<my domain name> tcp dpt:8585
3    ACCEPT     udp  --  anywhere             offrouter.<my domain name> udp dpt:8585
4    ACCEPT     udp  --  anywhere             pbx.<my domain name>  udp dpts:19900:20000
5    ACCEPT     udp  --  anywhere             pbx.<my domain name>  udp dpts:5004:5082

Chain wanout (1 references)
num  target     prot opt source               destination
1    triggers   0    --  anywhere             anywhere

NAT:

# iptables -t nat -L --line-numbers
Chain PREROUTING (policy ACCEPT)
num  target     prot opt source               destination
1    LOG        udp  --  anywhere             anywhere            limit: avg 5/sec burst 5 udp spts:19900:20000 LOG level debug prefix `--nat-PRE---'
2    DROP       0    --  anywhere             192.168.3.0/24
3    DNAT       icmp --  anywhere             rrcs<stringified ip address>.sw.biz.rr.com to:192.168.3.1
4    DNAT       tcp  --  anywhere             rrcs<stringified ip address>.sw.biz.rr.com tcp dpt:8585 to:192.168.3.10:8585
5    DNAT       udp  --  anywhere             rrcs<stringified ip address>.sw.biz.rr.com udp dpt:8585 to:192.168.3.10:8585
6    DNAT       udp  --  anywhere             rrcs<stringified ip address>.sw.biz.rr.com udp dpts:19900:20000 to:192.168.3.141
7    DNAT       udp  --  anywhere             rrcs<stringified ip address>.sw.biz.rr.com udp dpts:5004:5082 to:192.168.3.141
8    TRIGGER    0    --  anywhere             rrcs<stringified ip address>.sw.biz.rr.com TRIGGER type:dnat match:0 relate:0
9    upnp       0    --  anywhere             rrcs<stringified ip address>.sw.biz.rr.com

Chain POSTROUTING (policy ACCEPT)
num  target     prot opt source               destination
1    LOG        udp  --  anywhere             anywhere            limit: avg 5/sec burst 5 udp spts:19900:20000 LOG level debug prefix `--nat-POST---'
2    SNAT       tcp  --  192.168.3.0/24       offrouter.<my domain name> tcp dpt:8585 to:192.154.179.131
3    SNAT       udp  --  192.168.3.0/24       offrouter.<my domain name> udp dpt:8585 to:192.154.179.131
4    SNAT       udp  --  192.168.3.0/24       pbx.<my domain name>  udp dpts:19900:20000 to:192.154.179.131
5    SNAT       udp  --  192.168.3.0/24       pbx.<my domain name>  udp dpts:5004:5082 to:192.154.179.131
6    MASQUERADE  0    --  anywhere             anywhere

Chain OUTPUT (policy ACCEPT)
num  target     prot opt source               destination
1    LOG        udp  --  anywhere             anywhere            limit: avg 5/sec burst 5 udp spts:19900:20000 LOG level debug prefix `--nat-OUTPUT---'

Chain upnp (1 references)
num  target     prot opt source               destination
#

Log output:

Aug 24 21:23:47 rfirouter user.debug kernel: --nat-PRE---IN=br0 OUT= MAC=00:1d:7e:bb:98:ca:00:01:2e:4d:07:9e:08:00 SRC=192.168.3.141 DST=4.55.17.2 LEN=92 TOS=0x00 PREC=0x00 TTL=64 ID=0 DF PROTO=UDP SPT=19927 DPT=6547 LEN=72 
Aug 24 21:23:47 rfirouter user.debug kernel: --nat-POST---IN= OUT=vlan1 SRC=192.168.3.141 DST=4.55.17.2 LEN=92 TOS=0x00 PREC=0x00 TTL=63 ID=0 DF PROTO=UDP SPT=19927 DPT=6547 LEN=72 
Aug 24 21:23:49 rfirouter user.debug kernel: --filter-FORWARD---IN=br0 OUT=vlan1 SRC=192.168.3.141 DST=4.55.17.2 LEN=200 TOS=0x18 PREC=0xA0 TTL=63 ID=0 DF PROTO=UDP SPT=19926 DPT=6546 LEN=180 
Aug 24 21:23:49 rfirouter user.debug kernel: --filter-FORWARD---IN=br0 OUT=vlan1 SRC=192.168.3.141 DST=4.55.17.2 LEN=200 TOS=0x18 PREC=0xA0 TTL=63 ID=0 DF PROTO=UDP SPT=19926 DPT=6546 LEN=180 
Aug 24 21:23:49 rfirouter user.debug kernel: --filter-FORWARD---IN=br0 OUT=vlan1 SRC=192.168.3.141 DST=4.55.17.2 LEN=200 TOS=0x18 PREC=0xA0 TTL=63 ID=0 DF PROTO=UDP SPT=19926 DPT=6546 LEN=180 
Aug 24 21:23:49 rfirouter user.debug kernel: --filter-FORWARD---IN=br0 OUT=vlan1 SRC=192.168.3.141 DST=4.55.17.2 LEN=200 TOS=0x18 PREC=0xA0 TTL=63 ID=0 DF PROTO=UDP SPT=19926 DPT=6546 LEN=180 
Aug 24 21:23:50 rfirouter user.debug kernel: --filter-FORWARD---IN=br0 OUT=vlan1 SRC=192.168.3.141 DST=4.55.17.2 LEN=200 TOS=0x18 PREC=0xA0 TTL=63 ID=0 DF PROTO=UDP SPT=19926 DPT=6546 LEN=180 
Aug 24 21:23:50 rfirouter user.debug kernel: --filter-FORWARD---IN=br0 OUT=vlan1 SRC=192.168.3.141 DST=4.55.17.2 LEN=200 TOS=0x18 PREC=0xA0 TTL=63 ID=0 DF PROTO=UDP SPT=19926 DPT=6546 LEN=180 
Aug 24 21:23:50 rfirouter user.debug kernel: --filter-FORWARD---IN=br0 OUT=vlan1 SRC=192.168.3.141 DST=4.55.17.2 LEN=200 TOS=0x18 PREC=0xA0 TTL=63 ID=0 DF PROTO=UDP SPT=19926 DPT=6546 LEN=180 
Aug 24 21:23:51 rfirouter user.debug kernel: --filter-FORWARD---IN=br0 OUT=vlan1 SRC=192.168.3.141 DST=4.55.17.2 LEN=200 TOS=0x18 PREC=0xA0 TTL=63 ID=0 DF PROTO=UDP SPT=19926 DPT=6546 LEN=180 
Aug 24 21:23:51 rfirouter user.debug kernel: --filter-FORWARD---IN=br0 OUT=vlan1 SRC=192.168.3.141 DST=4.55.17.2 LEN=200 TOS=0x18 PREC=0xA0 TTL=63 ID=0 DF PROTO=UDP SPT=19926 DPT=6546 LEN=180

... more FORWARD log entries

In case anyone cares, it seems like there was a hardware or software issue with the particular device.
Changing out to another router, and forwarding the appropriate ports fixed the issue.
The problem device was a WRT54GL running tomato. Since parts of the iptable entries were auto generated by the tomato UI, I would guess some things got somewhat messed up. I had a similar NAT problem with a WRT54GL running dd wrt. Those tables were filled with unused chains and other hoopla. Disabling the SPI firewall and recreating the needed rules manually did the trick.