Again ... too many open files error

Hello,

I’m running asterisk 1.4.22 with realtime mysql /addons 1.4.7/. Kind of busy box.
Approx 10 hitting calls per second and ~ 60 simultaneous calls. There is a powerful machine - Core 2 Quad@ 2.5GHz, with 2 Gb ram SATA II HDD - not much loaded at all. But after a day/max/ I got “too many open files” error.

I start trace how many files are opened by asterisk with ls -l /proc/30842/fd/ |wc -l
so here is the output:
[root@www ~]# ls -l /proc/30842/fd/ |wc -l
44022
[root@www ~]# ls -l /proc/30842/fd/ |wc -l
44019
[root@www ~]# ls -l /proc/30842/fd/ |wc -l
44073

These 3 ls -l /proc/30842/fd/ |wc -l are issues in 2 seconds from each other.
Any ideas, clues how to fix this ?
Thanks.

What does your mem useage look like? We had a very similar problem but it was due to a mem leak and once the ram filled up we would see the too many open files error.
Here is the post i had on that… never got any replies and never found a solution :confused:
forums.digium.com/viewtopic.php? … highlight=
We had the same problem with 1.4.22 as well. Lastnight we upgraded to 1.6.0.1… still waiting to see how that goes.

Im really wandering if you are seeing the same problem with memory useage.

J

hm… "no such command memory show"
can you guide me what module to load to run memory command ?

should we post this “too many open files” issue as bug ?

The asterisk command is:
memory show summary
That gives a quick little break down of how much memory is in use and where its going.

Another helpfull command is:
memory show allocations
This will list every memory allocation… it may be a little much.

Here is what our looks like currently if you want to compaire:

Edit* I forgot to mention that Asterisk may need to be compiled with memory debugging enabled in order to get this command.

J

at this moment * is not very loaded . only a few calls .
This does not change the situation - with every call ls -l /proc/4231/fd/ |wc -l grows /pid is different due to restart - too many open files cause/

[quote] memory show summary
516 bytes in 1 allocations in file 'enum.c’
48 bytes in 1 allocations in file 'devicestate.c’
9312 bytes in 3 allocations in file 'mpool/mpool.c’
203 bytes in 4 allocations in file 'config.c’
16 bytes in 2 allocations in file 'channel.c’
35 bytes in 2 allocations in file 'pbx_config.c’
584 bytes in 2 allocations in file 'btree/bt_open.c’
12316 bytes in 1 allocations in file 'stdtime/localtime.c’
1200 bytes in 11 allocations in file 'sched.c’
10292 bytes in 7 allocations in file 'io.c’
1398 bytes in 20 allocations in file 'utils.c’
288 bytes in 2 allocations in file 'logger.c’
11300 bytes in 5 allocations in file 'chan_sip.c’
1080 bytes in 5 allocations in file 'file.c’
977 bytes in 36 allocations in file 'manager.c’
24440 bytes in 22 allocations in file '/root/asterisk-1.4.22/include/asterisk/'
9316 bytes in 36 allocations in file 'loader.c’
5056 bytes in 66 allocations in file 'pbx.c’
3017 bytes in 101 allocations in file 'asterisk.c’
3422 bytes in 211 allocations in file 'cli.c’
94816 bytes allocated in 538 allocations[/quote]

Lets hope that some of the * gurus will give a clue …
I will paste memory show summary when it is more busy

Standard Linux allows 1024 open files and on a busy asterisk server such as yours, you are probably running out of file handles. What we do is edit the /etc/init.d/asterisk start script and add the following line in the start() section:

ulimit -n 32768

This allows Linux to have up to 32768 open files, which solves the problem. You have to do this in the startup script. If you just open a console and run that command, only that instance will have the larger number of open files.

Thank you for your answer.

I tried that … but no success
here is my ulimit -a

[quote]# ulimit -a
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
file size (blocks, -f) unlimited
pending signals (-i) 1024
max locked memory (kbytes, -l) 32
max memory size (kbytes, -m) unlimited
open files (-n) 65535
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
stack size (kbytes, -s) 10240
cpu time (seconds, -t) unlimited
max user processes (-u) 16115
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
[/quote]

asterisk just hit the limit of 65535 open files and crashes …