DB corruption or ... ? Asterisk stopped registering for a while

[chconnor]

Hi – we have a small private PBX running in the cloud on a basic asterisk install.

It has been working fine. I was working on web frontend to see basic status of endpoints (over lighttpd) and somehow asterisk cancelled the registration of all the endpoints (not sure if this happened immediately or gradually, I just noticed it eventually.) Might have just expired due to age?

I checked the logs, stopped/started asterisk, made sure nothing relevant was banned in the fail2ban chain, etc, but could not find an explanation or solution. Eventually out of desperation I rebooted the server and everything was back to normal (and my web frontend also works fine.)

The weird part was that in the security log it looked as though the ATAs were contacting, but registration wasn’t happening (and there weren’t errors that I could see):

SECURITY[1030] res_security_log.c: SecurityEvent="ChallengeSent",EventTV="2025-03-15T04:32:16.630+0000",Severity="Informational",Service="PJSIP",EventVersion="1",AccountID="001234",SessionID="1129721659-5060-897@BJC.BGI.GI.IH",LocalAddress="IPV4/TLS/10.0.0.174/5061",RemoteAddress="IPV4/TLS/123.123.123.123/46044",Challenge=""
SECURITY[1030] res_security_log.c: SecurityEvent="SuccessfulAuth",EventTV="2025-03-15T04:32:16.669+0000",Severity="Informational",Service="PJSIP",EventVersion="1",AccountID="001234",SessionID="1129721659-5060-897@BJC.BGI.GI.IH",LocalAddress="IPV4/TLS/10.0.0.174/5061",RemoteAddress="IPV4/TLS/123.123.123.123/46044",UsingPassword="1"

…but in the “full” log there was never a line saying the endpoint was Reachable. The only thing interesting I could note in the “full” was log were lines like:

VERBOSE[179580] res_pjsip_registrar.c: Added contact 'sips:001234@123.123.123.123:47874;transport=TLS;x-ast-orig-host=192.168.0.90:5060' to AOR '001234' with expiration of 2700 seconds
WARNING[73704] db.c: Error executing SQL (COMMIT): database is locked

…my understanding is that the “database is locked” message happens when there is contention for the DB. Could DB corruption explain the issues?

My web frontend is just using “asterisk -x” commands to query some basic info about the endpoints. It would seem strange that it would step on its own toes in terms of database contention, so I assume that isn’t the issue?

There aren’t any third party apps accessing the DB, AFAIK.

Part of making my web frontend involved granting sudoer to www-data for asterisk so the CGI module can execute the asterisk command, in case that’s relevant.

Appreciate any thoughts… We definitely don’t want this to happen again!

[TheNextDay]

While this won’t help you a lot, I can tell you that I also have a corrupted astdb once in a while. It’s like really very rarely but it happens and leads to similar messages as you describe.
By now I didn’t find out the reason. I don’t think that your Web frontend work has something to do with it as you only read values.

When it happens, I usually stop asterisk, delete the sqlite file and start asterisk again.

[chconnor]

Hmm, thanks, it does help to have that perspective, although that’s mildly disturbing.

Deleting the DB – that just eliminates the current state of the system? For example, if I have custom key/value pairs in there from “database put”, I assume those are gone as well?

And even if my asterisk -x calls were writing to the DB, I assume it would cooperate with itself in terms of DB locking, yeah?

[TheNextDay]

I am quite straightforward with deleting the dB because I do not store own values. If you do that, of course you should take care of that.

And yes, I guess also writing should not be the problem. As said, I don’t know the real cause.