Troubleshooting: Imunify360 database is corrupt. Application cannot run with corrupt database

Error Message:

# service imunify360 start
Starting imunify360: WARNING [+ 3743ms] defence360agent.utils.check_db|DatabaseError detected: database disk image is malformed
WARNING [+ 3766ms] defence360agent.cli.subparsers.common.server|Imunify360 database is corrupt. Application cannot run with corrupt database. Please, contact Imunify360 support team at https://cloudlinux.zendesk.com

Detail Information & Explanation:

If you are using imunify360, an application firewall for linux servers by Cloudlinux team, you might incur an error where it says the database is corrupt. You might first see ‘Imunify360 is not started’ error from the WHM panel and end up getting the above error message as stated. Imunify360 uses a SQL database, located under ‘/var/imunify360/imunify360.db’. This image is checked everytime Imunfi360 tries to start, and if the database is malformed, it would not start. Fortunately, imunify360 comes with tools to handle this database and recover if corrupted.

How to Fix:

First, we start by running database integrity check. This can be done using the following:

imunfiy360-agent checkdb

(From Imunify360 Doc: checkdb  – Check database integrity)

Once done, you can now use ‘migratedb’ to repair and restore if the database is corrupted.

imunify360-agent migratedb

(From Imunify360 Doc: migratedb – Check and repair database if it is corrupted.)

If migratedb fails, the only way to recover this is to reinstall imunify360.

[Tue Dec 19 20:40:07.097202 2017] [lsapi:error] [pid 532140:tid 139848266454784] [client IP.IP.IP.IP:25021] mod_lsapi: [host domain.com] [req GET / HTTP/1.1] Could not connect to lsphp backend: connect to lsphp refused: 111 (possibly memory limit for LVE ID 1789 too small), referer: domain.com

If you are running Cloudlinux, cagefs and lsapi with cpanel, you are probably familiar with the error. If the error is appearing for one or two sites, then it is probably because the user is hitting the VM/PM limit you have set through Cloudlinux. But if the error is appearing for all the sites, then it is because the cagefs fails with the suexec permission for some reason.

[Tue Dec 19 20:40:07.097202 2017] [lsapi:error] [pid 532140:tid 139848266454784] [client *:25021] mod_lsapi: [host *] [req GET / HTTP/1.1] Could not connect to lsphp backend: connect to lsphp refused: 111 (possibly memory limit for LVE ID 1789 too small), referer: http://*/

One way to solve the problem is to remount all the users. Sometimes, it doesn’t work and you may require to reinitialize cagefs again:

cagefsctl –remount-all
cagefsctl -r

I have seen times, when nothing works, but reinstalling cagefs does the trick. If cagefs doesn’t work, you may try disabling virtual memory from the CloudLinux LVE manager to see if that fix the problem. CloudLinux also has a known Virtual Memory 503 error issue with LSAPI.