- Innodb table can crash if there is power lose for your dedicated server.
- Force shutdown of MySQL service can end up in Innodb crash.
- Space issue on the serve can case Innodb crash, if the server space is 100% filled and mysql not able to write on the disk.
- High load on the server can cause Innodb crash.
- High I/O wait on the disk can also cause MySQL innodb crash.

How to determine if innodb is crashed.?
You can determine this by checking MySQL logs, On most of the servers location of the Log is /var/lib/mysql/HOSTNAME.err
You can also use commands:

systemctl status mariadb.service
journalctl -xe

Full MySQL backup
Backup MySQL users and permissions

pt-show-grants > /root/pt-show-grants.sql

Backup Full MySQL folder, Make sure MySQL is STOPPED for this step

rsync -a /var/lib/mysql /home/mysql_bkp/ 

Instructions for full mysql reinstall and recovery
If you can't start mysql, you can try by adding the following code in file /etc/my.cnf.d/server.cnf under [galera] settings

innodb_force_recovery = 1

Now try to start/restart MariaDB

systemctl restart mariadb.service

* If started successfully then you can start with backup, please note that MySQL is now in read-only mode.
* If not starting then try increasing recovery number for one number in each test

innodb_force_recovery = 2

innodb_force_recovery = 3

innodb_force_recovery = 4

innodb_force_recovery = 5

innodb_force_recovery = 6

When the MySQL is started in recovery mode we need to take backup of all the crashed databases and restore it

Create backup location folders

mkdir /home/mysql_innodb_recovery
mkdir /home/mysql_innodb_recovery/database_backup

Create the list of databases

mysql -e 'show databases;' | grep -v information_schema | grep -v Database  > /home/mysql_innodb_recovery/database_list.txt

* Please confirm that all databases are listed in the file /home/mysql_innodb_recovery/database_list.txt

Make backup of each databases including the database called mysql which is very important.

for db in `cat /home/mysql_innodb_recovery/database_list.txt`; do mysqldump $db > /home/mysql_innodb_recovery/database_backup/$db.sql;done

* this process can take a lot of time depending on your MySQL disk usage.
* Make sure you confirm that all .sql files in folder /home/mysql_innodb_recovery/database_backup/ are not empty.

Drop current databases (We can remove them if the backup was successful)

for db in `cat /home/mysql_innodb_recovery/database_list.txt`; do mysqladmin drop $db;done

Some database won’t get dropped which can be directly removed using the rm -Rf command

rm -Rf /var/lib/mysql/DATABASENAME

Next is to remove ibdata and ib_log files

rm -f ibdata* ib_logfile*

Restarting MariaDB
Remove recovery line innodb_force_recovery from /etc/my.cnf.d/server.cnf and restart MariaDB

systemctl restart mariadb.service

Creating Databases
If the MySQL now works we can start restoring databases, first create databases

for db in `cat /home/mysql_innodb_recovery/database_list.txt`; do mysqladmin create $db;done

Restoring Databases
Now we will restore the backups we have taken.

for db in `cat /home/mysql_innodb_recovery/database_list.txt`; do mysqldump $db < /home/mysql_innodb_recovery/database_backup/$db.sql;done

* This process will take some time, after it is done confirm that there are no any issues in the MySQL logs.

In case of some issues you still have the exact replica of databases in folder /home/mysql_bkp/
MySQL users and permissions are stored in file /root/pt-show-grants.sql
Individual databases backups are stored in .sql files in folder /home/mysql_innodb_recovery/database_backup/

* We suggest to keep all those files few days after the successful restore.

Verify if MariaDB 10.2 is installed

rpm -qa|grep -i maria|grep "\-10.2."

First you need to change the MariaDB repo and replace it with MariaDB 10.4 repo:

sed -i 's/10.2/10.4/g' /etc/yum.repos.d/mariadb.repo

Second remove MariaDB 10.2 :

systemctl stop mariadb mysql mysqld
systemctl disable mariadb
rpm --nodeps -ev MariaDB-server

Third Install MariaDB 10.4 :

yum clean all
yum -y update "MariaDB-*"
yum -y install MariaDB-server
systemctl enable mariadb
systemctl start mariadb

Fourth you need to upgrade your database tables to the latest version:

mysql_upgrade --force

After upgrade, you can use "mysql" command to verify the MariaDB version running on your server

mysql --version

* We recommend whitelisting IP in the firewall as it is a much safer option.
Example of whitelisting IP from the command line

csf -a 10.10.23.124 "mysql remote connection"

Temp allow IP for 24h

csf -ta 86400 10.10.23.124 "mysql remote connection"

Remote MySQL user in CWP is under Host.
You can choose from % Any or Specify Remote IP.

Also, make sure all your scripts/sites are compatible with the new version as there is no return back to an old version.

Check which version of MariaDB is installed on your server

rpm -qa|grep -i maria

MariaDB Repository Changes
Modify repo file to the version you want to upgrade
Edit File: /etc/yum.repos.d/mariadb.repo

[mariadb]
name = MariaDB
baseurl = http://yum.mariadb.org/10.2/centos7-amd64
gpgkey=https://yum.mariadb.org/RPM-GPG-KEY-MariaDB
gpgcheck=1

* Modify 10.2 with the version you want to upgrade
Example for upgrade to MariaDB 10.5

[mariadb]
name = MariaDB
baseurl = http://yum.mariadb.org/10.5/centos7-amd64
gpgkey=https://yum.mariadb.org/RPM-GPG-KEY-MariaDB
gpgcheck=1

Stop and remove the old version of MariaDB (Databases will not be removed)

service mysql stop
service mariadb stop
systemctl disable mariadb
rpm --nodeps -ev MariaDB-server

Install the new version of MariaDB

yum clean all
yum -y update "MariaDB-*"
yum -y install MariaDB-server
systemctl enable mariadb
service mariadb start
mysql_upgrade

Check for the custom configuration

diff -Bw /etc/my.cnf.d/server.cnf.rpmsave /etc/my.cnf.d/server.cnf

Verify upgraded packages

rpm -qa|grep -i maria

The output should look like this (10.5.15 should be replaced with your new version)

MariaDB-server-10.5.15-1.el7.centos.x86_64
MariaDB-devel-10.5.15-1.el7.centos.x86_64
MariaDB-common-10.5.15-1.el7.centos.x86_64
MariaDB-compat-10.5.15-1.el7.centos.x86_64
MariaDB-client-10.5.15-1.el7.centos.x86_64
MariaDB-shared-10.5.15-1.el7.centos.x86_64

Yum repository list of the available versions
http://yum.mariadb.org/

Verify if MariaDB 10.1 is installed

rpm -qa|grep -i maria|grep "\-10.1."

Upgrade

service mysql stop
service mariadb stop
systemctl disable mariadb
rpm --nodeps -ev MariaDB-server
yum -y install MariaDB-server
yum -y update "MariaDB-*"
systemctl enable mariadb
service mariadb start
mysql_upgrade

Check for the custom configuration

diff -Bw /etc/my.cnf.d/server.cnf.rpmsave /etc/my.cnf.d/server.cnf

Verify upgraded packages

rpm -qa|grep -i maria|grep "\-10.2."

The output should look like this

MariaDB-compat-10.2.36-1.el7.centos.x86_64
MariaDB-server-10.2.36-1.el7.centos.x86_64
MariaDB-common-10.2.36-1.el7.centos.x86_64
MariaDB-shared-10.2.36-1.el7.centos.x86_64
MariaDB-client-10.2.36-1.el7.centos.x86_64

1.Download Adminer from the downloads page

cd /usr/local/cwpsrv/var/services/
mkdir adm;cd adm

https://github.com/vrana/adminer/releases/download/v4.7.1/adminer-4.7.1.php

mv adminer-4.7.1.php adminer.php
cd /usr/local/cwpsrv/var/services/
chown cwpsvc:cwpsvc -R adm

2.Add below config to /usr/local/cwpsrv/conf/cwp_services.conf

location /adm {
    root /usr/local/cwpsrv/var/services;
    index  index.html index.htm index.php adminer.php;

    location ~ \.php$ {
        try_files $uri =404;
        fastcgi_split_path_info ^(.+\.php)(/.+)$;
        fastcgi_read_timeout 600;
        fastcgi_pass    unix:/usr/local/cwp/php71/var/sockets/cwpsvc.sock;
        fastcgi_index   index.php;
        fastcgi_param   SCRIPT_FILENAME  $document_root$fastcgi_script_name;
        fastcgi_param   SCRIPT_NAME   $fastcgi_script_name;
        fastcgi_param   PHP_ADMIN_VALUE "open_basedir = /usr/local/cwpsrv/var/services/adm/:/tmp/";
        include                 fastcgi_params;
    }

    location ~* \.(js|css|png|jpg|jpeg|gif|ico)$ {
        access_log    off;
        log_not_found    off;
        expires 1M;
    }
}

3. Restart cwpsrv

sh /scripts/restart_cwpsrv

4.login to Adminer

https://IP:2031/adm

ERROR 1040 (HY000): Too many connections
PHP Warning: mysqli_connect(): (HY000/1040): Too many connections

The default value for the maximum permitted number of simultaneous client connections is 150.

If you have sites with high peaks or high traffic then you can increase the limit.

Check the current connection limits

mysql -e "show variables like '%connection%';"

To increase the limits you can edit this file /etc/my.cnf.d/server.cnf
Add max_connections = 500 under [mysqld] line.

For the new values to get accepted you need to restart the mysql server.

service mysql restart

Now you can recheck again if the limits got increased.

Most common issues are that you have added invalid config into mysql config files.
You can check this from the command line by using one of the following commands

mysql --version
mysql -e "show databases;"

For MariaDB which is default with CWP please note that you need to add config into file
/etc/my.cnf.d/server.cnf under [mysqld]

Example from the ssh/shell

cat /root/.my.cnf
[client]
password=ydVg5g3Wz2FH
user=root

** MySQL root password in this case would be ydVg5g3Wz2F

Users can log in to MySQL/PhpMyAdmin by using the same login credentials as for the user control panel.
You can also make additional mysql users by using the mysql manager in the admin or user panel.

Export database info file

mysqldump DATABASE-NAME > FILE-NAME.sql

Export database into compressed file to save space

mysqldump DATABASE-NAME | gzip FILE-NAME.sql.gz

Export database into compressed file with date name to save space

mysqldump DATABASE-NAME | gzip > "FILE-NAME_`date +%e_%b_%Y`.sql.gz"

Import database

mysql DATABASE-NAME < FILE-NAME.sql
mysql mojtest_test1 < mojtest_test1.sql

Import compressed database .gz file

gunzip FILE-NAME.sql.gz
mysql DATABASE-NAME < FILE-NAME.sql
mysql mojtest_test1 < mojtest_test1.sql

How to create MySQL/MariaDB database and user

For importing very big databases maybe you will also need to add this into /etc/my.cnf.d/server.cnf under [mysqld]

wait_timeout = 86400
max_allowed_packet=256M

phpMyAdmin (MySQL Database manager)
php server configuration: /usr/local/cwp/php71/php.ini