How to create Software RAID 1 on Fresh NVMe Drives in CentOS/RHEL

Let’s say, you just installed two NVMe drives. That means, you currently have the following devices on your system:

/dev/nvme0n1
/dev/nvme0n2

Now, to use Raid 1 on these devices, you need to first partition them. If your devices are less than 2TB, you can use label msdos with fdisk. But I prefer gpt with parted. I will partition the disks using parted.

Open the disk nvme0n1 using parted

parted /dev/nvme0n1

Now, set the label to gpt

mklabel gpt

Now, create the primary partition

mkpart primary 0TB 1.9TB

Assuming 1.9TB is the size of your drive.

Run the above process for nvme1n1 as well. This will create one partition on each device which would be like the following:

/dev/nvme0n1p1
/dev/nvme1n1p1

Now, you may create the raid, using mdadm command as follows:

mdadm --create /dev/md201 --level=mirror --raid-devices=2 /dev/nvme0n1p1 /dev/nvme1n1p1

If you see, mdadm command not found, then you can install mdadm using the following:

yum install mdadm -y

Once done, you may now see your raid using the following command:

[[email protected] ~]# cat /proc/mdstat
Personalities : [raid1]
md301 : active raid1 sdd1[1] sdc1[0]
      976628736 blocks super 1.2 [2/2] [UU]
      bitmap: 0/8 pages [0KB], 65536KB chunk

md201 : active raid1 nvme1n1p1[1] nvme0n1p1[0]
      1875240960 blocks super 1.2 [2/2] [UU]
      bitmap: 2/14 pages [8KB], 65536KB chunk

md124 : active raid1 sda5[0] sdb5[1]
      1843209216 blocks super 1.2 [2/2] [UU]
      bitmap: 4/14 pages [16KB], 65536KB chunk

md125 : active raid1 sda2[0] sdb2[1]
      4193280 blocks super 1.2 [2/2] [UU]

md126 : active raid1 sdb3[1] sda3[0]
      1047552 blocks super 1.2 [2/2] [UU]
      bitmap: 0/1 pages [0KB], 65536KB chunk

md127 : active raid1 sda1[0] sdb1[1]
      104856576 blocks super 1.2 [2/2] [UU]
      bitmap: 1/1 pages [4KB], 65536KB chunk

unused devices: <none>

Here are a few key pieces of information about software raid:

  1. It is better not to use Raid 10 with software raid. In case the raid configuration is lost, it is hard to know which drives were set as stripe and which like a mirror by the mdadm. It is a better practice to use raid 1 as a rule of thumb with software raid.
  2. Raid 1 in mdadm doubles the read request in parallel. In raid 1, one request reads from one device, while the other request in parallel would read from the next device. This gives double read throughput when there is a parallel thread running. It still suffers from the write cost for writing data in two devices.

Unable to connect to any wifi with NetworkManager due to error: Secrets were required, but not provided

I was trying to connect to wifi from a CentOS 7 minimal installation using the following article:

But I started seeing the following error:

Unable to connect to any wifi with NetworkManager due to error: Secrets were required, but not provided

The reason the above error is showing is that NetworkManager is trying to reuse an existing connection, which shouldn’t happen. You need to delete the connection first.

nmcli con delete <SSID>

Now, you can connect with the following:

nmcli dev wifi connect <SSID> password <password>

Apache detected an error in the Rewrite config. httpd_ls_bak: Syntax error in -C/-c directive: Include/IncludeOptional: Could not open directory /usr/local/apache/conf.modules.d: No such file or directory Please try again. – Cpanel Error

You might see the following set of errors with Cpanel

When trying to remove the redirect from Cpanel

Apache detected an error in the Rewrite config. <pre>httpd_ls_bak: Syntax error in -C/-c directive: Include/IncludeOptional: Could not open directory /usr/local/apache/conf.modules.d: No such file or directory </pre> Please try again.

/scripts/rebuildhttpdconf generates an error like the following:

httpd: Syntax error in -C/-c directive: Include/IncludeOptional: Could not open directory /usr/local/apache/conf.modules.d: No such file or directory

Resolution

Previously, we reinstalled apache24 to solve the issue like the following:

yum reinstall ea-apache24

Although you may also reinstall the apache config runtime extension, that shall fix the issue as well.

rpm -e --nodeps --justdb ea-apache24-config-runtime.noarch
yum install ea-apache24-config-runtime.noarch

Now you may rebuild the httpd conf or remove the redirect from cpanel without any problem.

Unable to register authentication agent: GDBus.Error:org.freedesktop.PolicyKit1.Error.Failed: Cannot determine user of subject – Cyberpanel – CentOS 7

Error details:

After installing cyberpanel, you see the below error from shell when running any command:

** (pkttyagent:8559): WARNING **: 02:28:34.621: Unable to register authentication agent: GDBus.Error:org.freedesktop.PolicyKit1.Error.Failed: Cannot determine user of subject
Error registering authentication agent: GDBus.Error:org.freedesktop.PolicyKit1.Error.Failed: Cannot determine user of subject (polkit-error-quark, 0)

Resolution:

The error is appearing because cybepranel adds the following line in your fstab:

proc    /proc        proc        defaults,hidepid=2    0 0

This is hiding the /proc from your shell that uses polkit. To fix this, keeping the above mount rule, you need to add a group for polkitd user and assign that group to to access /proc. You may do this using the following:

groupadd nohidepoc
usermod -a -G nohideproc polkitd
mount -o remount,rw,hidepid=2,gid=nohideproc /proc
systemctl restart polkit
systemctl restart polkit (do this twice)

Once done, you may edit your fstab, and comment the old line, add the following line:

# Edit your fstab:
nano /etc/fstab

# comment the following line :
from : proc    /proc        proc        defaults,hidepid=2    0 0
to : # proc    /proc        proc        defaults,hidepid=2    0 0

# add the following line
proc /proc proc defaults,hidepid=2,gid=nohidproc 0 0

Good luck.

How to measure the sequential write/read speed of a Hard Disk or SSD?

Question: How to measure the sequential write speed of a hard disk or SSD?

Answer

We may use two ways. The most popular one is to use dd tools. For hard disks, we may not avoid using cache to measure the disk performance. As sequential writes can write the data to the cache, which is basically a DRAM and can perform excessively fast masking the original speed of the HDD. Let’s check how can we measure the HDD speed using dd bypassing HDD write cache:

dd if=/dev/zero of=/hddvz/testfile.img bs=1G count=1 oflag=direct

/hddvz is my HDD mount, oflag=direct instruct the dd to confirm writes to the disk, not just the cache before saying the write is completed.

If it’s a good quality hard disk alone or soft raid 1, you can get speed up to 157-161MBps:

[[email protected] ~]# dd if=/dev/zero of=/hddvz/testfile.img bs=1G count=1 oflag=direct
1+0 records in
1+0 records out
1073741824 bytes (1.1 GB) copied, 6.82675 s, 157 MB/s
[[email protected] ~]#

With a good hardware raid controller and raid 10, you may be able to reach rates like 320MBps.

There is something to remember, software raid mdadm is able to read simultaneously from two disks. You may have the evidence, with two dd one by one, both in background as following:

[[email protected] ~]# dd if=/dev/zero of=/hddvz/testfile.img bs=1G count=1 oflag=direct &
[1] 12039
[[email protected] ~]# dd if=/dev/zero of=/hddvz/testfile1.img bs=1G count=1 oflag=direct &
[2] 12040
[[email protected] ~]# 1+0 records in
1+0 records out
1073741824 bytes (1.1 GB) copied, 11.1064 s, 96.7 MB/s
1+0 records in
1+0 records out
1073741824 bytes (1.1 GB) copied, 10.7155 s, 100 MB/s

[1]-  Done                    dd if=/dev/zero of=/hddvz/testfile.img bs=1G count=1 oflag=direct
[2]+  Done                    dd if=/dev/zero of=/hddvz/testfile1.img bs=1G count=1 oflag=direct

As you can see, both the request was able to reach 100MBps roughly while running in parallel.

Testing Read Speed Using hdparm

To check the read speed, you may use hdparm like the following:

[[email protected] ~]# hdparm -Tt /dev/sda

/dev/sda:
 Timing cached reads:   40444 MB in  1.99 seconds = 20365.12 MB/sec
 Timing buffered disk reads: 396 MB in  3.06 seconds = 129.43 MB/sec
[[email protected] ~]# hdparm -Tt /dev/sdb

/dev/sdb:
 Timing cached reads:   41006 MB in  1.99 seconds = 20649.43 MB/sec
 Timing buffered disk reads: 562 MB in  3.01 seconds = 186.81 MB/sec
[[email protected] ~]#

How to connect CentOS Server to the Internet using USB Wifi Adapter?

I was recently working on an Asus RS300 server, installed with CentOS 7 minimal installation before placing it to the datacenter. I didn’t have any RJ45 cable to get this connected to the internet at home. I had a spare ‘Linksys wusb600n’ USB wifi adapter, that I wanted to try out. I connected it to the USB, and got the interface listed as wlp18s0b1 using ifconfig.

To connect this to wifi, we can use nmcli like the following:

Show the connection:

nmcli connection show

To connect to the wifi with the password, use the following:

nmcli dev wifi connect your-ssid password your-wifi-pass

Remember to replace ‘your-ssid’ with the wifi name of your and ‘your-wifi-pass’ with the password for your wifi.

Once you run the above command, this should get connected to the wifi. Now, you can see the connection details and up the device like the following:

nmcli connection show
nmcli connection up wlp18s0b1

Remember to replace ‘wlp18s0b1’ with the one you can see in ifconfig.

Good luck.

How to Use Memtest+ to Test Your RAM

If you suspect an issue with your system RAM, you should try to use a tool, that will write data in 100% of your RAM and let you know the errors it could get. One such tool available for linux is ‘Memtest+’

How to install memtest+ in CentOS 7

To install memtest+, run the following command in yum:

yum install memtest86+

Now, this will install memtest+ for you. But memtest+ runs the memory test at boot time. To accomplish that, you need to install memtest+ in grub. Memtest+ comes with a command, that does it for you. Run the following to do this:

memtest-setup

This would install the memtest for grub. But you would still need to remake the grub.cfg file for CentOS 7. To do that, run the following:

grub2-mkconfig -o /boot/grub2/grub.cfg

Now, reboot the server and select memtest from the boot screen. Memtest will automatically complete the process and let you know the result on that screen.

CENTOS 6 : YUM ERROR : ALL MIRROR URLS ARE NOT USING FTP, HTTP[S] OR FILE

If you are still using CentOS 6, and trying to update the system using the following command:

yum update -y

You probably end up with the following:

Loaded plugins: fastestmirror, refresh-packagekit, security
Setting up Update Process
Determining fastest mirrors
YumRepo Error: All mirror URLs are not using ftp, http[s] or file.
Eg. Invalid release/repo/arch combination/
removing mirrorlist with no valid mirrors: /var/cache/yum/x86_64/6/base/mirrorlist.txt
Error: Cannot find a valid baseurl for repo: base

The error is coming up because CentOS 6 has now hit the end of life, and the URLs are moved to centos vault. Now to update the CentOS, you would need to replace the URLs.

How to Fix

To fix the error, you need to replace your CentOS-base.repo file. Open up the following file using nano:

nano /etc/yum.repos.d/CentOS-Base.repo

Now, replace the full code using the following:

[base]
name=CentOS-$releasever - Base
# mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=os&infra=$infra
# baseurl=http://mirror.centos.org/centos/$releasever/os/$basearch/
baseurl=https://vault.centos.org/6.10/os/$basearch/
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6

# released updates
[updates]
name=CentOS-$releasever - Updates
# mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=updates&infra=$infra
# baseurl=http://mirror.centos.org/centos/$releasever/updates/$basearch/
baseurl=https://vault.centos.org/6.10/updates/$basearch/
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6

# additional packages that may be useful
[extras]
name=CentOS-$releasever - Extras
# mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=extras&infra=$infra
# baseurl=http://mirror.centos.org/centos/$releasever/extras/$basearch/
baseurl=https://vault.centos.org/6.10/extras/$basearch/
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6

Notice, how we replaced mirror.centos.org to valut.centos.org in the repo file.

Now, clean the yum, and update

yum clean all
yum update -y

You should be good to go now.

ERROR: Cannot open TUN/TAP dev /dev/net/tun: No such file or directory (errno=2) – LXC/LXD

If you are seeing the above error in LXC, you need to do two things.

  1. Make sure the LXC container is running on privileged mode.
  2. Run the following commands inside the container:
mknod /dev/net/tun c 10 200

Now, you may run the OpenVPN command to start the VPN client:

openvpn --pull-filter ignore redirect-gateway --config ovpn.ovpn

# assumming your vpn config file is ovpn.ovpn

failed to open db file /var/spool/exim/db/ratelimit: permission denied

Cpanel incoming mails are failing, with an error in the exim_mainlog as following:

failed to open db file /var/spool/exim/db/ratelimit: permission denied

The error is appearing due to some permission issues with the exim db or the files are corrupted. These files recreate when the exim restart. Hence, we can do the following:

# delete the db files
rm -rf /var/spool/exim/db/*

# restart exim
service exim restart

# fix permission of exim spool
chown -Rf mailnull.mail /var/spool/exim
chmod 0750 /var/spool/exim

You should be done now.