How to measure the sequential write/read speed of a Hard Disk or SSD?

Question: How to measure the sequential write speed of a hard disk or SSD?

Answer

We may use two ways. The most popular one is to use dd tools. For hard disks, we may not avoid using cache to measure the disk performance. As sequential writes can write the data to the cache, which is basically a DRAM and can perform excessively fast masking the original speed of the HDD. Let’s check how can we measure the HDD speed using dd bypassing HDD write cache:

dd if=/dev/zero of=/hddvz/testfile.img bs=1G count=1 oflag=direct

/hddvz is my HDD mount, oflag=direct instruct the dd to confirm writes to the disk, not just the cache before saying the write is completed.

If it’s a good quality hard disk alone or soft raid 1, you can get speed up to 157-161MBps:

[root@bd3 ~]# dd if=/dev/zero of=/hddvz/testfile.img bs=1G count=1 oflag=direct
1+0 records in
1+0 records out
1073741824 bytes (1.1 GB) copied, 6.82675 s, 157 MB/s
[root@bd3 ~]#

With a good hardware raid controller and raid 10, you may be able to reach rates like 320MBps.

There is something to remember, software raid mdadm is able to read simultaneously from two disks. You may have the evidence, with two dd one by one, both in background as following:

[root@bd3 ~]# dd if=/dev/zero of=/hddvz/testfile.img bs=1G count=1 oflag=direct &
[1] 12039
[root@bd3 ~]# dd if=/dev/zero of=/hddvz/testfile1.img bs=1G count=1 oflag=direct &
[2] 12040
[root@bd3 ~]# 1+0 records in
1+0 records out
1073741824 bytes (1.1 GB) copied, 11.1064 s, 96.7 MB/s
1+0 records in
1+0 records out
1073741824 bytes (1.1 GB) copied, 10.7155 s, 100 MB/s

[1]-  Done                    dd if=/dev/zero of=/hddvz/testfile.img bs=1G count=1 oflag=direct
[2]+  Done                    dd if=/dev/zero of=/hddvz/testfile1.img bs=1G count=1 oflag=direct

As you can see, both the request was able to reach 100MBps roughly while running in parallel.

Testing Read Speed Using hdparm

To check the read speed, you may use hdparm like the following:

[root@bd3 ~]# hdparm -Tt /dev/sda

/dev/sda:
 Timing cached reads:   40444 MB in  1.99 seconds = 20365.12 MB/sec
 Timing buffered disk reads: 396 MB in  3.06 seconds = 129.43 MB/sec
[root@bd3 ~]# hdparm -Tt /dev/sdb

/dev/sdb:
 Timing cached reads:   41006 MB in  1.99 seconds = 20649.43 MB/sec
 Timing buffered disk reads: 562 MB in  3.01 seconds = 186.81 MB/sec
[root@bd3 ~]#

How to connect CentOS Server to the Internet using USB Wifi Adapter?

I was recently working on an Asus RS300 server, installed with CentOS 7 minimal installation before placing it to the datacenter. I didn’t have any RJ45 cable to get this connected to the internet at home. I had a spare ‘Linksys wusb600n’ USB wifi adapter, that I wanted to try out. I connected it to the USB, and got the interface listed as wlp18s0b1 using ifconfig.

To connect this to wifi, we can use nmcli like the following:

Show the connection:

nmcli connection show

To connect to the wifi with the password, use the following:

nmcli dev wifi connect your-ssid password your-wifi-pass

Remember to replace ‘your-ssid’ with the wifi name of your and ‘your-wifi-pass’ with the password for your wifi.

Once you run the above command, this should get connected to the wifi. Now, you can see the connection details and up the device like the following:

nmcli connection show
nmcli connection up wlp18s0b1

Remember to replace ‘wlp18s0b1’ with the one you can see in ifconfig.

Good luck.

How to Use Memtest+ to Test Your RAM

If you suspect an issue with your system RAM, you should try to use a tool, that will write data in 100% of your RAM and let you know the errors it could get. One such tool available for linux is ‘Memtest+’

How to install memtest+ in CentOS 7

To install memtest+, run the following command in yum:

yum install memtest86+

Now, this will install memtest+ for you. But memtest+ runs the memory test at boot time. To accomplish that, you need to install memtest+ in grub. Memtest+ comes with a command, that does it for you. Run the following to do this:

memtest-setup

This would install the memtest for grub. But you would still need to remake the grub.cfg file for CentOS 7. To do that, run the following:

grub2-mkconfig -o /boot/grub2/grub.cfg

Now, reboot the server and select memtest from the boot screen. Memtest will automatically complete the process and let you know the result on that screen.

How to Set Timezone in Postgres Databases

Question: How to Set Timezone in Postgres Databases

Answer

Postgres allows you to set a timezone per database. To view the list of available timezones, you may use the following query from psql:

select * from pg_timezone_names;

To set a timezone for say a database called ‘inventory_report’, you may use the following query:

alter database inventory_report set timezone to 'Asia/Dhaka';

Tips:

To List Postgres Databases, you may use the following command from psql:

\l

To List Postgres Tables in a Database:

Connect to the database first:

\c my_prod

Now, list:

\dt

How to Allow Remote User Access in PostgreSQL

Configuration PostgreSQL is a bit tricky and complicated compared to MySQL for a few cases. One such case would be allowing remote user access in PGSQL. You might be interested to allow your home network to connect to your test or production database to run a few queries. To do that, if you are not using a tool like Percona Distribution for Postgres or a Cluster tool like Clustercontrol, it might get a bit complicated.

I have hence, set a list of steps, we need to do to achieve the goal.

Set PostgreSQL to Allow Remote Addresses

To do that, open up your postgresql.conf file. It is usually located under /var/lib/pgsql/9.6/data/postgresql.conf file. I am running PGSQL 9.6, hence the folder is in 9.6. If you are running any other version of PGSQL, it shall change based on the version.

nano /var/lib/pgsql/9.6/data/postgresql.conf

Find the directive ‘listen_address’ and uncomment it, if it is commented. Also, set this to wildcard like the following:

listen_address = "*"

Create a PostgreSQL User with Encrypted Password

Open up your psql console from the user postgres and run the query given below:

su - postgres
psql
create user new_user with encrypted password 'testu1234'

You should always use ‘with encrypted password’, as this would avoid storing your password in plaintext instead of md5.

Now, grant the user on the database, you would like it to access:

grant all privileges on database my_prod to new_user;

Add Remote Access for User in PostgreSQL

PostgreSQL manages a remote access file, to allow access from remote sources. The file is located in /var/lib/pgsql/9.6/data/pg_hba.conf. To allow remote access from an IP, we need to add a line in this file:

Edit the file in nano

nano /var/lib/pgsql/9.6/data/pg_hba.conf

Add the following line

host    all             new_user           27.147.176.2/32       md5

Here 27.147.176.2 is the IP from where I would like to access the database.

Restart PostgreSQL

Now, restart your PostgreSQL instance

systemctl restart postgresql-9.6.service

You should be now set.

Few Additional Things For Reference

  1. How to list the users of PostgreSQL from the console?

    From the Postgres console (psql), run the following command:
    \du
  2. How to change the password for a PostgreSQL user?

From the Postgres console (psql), run the following query:

ALTER USER username WITH ENCRYPTED PASSWORD 'password';

where username is the username you would like the password changed. Remember to use encrypted password keyword to save the password in md5.

3. How to change the user to a superuser?

You may use the following query:

ALTER USER new_user WITH SUPERUSER;

CENTOS 6 : YUM ERROR : ALL MIRROR URLS ARE NOT USING FTP, HTTP[S] OR FILE

If you are still using CentOS 6, and trying to update the system using the following command:

yum update -y

You probably end up with the following:

Loaded plugins: fastestmirror, refresh-packagekit, security
Setting up Update Process
Determining fastest mirrors
YumRepo Error: All mirror URLs are not using ftp, http[s] or file.
Eg. Invalid release/repo/arch combination/
removing mirrorlist with no valid mirrors: /var/cache/yum/x86_64/6/base/mirrorlist.txt
Error: Cannot find a valid baseurl for repo: base

The error is coming up because CentOS 6 has now hit the end of life, and the URLs are moved to centos vault. Now to update the CentOS, you would need to replace the URLs.

How to Fix

To fix the error, you need to replace your CentOS-base.repo file. Open up the following file using nano:

nano /etc/yum.repos.d/CentOS-Base.repo

Now, replace the full code using the following:

[base]
name=CentOS-$releasever - Base
# mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=os&infra=$infra
# baseurl=http://mirror.centos.org/centos/$releasever/os/$basearch/
baseurl=https://vault.centos.org/6.10/os/$basearch/
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6

# released updates
[updates]
name=CentOS-$releasever - Updates
# mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=updates&infra=$infra
# baseurl=http://mirror.centos.org/centos/$releasever/updates/$basearch/
baseurl=https://vault.centos.org/6.10/updates/$basearch/
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6

# additional packages that may be useful
[extras]
name=CentOS-$releasever - Extras
# mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=extras&infra=$infra
# baseurl=http://mirror.centos.org/centos/$releasever/extras/$basearch/
baseurl=https://vault.centos.org/6.10/extras/$basearch/
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6

Notice, how we replaced mirror.centos.org to valut.centos.org in the repo file.

Now, clean the yum, and update

yum clean all
yum update -y

You should be good to go now.

How to Backup/Dump and Restore PostgreSQL Databases

Here is the situation. We have a customer, who runs a POS backed by PostgreSQL. Customers want a second POS with the updated SQL till last night. Point to note, he didn’t want to use the Master/Replica or Master/Master replication, instead, a day earlier backup to be restored.

A similar situation is probably applicable for developers who are trying to work with staging and production environments and vice versa. In such a situation, you need a technique for dumping and restoring databases.

Backup/Dump Postgresql Databases

To dump the PostgreSQL database, you need to use pg_dump. First ssh to your server, change the user to Postgres, and then run the dump

su - postgres
pg_dump your_database > your_database_2021_13_11.sql

Here, your_database is the name of the database you are trying to dump.

Copy the SQL to the Remote Server

Once the dump is done, now you may copy the SQL file to the remote server. You may do that using rsync:

rsync -vrplogDtH --progress your_database_2021_13_11.sql [email protected]:/var/lib/pgsql/

Replace your_database_2021_13_11.sql with your SQL dump name and ‘ip.ip.ip.ip’ with the IP of your remote destination.

Restore the Dump

Now, ssh to the remote server, and continue with the following command:

# change to postgres user
su - postgres
# drop the old database
psql -c 'drop database your_database;'
# create database (replace your_username with the one you want to use for the database)
psql -c 'create database your_database with owner your_username;'
# restore database
psql your_database < /var/lib/pgsql/your_database_2021_13_11.sql

Once done, you should be able to use the new restored database. Good luck.

How to Add RAW Image into An Existing OpenVZ 7 VM

OpenVZ 7 supports both KVM VM and OpenVZ containers. For a customer, we were trying to import a CentOS 7 based KVM VM to OpenVZ 7 KVM VM. But the problem is, the VM was created in RAW format. But OpenVZ 7 KVM does not support RAW files for KVM, it supports QCOW2. Hence, we had to first convert the RAW image into a QCOW2 image first.

How to convert RAW KVM VM Image to QCOW2 VM Image

To convert raw to qcow2, you may use qemu-img kvm tool. This tool comes with the KVM setup. Let’s say your raw image is ‘harddisk.hdd.raw’, and you would like to produce a qcow2 image, called ‘harddisk.hdd’. To do that, you may run the following:

qemu-img convert -f raw -O qcow2 /root/harddisk.hdd.raw /root/harddisk.hdd.qcow2

Make sure, you are running this script from the place where you have the harddisk.hdd.raw stored.

Replace the disk in OpenVZ 7

In our case, our files are stored under /vz/vmprivate/. Under this folder of OpenVZ, you would find folders with VM id. In our case, it was under the following:

/vz/vmprivate/9d07cfef-42bf-4fbe-ac6a-7af9c85c9475

You can also see the list of VMs UUID, by typing the following command:

prlctl list --all

First, stop the VM

prlctl stop 9d07cfef-42bf-4fbe-ac6a-7af9c85c9475

Now, all you need to do, is to replace the ‘harddisk.hdd’ with the one we had converted

cd /vz/vmprivate/9d07cfef-42bf-4fbe-ac6a-7af9c85c9475
mv harddisk.hdd harddisk.hdd_backup
mv /root/harddisk.hdd.qcow2 harddisk.hdd

Now, start the VM, and you should be good to go

prlctl start 9d07cfef-42bf-4fbe-ac6a-7af9c85c9475

ERROR: cannot drop the currently open database – PostgreSQL

If you are trying to drop a database and getting an error like the following in PostgreSQL:

ERROR: cannot drop the currently open database

Then, the easiest technique to solve the problem is to restart your PostgreSQL server. I am using PGSQL 9.6, here was my command:

systemctl stop postgresql-9.6.service
systemctl start postgresql-9.6.service

If you are using PostgreSQL 13 or above, you have an option to drop the database with force now. You may drop the database using the following:

DROP DATABASE database_to_drop WITH (FORCE);

Replace ‘database_to_drop’ with the database you would like to drop.