ITCooky Recipes

Lets cooky it yammy things!

How to store critical data, at low cost, in Soft Mirror RAID in Ubuntu!

дата March 1, 2020

I have noticed one thing, I stopped paying attention to the best solutions in computer technology. Perhaps thanks to the understanding of own technical needs. Everything I have will soon turn 10 years old, some things have already fulfilled more! The shock and restlessness I had after installing Voodoo 2 and playing Quake II at 640×480 will no longer be possible, even with the 8K image. So I am updating something when, well, it has already died or should already die!

At home, the data traffic is not that large, so the server still works on SATA II. To store particularly valuable data, I used 2 HDDs in a duplicate software array, but they are already very old in 2008, that is also the reason why RAID works in mirror mode!

In 10 years, little has changed in domestic hard drives. The declared theoretical speed of SATAII is 300 megabytes per second, for SATA III it is 600 megabytes, in fact from SATAII to SATAIII, 390 gigabytes are copied for 4 hours, at a speed of 27 megabytes per second. At the same time, IT bloggers are assembling systems in PCIE4.0 with NVe disks, and desepting with the write speed of “only” 10 gigabytes, it should be 15-20 (hello Linus). For them it is crucial, of course as it has all video in 4k. Another example, the capacity of SD cards has grown a lot, but the speed has not. Recording an image on a 64 Gb card takes hours! So there are few real reasons to update your PC if you are not a video blogger or gamer, of course!

So I will have the most correct, greenest, greenest update! In order not to undermine our planet Earth with a waste of resources, I will squeeze out everything I can from the old hardware, while resorting to the help of Chinese and Japanese comrades (this should indicate how cheap, but the fact that the Japanese turned out to be more cheap surprised me!)

The old motherboard only has 2 SATA III ports that are already occupied, so I take here is a Chinese PCIE-SATA adapter in Marvell 88SE9125 ! You need PCI-E 2.0, I have one! How strange it works only in the PCI-E x1 slot (the smallest), in my opinion, it should have worked in both PCI-E x4 and PCI-E x16, but no, the system can’t even boot!

And I take the cheapest HDD of 1 Tb. Surprisingly, they are made by the Japanese TOSHIBA in the Philippines. Out of respect for the environment (here the truth is why, and not because of the empty pockets) I took 2.5 laptops, they use the metals with more caution (smaller than those of 3.5, although they are cheaper at the same time) and consume less energy . Even when you turn on the matrix with them, they growl hardly noticeably, unlike the old ones, they growl to full! I took the boxed 1 TB version Toshiba L200 Slim [HDWL110EZSTA] in that store www.dns-shop.ru for 3050 RUR , it turned out to be 50 rubles cheaper than without box (apparently because the boxes had new discs of October 2019).

To create an array in Ubuntu, you only need to execute a command.
sudo mdadm --create --verbose /dev/md0 --level=1 --raid-devices=2 /dev/sde /dev/sdf
/dev/md0 – It’s like a collective name for a matrix, but then it changes by itself and I don’t understand what it is for
level=1 – in mirror mode
/dev/sde /dev/sdf – two new discs

We see what is in the data array
cat /proc/mdstat
this

Personalities : [raid1] [linear] [multipath] [raid0] [raid6] [raid5] [raid4] [raid10] 
md0 : active raid1 sde[1] sdf[0]
      976630464 blocks super 1.2 [2/2] [UU]
      [>....................]  resync =  0.1% (1876672/976630464) finish=164.4min speed=98772K/sec
      bitmap: 8/8 pages [32KB], 65536KB chunk

We see that two discs are rolled in the matrix and that the synchronization has begun, although they are completely empty.

We wait almost three hours and at the end of the synchronization we create a file system!
sudo mkfs.ext4 /dev/md0

Now the array is called md0, but after rebooting it will be called md127 . In addition, some people register it in the mdm and fstab settings. I do not register it and it starts when I click on Ubunto. It’s good for me that it’s not mounted all the time, but only when I need it!

Here I pulse it when necessary and it immediately starts

To send status messages from the RAID array
An interesting feature of the program array in Ubuntu is not integrated in Ubuntu in terms of warnings: if the disk crashes, Ubuntiño will remain silent and, by the way, it will not start to recover it!

It is necessary to install the mail
For this we do
sudo apt-get install ssmtp
Customize the file /etc/ssmtp/ssmtp.conf
sudo vi /etc/ssmtp/ssmtp.conf
leave that text in the file, with our own data

root=username@gmail.com
mailhub=smtp.gmail.com:465
rewriteDomain=gmail.com
AuthUser=username@gmail.com
AuthPass=password
FromLineOverride=YES
UseTLS=YES

Now go to your gmail account myaccount.google.com/security and in Security activet Unreliable applications that have access to the account
That setting must be constantly checked, it often goes off by itself!

put in the md configuration file
sudo vi /etc/mdadm/mdadm.conf
change line

MAILADDR root

Run test
sudo mdadm --monitor /dev/md1 -t
Test letter comes immediately

You do not need to run it separately, it is already running!

Now we are doing a test, putting down one of the units of the old matrix md1 !
No messages appear! But Ubuntu begins to think for a long time when opening windows, although the data array itself works!

I turn on the disk again, restart.

I begin to monitor, here it is necessary to write which matrix
sudo mdadm --monitor /dev/md1
I receive a letter immediately
Headline

DegradedArray event on /dev/md1

We look at the state
cat /proc/mdstat
this

Personalities : [raid1] [linear] [multipath] [raid0] [raid6] [raid5] [raid4] [raid10] 
md127 : active (auto-read-only) raid1 sde[1] sdf[0]
      976630464 blocks super 1.2 [2/2] [UU]
      bitmap: 0/8 pages [0KB], 65536KB chunk

md1 : active raid1 sdd1[0]
      488252224 blocks super 1.2 [2/1] [U_]
      
unused devices: <none>

In the md1 entry, this is why [2/1] [U_] you can understand that something is wrong.

Let’s see in detail
sudo mdadm --query --detail /dev/md1
this

/dev/md1:
           Version : 1.2
     Creation Time : Sun Sep 13 14:03:58 2015
        Raid Level : raid1
        Array Size : 488252224 (465.63 GiB 499.97 GB)
     Used Dev Size : 488252224 (465.63 GiB 499.97 GB)
      Raid Devices : 2
     Total Devices : 1
       Persistence : Superblock is persistent

       Update Time : Sun Dec 29 09:30:06 2019
             State : clean, degraded 
    Active Devices : 1
   Working Devices : 1
    Failed Devices : 0
     Spare Devices : 0

Consistency Policy : resync

              Name : computer:1
              UUID : 930a7523:41a7e66d:086c7a74:9c300fdc
            Events : 9479

    Number   Major   Minor   RaidDevice State
       0       8       49        0      active sync   /dev/sdd1
       -       0        0        1      removed

You can see that there is a disk in the array. And here we see

Consistency Policy : resync

I have to execute resynchronization

The resynchronization does not start!

To do this, disassemble the matrix
sudo umount /dev/md1p1
to stop
sudo mdadm --stop /dev/md1
look discs status
sudo mdadm --examine /dev/sdb1
sacne for discs
sudo mdadm --assemble --scan -v
here at the end it says more or less clear

mdadm: Fail create md1 when using /sys/module/md_mod/parameters/new_array
mdadm: /dev/sdd1 is identified as a member of /dev/md/1, slot 0.
mdadm: /dev/sdb1 is identified as a member of /dev/md/1, slot 1.
mdadm: added /dev/sdb1 to /dev/md/1 as 1 (possibly out of date)
mdadm: added /dev/sdd1 to /dev/md/1 as 0
mdadm: /dev/md/1 has been started with 1 drive (out of 2).

By the way, here immediately try to connect arrays, after this command, you must also run
sudo mdadm --stop /dev/md1

Well, try to execute resynchronization
sudo mdadm --assemble --run --force --update=resync /dev/md1 /dev/sdd1 /dev/sdb1

The connection of a second disk does not occur

Fail create md1 when using /sys/module/md_mod/parameters/new_array

In short, a harmless shutdown of energy in a work data array threw out a second disk, but it didn’t kill the data array!

I try to connect it like new
sudo mdadm --manage /dev/md1 -a /dev/sdb1
and it worked, the synchronization starts lasting an hour and a half
cat /proc/mdstat
this

Personalities : [raid1] [linear] [multipath] [raid0] [raid6] [raid5] [raid4] [raid10] 
md1 : active raid1 sdb1[2] sdd1[0]
      488252224 blocks super 1.2 [2/1] [U_]
      [>....................]  recovery =  0.0% (267968/488252224) finish=91.0min speed=89322K/sec

So trying to do the tests, huh
Fio is not the one before

I do it differently, I make a large 50gb file
dd if=/dev/zero of=test.file count=50000 bs=1048576
copied it to ssd
pv ./test.file > /tmp/test.file

Here the results

Conclusion
1 For new HDD hard drives, although they are SATAIII, SATAII is enough
2 SSDs are faster in SATAIII, but not twice
3 PCIE-SATA in Marvell 88SE9125 works as SATAIII

I left the data array on the motherboard in SATA II as it shows the same copy speeds as the SATAIII on motherboard

In my old but extremely successful case AeroCool PGS Q Qs-200 , there is now more free space.


Leave a Reply

Your email address will not be published. Required fields are marked *