Red Hat Software Raid

This is a quick and dirty document on software raid there are many more documents on the web that go into greater detail, the following is covered in this document:

  • Creating a raid array with a hot spare
  • Checking the array
  • Simulating a drive failure
  • Removing a disk from an array
  • Adding a disk to the array
  • Extend the array
  • Starting stopping array

The following was tested using a centos 4 installation on Dell hardware, three partitions have already been created /dev/sdb10, /dev/sdb11 and /dev/sdb12 all are 1Gb in size.

Set the partition type

Set partition type

[root]# fdisk /dev/sdb
Command (m for help): t
Paratition number (1-12): 10

Partition ID (L to list options): fd
Command (m for help): w
Command (m for help): q

Note: repeat above for /dev/sdb11 and /dev/sdb12

fd = raid autodetect

Create partiton copy to new disk

create partition table of new disk

sfdisk -d /dev/sda | sfdisk /dev/sdb

Note: this is like the prtvtoc … | fmthard … command in Solaris

Create the array

Using mkraid 1. Update the configuration file /etc/raidtab with these lines of code

[root]# vi /etc/raidtab
raiddev /dev/md0
raid-level 1
nr-raid-disks 2
nr-spare-disks 1
persistent-superblock
chunk-size 4
device /dev/sdb10
raid-disk 0
device /dev/sdb11
raid-disk 1
device /dev/sdb12
spare-disk 0

2. Now make the RAID device md0 and create a filesystem on it

[root]# mkraid /dev/md0
[root]# mke2fs -j –b 4096 –R stride=8 /dev/md0

3. Now add entry to /etc/fstab and mount it:

Add line to fstab:
/dev/md0   /mirror1    ext3       defaults 1   2

Create the mount point:
[root]# mkdir /mirror1

Mount the mirror:
[root]# mount /mirror1

Using mdadm

mdadm -C /dev/md0 -l1 -n2 /dev/sdb10 /dev/sdb11 -x1 /dev/sdb12

-C   create an array
-l   the raid level (raid 1 in this case)
-n   Number of devices in raid (2 devices in this case)
-x   Number of spare disks in the raid (1 in this case)

mdadm configuration file

detail, examine and assemble detail = applies to the whole array which is currently active
examine = applies to the devices which are a component of the array
assemble = assemble the array using all devices (will use config file first then scan devices)
Commands
Saving the configuration

echo “DEVICE partitions” > /etc/mdadm.conf
mdadm –detail –scan >> /etc/mdadm.conf

Note: by default the system will look in /etc/mdadm.conf then /etc/mdadm/mdadm.conf

Lost the configuration file echo “DEVICE partitions” > /etc/mdadm.conf
mdadm –examine –scan >> /etc/mdadm.conf
mdadm –assemble –scan

Check the raid array

/proc/mdstat cat /proc/mdstat
lsraid lsraid –a /dev/md0
mdadm mdadm –detail /dev/md0
mdadm -D /dev/md0
Display device configuration mdadm –misc -D /dev/md0
mdadm –misc -E /dev/sdb11

Simulating a drive failure

To test the raid integrity you might want to simulate a disk failure again there are a number of ways to do this.

raidtools raidsetfaulty /dev/md0 /dev/sdb11
mdadm mdadm -–manage /dev/md0 –f /dev/sdb11

Remove a disk from the array

To remove a disk from the raid array use the following commands, the disk has to be faulty not faulted (see above) to allow this option to work, at this point the disk can be physically removed.

raidtools raidhotremove /dev/md0 /dev/sdb11
mdadm mdadm –manage /dev/md0 –r /dev/sdb11

Add a disk to the array

Adding a disk to the array could result in two outcomes, if the array is already degraded the new will be used to fix the fault (if a hot spare has not already been used), if not then the disk will be used as a hot spare.

raidtools raidhotadd /dev/md0 /dev/sdb11
mdadm mdadm –manage /dev/md0 –a /dev/sdb11

Extend/Shrink the array

grow ## First you need to change the configuration
mdadm –grow /dev/md0 -n3

## Then you can add the disks
mdadm –manage /dev/md0 -a /dev/sdb12

## Now you can resize the filesystem
resize2fs /dev/md0

shrink ## MAKE SURE EVERYTHING IS BACKED UP

## First fail the disk you want to remove,
mdadm –manage /dev/md0 -f /dev/sdb12

  1. Then alter the configuration, yes I know the grow actually shrinks the array????

mdadm –grow /dev/md0 -n2

## Now you can remove the disk
mdadm –manage /dev/md0 -r /dev/sdb12

Change the type from raid 1 to raid 5

It is possible to change the type of the array from a raid 1 to a raid 5, I would advise to backup first but it worked without any problems for me

change from raid 1 to raid 5 ## First you need to unmount any filesystem from the array
umount /array1

## Then you need to stop the array
mdadm -S /dev/md0

## Then recreate the array
mdadm -C /dev/md0 -l5 -n3 /dev/sdb10 /dev/sdb11 /dev/sdb12 -x1 /dev/sdb13

## you will get errors stating the below, select Y to carry on creating the array

mdadm: /dev/sdb1 appears to contain an ext2fs file system
size=51072K mtime=Tue Feb 8 10:30:06 2011
mdadm: /dev/sdb1 appears to be part of a raid array:
level=raid1 devices=2 ctime=Tue Feb 8 10:26:56 2011
mdadm: /dev/sde1 appears to contain an ext2fs file system
size=51072K mtime=Tue Feb 8 10:30:06 2011
mdadm: /dev/sde1 appears to be part of a raid array:
level=raid1 devices=2 ctime=Tue Feb 8 10:26:56 2011
Continue creating array? y
mdadm: array /dev/md0 started.

## Once the array has started then mount it again and check everything
mount /dev/md0 /array1

## check the array
mdadm -D /dev/md0

## Don’t forget to update the configuration file
cp /etc/mdadm.conf /etc/mdadm.conf_old
echo “DEVICE partitions” > /etc/mdadm.conf
mdadm –detail –scan >> /etc/mdadm.conf

Starting and stopping array

raidtools raidstop /dev/md0
raidstart /dev/md0
mdadm ## Make sure you have the configuration /etc/mdadm.conf file created
mdadm -S /dev/md0
mdadm -A -R /dev/md0

-S  stopping
-A  assemble
-R  starting

Remove the Array

Remove the array mdadm –stop /dev/md0
mdadm –remove /dev/md0

## Once you run the below command there is no going back
mdadm –zero-superblock /dev/sdb11
mdadm –zero-superblock /dev/sdb12

Restore the Array

Restore the array ## standard header for mdadm.conf file
echo “DEVICE partitions” > /etc/mdadm.conf

## get the information from the disks and append that information to mdadm.conf file
mdadm –examine –scan /dev/sdb1 /dev/sdb2 /dev/sdb3 >> /etc/mdadm.conf

## use the mdadm.conf file to start the array
mdadm -A -s

Persistant-superblock
When an array is initialised with the persistent-superblock option in the /etc/raidtab file, a special superblock is written in the beginning of all disks participating in the array. This allows the kernel to read the configuration of RAID devices directly from the disks involved, instead of reading from some configuration file that may not be available at all times.

Autodetection
Autodetection allows the RAID devices to be automatically recognized by the kernel at boot-time, right after the ordinary partition detection is done.