User Tools

Site Tools


raid10_on_linux

Create RAID10 on Linux

Your system needs to have packages mdadm and LVM2 installed:

sudo apt-get install mdadm lvms2

Reboot after installing mdadm, I ran into issues where mdadm couldn't access the block device of the empty drives.

In this example I use 2 separate spindles, e.g. /dev/sdb & /dev/sdc, with 160GB each.

Also, the following steps assume you create the separate volumes after the installation of the OS.

Prepare disks

sudo fdisk /dev/sdb
  • create new & empty partition table ('o')
  • create primary partition ('n', then 'p' and 1; accept default start and end sectors)
  • change type to “Linux raid auto” ('t', hex code is 'fd')
  • display table ('p')
  • write and exit ('w')

Create the RAID10 device

sudo mdadm -v --create /dev/md0 --level=raid10 --raid-devices=2 /dev/sdb1 /dev/sdc1

Check progress of the drive creation with:

cat /proc/mdstat

Make sure to update the /etc/mdadm/mdadm.conf to include the new array. Add the output of the following command in the line following: 'DEVICE partitions'

sudo mdadm --examine --scan

You'll end up with lines like this:

DEVICE partitions
ARRAY /dev/md0 level=raid10 num-devices=2 UUID=06ab9896:de85cbd1:12f1ed46:eb5cad4c

Create logical volume groups and volumes

This creates the future volumes for /home (80GB), /opt (30GB), /var (20GB) and /tmp (9GB)

sudo pvcreate /dev/md0
sudo vgcreate vg0 /dev/md0
sudo lvcreate -L80g -nlv_home vg0
sudo lvcreate -L20g -nlv_var vg0
sudo lvcreate -L30g -nlv_opt vg0
sudo lvcreate -L10g -nlv_tmp vg0

Get the summary of the devices that got created:

sudo  pvs
sudo  lvs

Create file systems

sudo mkfs.ext4 -t ext4 /dev/vg0/lv_home
sudo mkfs.ext4 -t ext4 /dev/vg0/lv_var
sudo mkfs.ext4 -t ext4 /dev/vg0/lv_opt
sudo mkfs.ext4 -t ext4 /dev/vg0/lv_tmp

Transfer contents to new volume

for each directory that needs to be moved, do the following steps:

  • mount new volume under a temporary mount point directory, e.g. /mnt/newfs
  • copy files, nested directories and links etc. to the new volume but use cpio to preserve links
  • unmount new volume
  • rename old top level directory, e.g. to /home_old
  • create mount point, e.g. /home
  • mount new volume
  • add mount point and volume to /etc/fstab

E.g. to move /home to the new logical volume 0 (/dev/vg0/lv_home):

sudo mount /dev/vg0/lv_home /mnt/newfs
cd /home
#run the actual copying command as su (not sudo)
su
find . -depth -print0 | cpio --null --preserve-modification-time --sparse -pvd /mnt/newfs/
exit
sudo umount /mnt/newfs
sudo /home /home_old
mkdir /home
sudo mount /dev/vg0/lv_home /home

Verify your freshly moved directory, check e.g. chmod and chown entries for your new ~/ directory with ls -l

For the /var and /opt folders, bring the system into single-user mode:

sudo /sbin/telinit 1

Cleanup after copying

cpio doesn't seem to get the owner/group right for the user's files/directories under /home. One option is to set the owner explicitly after copying (obviously, replace userA & groupA with the actual user and group names):

pushd /mnt/newfs/userA
sudo chown userA:groupA -R
popd

NB1: Google Chrome: After cpio copying the /home directory, 2 files need to have their owners explicitly reset. This is indicated if Chrome pops up a error message like this: “Your preferences cannot be read”. To remedy this:

sudo chown userA:groupA ~/.config/google-chrome/Local\ State
sudo chown userA:groupA ~/.config/google-chrome/Default/Preferences

Mount new volumes at boot time

add the following lines to /etc/fstab:

/dev/vg0/lv_home /home ext4 nodev,async 0 2
/dev/vg0/lv_opt  /opt  ext4 nodev,async 0 2
/dev/vg0/lv_var  /var  ext4 nodev,async 0 2
/dev/vg0/lv_tmp  /tmp  ext4 nodev,async 0 1
  • NB1: If you install Google Chrome, it is installed to /opt by default. Make sure to not set mount option 'nosuid' since the sandboxing of the browser requires suid. If 'nosuid' is set in /etc/fstab, the following error message will appear: “Failed to move to new PID namespace: Operation not permitted”

Reboot and cleanup

After reboot all the new volumes should be mounted, check with e.g.

df

Once satisfied, delete the old original of the directories, e.g.

rm -r /home_old

Managing logical volumes

Renamin a volume

Unmount, rename, update /etc/fstab with the new device path and remount. In this example, since it affects the /home partition, it's a good ide to be logged in as root and to bring the system into single user mode to log out any users and shutdown X and its display managers:

telinit 1
unmount /home
lvrename /dev/vg0/lv1 /dev/vg0/lv_home
vi /etc/fstab
fsck /dev/vg0/lv_home
mount /home

Extending a volume

Unmount the volume; if necessary bring system into single-user mode: Login as root on console1:

telinit 1
umount /home
fsck /dev/vg0/lv_home
lvdisplay /dev/vg0/lv_home

Ext2 filesystems need to be unmounted before resizing; technically, an ext3 could stay mounted, but there is less risk involved with the volume unmounted. the lvdisplay spew informs us to the current size. Next determine how much space is left in the parent volume group:

vgdisplay vg0

Now extent the volum and extend its file system.

lvextend -L+40G /dev/vg0/lv_home
fsck -f /dev/vg0/lv_home
resize2fs /dev/vg0/lv_home

The resize2fs will assume to extend to the full size of the logical volume, so no size parameter is necessary. Typically, the resize2fs will require a full fsck to have run before resizing.

Links

raid10_on_linux.txt · Last modified: 2014/11/01 14:14 (external edit)