Tag Archives: raspberrypi raspberry pi RAID RAID5

Raspberry Pi 2B RAID 5 build – Part 3

Alright lets continue with our stock Raspberry Pi – Jessie build. It’s a long post, hang on to the handle bars!

First, install mdadm using the following commands:

sudo apt-get update #Updates our repository list
sudo apt-get install mdadm #
Accept the prompt during the installation.

Second, we’ll assume the disks are intended for a built from scratch RAID 5 array, so we’ll destroy all data on the disk (technically all we’re doing is remove previous partition table and formatting information, it could still be recovered  with data recovery software, many available on the Internet).

We’ll also assume 3 disks. More can be used, but not less than 3 for a RAID 5 array, that’s a requirement. In my case I have 3 powered external USB disks directly plugged in to the Pi.

Third, identify the disks using lsblk -o NAME,SIZE,FSTYPE,TYPE,MOUNTPOINT 

lsblk

Here are my three disks, sda1, sdb1 and sdc1 (a through c denoting the sequence of the disks). My disks are partitioned already in the above picture (we’ll talk about that below) so they have numeric sequencing for their disks (the number 1 as all use the first partition). If they never partitioned then you’ll see sda, sdb, sdc and need to partition them before use.

Be very careful and verify that the disks identified are indeed disks that you intend to use! Once we proceed all previous data will be erased!

Fourth, we’ll start by partioning them using the command sudo parted. Once launched you’ll see a prompt with (parted). We’ll enter the following commands at this prompt. Start creating the partitions.

select /dev/sda
mklabel gpt
mkpart primary ext4 1049KB 3TB

select /dev/sda selects the first disk to be partitioned.
mklabel creates a partition table of type gpt (specifically needed for Linux and large multi Terabyte disks).
mkpart creates the partition. In this case I am starting at 1049KB (never mind the reasoning …) and ending at 3TB as that is the size of the disks I have (a 3TB external USB drive).

Proceed to the other disks (sdb and sdc replacing sda) the same way.
Enter quit to exit out of parted.

Fifth, we create the RAID 5 array using the following commands. This will prevent from the array created with spares (which we don’t want, we want to use all the disks for the maximum capacity we can get, we’re already losing 1/3 of the total disk space for parity information).

sudo mdadm –create –verbose –force –assume-clean /dev/md0 –level=5 \
–raid-devices=3 /dev/sda1 /dev/sdb1 /dev/sdc1

Check that they’re running with the following command

cat /proc/mdstat

You should see:

mdstat

Lets create the filesystem (this is what most resembles the Format command in Windows).

sudo mkfs.ext4 -F /dev/md0

You should see:

mkfs

It is now ready to mount so we can access it

sudo mkdir -p /mnt/raid5

You can now mount it using

sudo mount /dev/md0 /mnt/raid5

and see it’s content by

ls -al /mnt/raid5

mount

You can also confirm its capacity by (note /dev/md0 with 5.5T of size).

df -h -x devtmpfs -x tmpfs

mount

We’ll save the configuration in /etc/mdadm/mdadm.conf so it starts up at boot time automatically.

sudo mdadm –detail –scan | sudo tee -a /etc/mdadm/mdadm.conf

Update the initial file system (Raspberry Pi uses a RAM disk image when booting up and we want to include our array).

sudo update-initramfs -u

We’ll add the drive array to our filesystem table also so it will be mounted automatically when booting up.

echo ‘/dev/md0 /mnt/raid5 ext4 defaults,nofail,discard 0 0’ | sudo tee -a /etc/fstab

Finally confirm the array gets mounted by rebooting the Pi.

sudo reboot

You should see the following when it has booted.

ls

If “lost+found” entry isn’t listed there is more work to do (which I had to do with my Pi).

Edit /etc/rc.local file

sudo pico /etc/rc.local

Add the following line right before the line containing “exit 0”

/usr/local/bin/raid5 &

Then we create a shell script at /usr/local/bin/raid5

sudo pico /usr/local/bin/raid5

Enter the following lines in it

#!/bin/bash
# Sleep for 20 seconds after boot to make all the disks available for array assembly
sleep 20
sudo mdadm –assemble –scan # Assemble the RAID 5 array

Then set the execute permissions for the file

sudo chmod +x /usr/local/bin/raid5

Now reboot and the raid array should mount automatically. If it still doesn’t then add the following to /usr/local/bin/raid5

mount -a

Once mounted run the following command so I can start populating the /mnt/raid5 folder.

sudo chmod ug+wrx /mnt/raid5

That’s it!

Next I’ll setup samba (SMB Windows shares for Linux) and share out the folder so I can connect to it from other machines …

Next move on to part 4, the end of the journey: https://caesarsamsi.wordpress.com/2017/01/03/raspberry-pi-2b-raid-5-build-part-4/

Advertisements

Raspberry Pi 2B RAID 5 build – Part 2

Now we’re ready to do the semi difficult parts.

Write the Raspbian Linux OS image to the micro SD card. I use a Mac so I use the following instructions: https://www.raspberrypi.org/…/inst…/installing-images/mac.md

Note that when you are in the Mac’s System Report window, expand Hardware -> USB -> USB2.0 Hub -> USB 2.0 Hub then click on USB Storage. Or whatever you do, find that USB Storage node. In the details panel below on the right you’ll find “BSD Name”, in my case it is disk5, that’s the micro SD card we’re writing the image to (the Pi instructions missed this very tiny important detail).

system-reporthardware

If you use Windows then use the following instructions:
https://www.raspberrypi.org/…/…/installing-images/windows.md

For Macs, when you use the sudo dd command to write the image to the micro SD card, a progress update won’t be displayed. Instead press Control-T to see a status update when you want.

The process takes about 4.5 minutes on my Mac. It’ll be roughly the same on other computers as the time is dependent on the USB 2.0 port and microSD speed. I have a Class 10 micro SD card, which is pretty fast at the time this is posted.

Once you’re done with writing the Raspbian Linux OS image, boot up the Pi.

It is already preconfigured to provide X Window for a GUI display, and the network is preconfigured to use DHCP.

It’s now a ready to use Linux desktop, though I like the term tinypute myself

Next we’ll install and build the RAID 5 array itself …

On to Part 3 of the journey https://caesarsamsi.wordpress.com/2017/01/03/raspberry-pi-2b-raid-build-part-3/

Raspberry Pi 2B RAID 5 build – Part 1

First, you need a Raspberry Pi 2B computer. You can purchase it anywhere. It’s home is here: https://www.raspberrypi.org/

Remember to purchase a micro USB cable with a power charger which is used as the Pi’s power supply, also remember to purchase an HDMI cable.

The HDMI cable is used to display the Pi’s video output on an HDTV monitor (1080p). I’ve only used the HDMI display once to find the IP address of the Pi, after that I typically use the terminal/ssh to connect and work with the Pi.

In case they don’t come with the USB 2.0 cable (they should), also remember to buy 4 USB 2.0 cables.

Second, you need 4 USB 2.0 external hard disks. The Pi has 4 USB 2.0 ports so up to 4 disks can be connected directly. You can use more disks but you’ll have to use a USB hub. Using a hub to use more than 4 USB disks will degrade performance as a single USB 2.0 connection is shared amongst many disks.

Third, you’ll need a micro SD card, 8GB will do. This will be the Pi’s Linux OS hard disk.

Fourth, you’ll need to download the OS (it’s called Raspbian and is actually Linux in the Debian distribution form. Get it? Rasp(berry) and (De)bian .

You can download it from here: https://www.raspberrypi.org/downloads/raspbian/

Next we’ll write the image to the SD Card …

On to part 2 of the journey https://caesarsamsi.wordpress.com/2016/12/31/raspberry-pi-2b-raid-build-part-2/

Raspberry Pi 2B RAID 5 build – the Journey

This is my journey in building my own at home private cloud

It has no high availability from the processing node perspective as there is no fail-over/backup processing hardware to make this a highly available cluster. There is a solution to this as described in http://www.linuxjournal.com/content/two-pi-r?page=0,1 if you like the challenge. I might do that at another time.

This implementation only has disk high availability from 3 x 3TB USB 3.0 external disks turning this build into a nice NAS for my PCs, VMs, and laptops. We’ll see how slowly it crawls. It actually has decent performance for daily routines. Large file size & number copies will make it crawl. I just do that overnight (copied my iTunes library over).

Yes, I’m a digital pack-rat .

Go on part 1 of the journey! https://caesarsamsi.wordpress.com/2016/12/31/raspberry-pi-2b-raid-build-part-1-2/

pichristmas