How to use a btrfs RAID on your proxmox server

Now that proxmox v4 is out, we finally have access to a recent kernel (v4.2). This makes running a btrfs RAID much more plausible.

How to use btrfs in proxmox

I prefer to always keep my system drive separate. So I use the default ext4 installation on a small SSD for the base installation.

For everything else, (images, ISOs, etc..) I use the btrfs RAID.

After installing proxmox, at the terminal (or over ssh) run lsblk and you should see something similar.

My server has 1 128GB SSD for the system and 5 1TB spinning disks for the RAID.

root@pve:~# lsblk  
NAME         MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT  
sda            8:0    0 111.8G  0 disk  
├─sda1         8:1    0  1007K  0 part
├─sda2         8:2    0   127M  0 part /boot/efi
└─sda3         8:3    0 111.7G  0 part
  ├─pve-root 252:0    0  27.8G  0 lvm  /
  ├─pve-swap 252:1    0  13.9G  0 lvm  [SWAP]
  └─pve-data 252:2    0  56.2G  0 lvm  /var/lib/vz
sdb            8:16   0 931.5G  0 disk  
sdc            8:32   0 931.5G  0 disk  
sdd            8:48   0 931.5G  0 disk  
sde            8:64   0 931.5G  0 disk  
sdf            8:80   0 931.5G  0 disk  

With a freshly installed proxmox server we are ready to create our RAID array.

Install btrfs-tools
apt-get install btrfs-tools  
Create Filesystem

Look at the output of lsblk to be sure which drives you want in your raid. Modify the command below to suit your hardware setup.

Then run mkfs.btrfs.

mkfs.btrfs -m raid10 -d raid10 /dev/sdb /dev/sdc /dev/sdd /dev/sde /dev/sdf  

-m raid10 -d raid10 Tells btrfs to use RAID level 10 for data and metadata.

If you have fewer than 4 drives you could also use -m raid1 -d raid1. Unlike traditional RAID you do not need exactly 4 disks for RAID10. You only need a minimum of 4 disks. Also, for RAID1 you only need a minimum of 2 disks. You can add as many beyond that as you want.

You should now have a working raid, congratulations! That was easy right?

Mount filesystem on startup

We need to create a directory to mount our raid. I personally call this /primary as it is my primary storage for proxmox.

mkdir /primary  
Run blkid
root@pve:~# blkid  
/dev/sda2: UUID="2349-0B0A" TYPE="vfat" PARTUUID="97fb7619-d854-4063-aa26-ec078bba7382"
/dev/sda3: UUID="QdJG3q-3kgR-K7FL-NXvr-Ll1H-nBY2-HV3je9" TYPE="LVM2_member" PARTUUID="3a8091d7-23c5-4988-8aad-df45f8b55750"
/dev/sdb: UUID="be8c8b76-cd39-44a6-adea-c8215251fd66" UUID_SUB="c05dc541-1dcc-4047-a567-1f56d719d904" TYPE="btrfs"
/dev/sdc: UUID="be8c8b76-cd39-44a6-adea-c8215251fd66" UUID_SUB="ea4d8a81-5c1e-472e-9b66-f6da293b404f" TYPE="btrfs"
/dev/sdd: UUID="be8c8b76-cd39-44a6-adea-c8215251fd66" UUID_SUB="d6f17437-9445-421c-a624-cbed334c207e" TYPE="btrfs"
/dev/sde: UUID="be8c8b76-cd39-44a6-adea-c8215251fd66" UUID_SUB="b1d675dd-5f3c-42df-8a3a-0a1003350193" TYPE="btrfs"
/dev/mapper/pve-root: UUID="ae77fdaa-2064-40da-aff4-e21567bb2b51" TYPE="ext4"
/dev/sdf: UUID="be8c8b76-cd39-44a6-adea-c8215251fd66" UUID_SUB="0b74e275-0ee6-4075-b83b-faf603cb2c6e" TYPE="btrfs"
/dev/mapper/pve-swap: UUID="524ff036-dd21-4e4e-83b4-e6d19387a1a7" TYPE="swap"
/dev/mapper/pve-data: UUID="058779b9-0614-47e4-af14-12d503e99a08" TYPE="ext4"
/dev/sda1: PARTUUID="dc0ca0c2-2127-43b0-81f0-0113d7b343fd"

This mostly looks like jargon. But the key thing is to get the UUID for our new btrfs RAID. You will notice that all of the disks that we added to our RAID have the same UUID. That is the value we want.

UUID="be8c8b76-cd39-44a6-adea-c8215251fd66"

Now that we have the UUID of our new filesystem, we can edit fstab.

nano /etc/fstab  

Add the following line to your fstab

UUID="<your uuid>"     /primary        btrfs   defaults,nodatacow,autodefrag,noatime   0       0  

The mount options are chosen specifically for running virtual machine images. The default btrfs mount options can have very poor performance for running virtual machines.

Add new storage to proxmox

Login to the proxmox web admin. and goto: Datacenter > Storage > Add > Directory

Add Directory Storage

For directory use whatever you created with mkdir. All other options are up to you.

Create new storage

Thats it! You now have a very affordable RAID with btrfs.