Page 1 of 1

Install Maya on BTRFS RAID1 - complete guide! No ext4!

Posted: Sun Jul 01, 2012 2:21 pm
by fraze
Update! 2012-07-13
I've been using this setup for a few weeks now and now believe that BTRFS sucks for SSDs. It's slow and locks up occasionally while the SSDs do massive amounts of copying (this is a COW filesystem after all), and runs out of space easily. Sort of. It's really just the funky way that BTRFS reports free space that encourages you to use it, thinking it's there, but BTRFS is really reporting twice as much as you've got. In other words, BTRFS has brain-damage that affects SSDs and you probably shouldn't be using it unless you really, really hate mdadm.


UPDATE 2012-07-16: After using this setup for less than one month, struggling with "ENOSPC" (error no space left on device) and deleting files to fix it, only to limp along for a few days before running out of space again, I must conclude that BTRFS is fit only for masochists in its current form. DO NOT USE THIS GUIDE. Go back to safe, usable xfs (or ext4) for /home and forget all about the following guide. In case anybody is wondering, here is the output of btrfs filesystem show:

Code: Select all

Total devices 2 FS bytes used 47.33GB
devid    1 size 101.56GB used 101.56GB path /dev/sda2
devid    2 size 101.56GB used 101.56GB path /dev/sdb2
That math doesn't add up. Somebody is lying.... and I think we know who! It's btr-fscking-s. I had to fix it by deleting some of my last remaining files and "rebalancing" /home. If you can handle this kind of BS then proceed. If you'd rather have a useful, usable system then back to mdadm for you.[/color]

Hi! After going through several days of troubleshooting, reading dozens of forums, making every mistake you can think of, I've refined the installation process of Mint 13 and conquered some obstacles to suit my particular needs. If your needs are similar to mine this guide will save you a TON of time.

Some background: I jumped ship from Ubuntu after they crapped Unity all over the place. Moved to Kubuntu 11.10 but the non-stop printer-configuration crashes after upgrading to 12.04 necessitated a new setup. Tried the LiveCD of Mint 13 and was hooked. Now here we are.

So, take a look at this list and decide if your situation is similar to mine - and if it is, get ready for some fun :)
1. RAID1 is a priority. I've got a couple of 128GB SATA6 SSDs as /dev/sda and /dev/sdb and want the / and /home directories mirrored.
2. No ext4 /boot partitions or other weird stuff. We're going to have a / partition and a separate /home partition. Nothing too fancy.
3. Trying out BTRFS! Snapshots, growable, shrinkable, built-in RAID, you get it all. Now it's got fsck too.
4. Encrypted /home/myusername directory.

Now, out of the box Mint 13 will mangle a couple important things, which appear to be installer bugs related to BTRFS. Can't do RAID1. Can't do /home on btrfs. Can't do an encrypted user directory on btrfs. We'll work around these problems and make it perfect.

Let's begin.

Re: Install Maya on BTRFS RAID1 - complete guide! No ext4!

Posted: Sun Jul 01, 2012 2:34 pm
by fraze
If there is ANY important data on the drives you're using, BACK IT UP!!!!! Don't come back until you have two empty (or erasable) drives.

Are your disks ready to be primed?
We'll partition the disks from a LiveCD of Mint 13. Boot up Mint 13 and start a terminal (shortcut CTRL-ALT-T).
In the terminal, gain root privileges:

Code: Select all

sudo -i
Because SSDs can be wiggly about deleting old partitions, zero out the initial bunch of sectors.
NOTE: THIS WILL WIPE OUT ALL DATA ON YOUR DRIVE.

Code: Select all

dd if=/dev/zero of=/dev/sda bs=2048 count=30
Now create empty partitions with gparted on /dev/sda of type “unformatted”

Code: Select all

gparted
I don't have any screenshots, but here's what you need to do. Create a new primary partition of type "unformatted" the size you want the root partition to be. In my case, 10000MB is easily enough for Mint 13 and all the programs I wish to install later.

Then create a primary partition of type "unformatted" to hold all your user's data. This is the /home partition. It'll probably be the remainder of your disk.
Since I have 16GB of RAM, no swap partition is needed. If you have 4GB RAM or less, make a tiny primary partition of type "linux swap" - 1GB is probably enough.
Now close gparted and go back to the Terminal.

In terminal, copy partition table from /dev/sda to /dev/sdb

Code: Select all

sfdisk -d /dev/sda | sed -e s/sda/sdb/ | sfdisk --force /dev/sdb
Now open GParted again to set the boot flags on /dev/sda1 and /dev/sdb1

Code: Select all

gparted
Right-click on your /dev/sda1 partition and choose Manage Flags. Then click the "boot" flag, and close.
Choose /dev/sdb and do the same for /dev/sdb1. Then close GParted.

Next we'll create a RAID1 filesystem using 2 disks, mirroring both data and metadata for full redundancy. In Terminal, type:

Code: Select all

mkfs.btrfs -m raid1 -d raid1 /dev/sda1 /dev/sdb1

Code: Select all

mkfs.btrfs -m raid1 -d raid1 /dev/sda2 /dev/sdb2
Next step: install mint 13 onto /dev/sda, create user with encrypted /home/userdir!

Re: Install Maya on BTRFS RAID1 - complete guide! No ext4!

Posted: Sun Jul 01, 2012 3:02 pm
by fraze
Now it's time to start up the installation.
The goal for this section: Start the installer but DO NOT FORMAT ANYTHING. Leave /dev/sdb alone.

When asked where to install Mint:
Choose /dev/sda1 and "Change" it - use as btrfs, and mount point is /
Choose /dev/sda2 and "Change" - use as btrfs, mount as /home

Force bootloader installation to /dev/sda even though it will probably default to /dev/sdb

The installer will automatically pick up any swap partitions, you don't need to worry about those.

Make sure your Terminal window is running in the background, then Install.

Re: Install Maya on BTRFS RAID1 - complete guide! No ext4!

Posted: Sun Jul 01, 2012 3:15 pm
by fraze
When installation starts (after pressing “Install Now” button) go to your Terminal window:

Code: Select all

btrfs device scan
This starts up the RAID and ensures your installation will be mirrored on both the /dev/sda and /dev/sdb drives.

If you have a large monitor, you may wish to have another terminal running to see the progress of your installation;

Code: Select all

sudo -i && watch btrfs filesystem show
Don't worry if the sizes don't match completely. They will match before we're finished.

Go through the installation but STOP at the point where you are asked to create the first user account. Otherwise, the installer will create the user’s home directory outside the Ubuntu-specific btrfs @home folder on the /home partition. (it’s an installer bug)

We will proceed to force the installer to create the user’s profile directory in the proper place instead by remounting the /home partition properly using the subvol option. If you don't, upon reboot you'll be unable to login because /home is mounted incorrectly, therefore there's no userdir present in the /home partition.

Go back to your waiting root terminal (the one with the # prompt) and unmount the incorrectly mounted /home directory

Code: Select all

umount /target/home
Now let's mount it properly.

Code: Select all

mount -t btrfs -o ssd,discard,noatime,subvol=@home /dev/sda2 /target/home
The only major difference is that we've specified a special folder on /dev/sda2 called @home which is specific to the btrfs filesystem in Ubuntu (and therefore also in Mint). I've added the ssd,discard options because we're using a SSD. Don't worry, it will still work even if your disks aren't SSD.

Now go back to the Install window and create the user, choosing “Encrypt my home folder” if you wish. Finish the installation.

When the installer is finished, it will unmount /target (your root partition) and /target/home (your /home partition) and give you the choice to “Continue Testing” or “Reboot Now”. Choose Continue Testing.

Re: Install Maya on BTRFS RAID1 - complete guide! No ext4!

Posted: Sun Jul 01, 2012 3:29 pm
by fraze
Now we can fix some mounting options before our newly installed Mint boots up for the first time.
In the Terminal running as root, with the # prompt:

Code: Select all

mount -t btrfs -o ssd,discard,noatime /dev/sda1 /target

Code: Select all

mount -t btrfs -o ssd,discard,noatime,subvol=@home /dev/sda2 /target/home

Code: Select all

btrfs device scan

Code: Select all

cd /target/etc && gedit fstab
Here's where we can change the default mounting options for our SSDs. UUID refers to your devices by a string of numbers and lower-case letters, and enables you to add or remove devices without worrying as much which SATA ports they are attached to. We prefer to use UUID because the /dev/sdX letters can change depending on which port the drive is attached.

Where it says "defaults,subvol=@" (the root partition) change to:

Code: Select all

defaults,noatime,discard,ssd,subvol=@
Where it says "defaults,subvol=@home" (the /home partition) change to:

Code: Select all

defaults,noatime,discard,ssd,subvol=@home
At the end of your fstab file, add these lines:

Code: Select all

## move /tmp to RAM
tmpfs	/tmp	tmpfs	relatime,nodev,nosuid,size=40%,mode=1777 0 0
Now close Gedit and go back to the Terminal:

Code: Select all

gedit /etc/sysctl.conf
Add these lines underneath "#kernel.printk = 3 4 1 3"

Code: Select all

## Reduce needless swap usage
vm.swappiness=1
vm.vfs_cache_pressure=50
This keeps your Mint from using swap unless it really, genuinely must.

Now, Mint has inherited a bug from the earliest days of Ubuntu - I remember this one from many years ago and it has persisted through the ages. If you disable antialiasing, Firefox will completely ignore your wishes. Only do this fix if you want to disable antialiasing (also known as "who smeared vaseline on my screen", or "fuzzy fonts") in all programs, for all users. Of course you can change it back later if you wish.
In the Terminal:

Code: Select all

gedit /etc/fonts/conf.d/10-antialias.conf
Change "true" to "false" to disable antialiasing in Chromium and Firefox, then close Gedit.

Now back to the ultra-important stuff!

Re: Install Maya on BTRFS RAID1 - complete guide! No ext4!

Posted: Sun Jul 01, 2012 3:31 pm
by fraze
Our next step is to balance the root filesystem; this ensures that the RAID1 is fully up to date and all metadata + data are mirrored correctly.
In Terminal:

Code: Select all

btrfs device scan
btrfs filesystem balance /target
btrfs filesystem balance /target/home
Wait until finished (won’t take long!), then:

Code: Select all

umount /target/home
umount /target
Next step: install GRUB2 onto /dev/sdb so we can still boot even if /dev/sda takes a dirt-nap.

Re: Install Maya on BTRFS RAID1 - complete guide! No ext4!

Posted: Sun Jul 01, 2012 3:36 pm
by fraze
Objective: Install grub onto /dev/sdb

Open Nautilus
click on one of the 10GB filesystems (or whatever size your root partition may be)
Did we get /dev/sdb1? Find out in Terminal:

Code: Select all

cat /etc/mtab
Look for /dev/sdb1 near the end of the list. If you see /dev/sda1 instead, click the "eject" symbol next to the drive in Nautilus and try again with the other 10GB filesystem.

Code: Select all

cat /etc/mtab
If we got /dev/sdb1 then in Nautilus, CTRL-L to display the full Location path
The path looks like: /media/XXXX where XXXX is the UUID of /dev/sdb1

Code: Select all

sudo grub-setup -d /media/XXXX/boot/grub /dev/sdb
Replace XXXX with the UUID of your /dev/sdb1 in the code above
Done! Now restart into our newly installed Mint 13 on a btrfs fully-mirrored array

Code: Select all

shutdown -r now

Re: Install Maya on BTRFS RAID1 - complete guide! No ext4!

Posted: Sun Jul 01, 2012 3:54 pm
by fraze
Post-Reboot
Start up a root terminal and show the RAID level in use:

Code: Select all

btrfs fi df /
btrfs fi df /home
rebalance / and /home (again) to ensure redundancy - probably unnecessary, but I'm paranoid and slightly superstitious. If you're paid by the hour, this is a required step:

Code: Select all

btrfs filesystem balance /
btrfs filesystem balance /home
Check the status of btrfs filesystems and RAID1 in another root terminal:

Code: Select all

watch btrfs filesystem show
Once RAID1 is verified working properly:
Upgrade the system and install your apps.
In a root terminal:

Code: Select all

synaptic
From Synaptic, you can enable the backports repository (useful) and add PPAs for special apps that aren't in the main repos. For instance, Handbrake has a good PPA. If you do use the PPAs within Synaptic, remember to edit them after adding and change "maya" to "precise".

Here is a list of the apps I find most useful:
FileZilla, Chromium, focuswriter, gparted, p7zip-full && p7zip-rar, geany, deluge (remove transmission first), pyrenamer, hugin, enfuse, enblend, calibre.

In Mint 13 fast user switching is broken. If you lock your screen, nobody else can use the computer until they enter your password or, as in my case, get frustrated and unplug the machine, thereby causing the loss of many hours of work. If your machine is meant for multiple users then you need to fix this ASAP.
In a root terminal let's fix Fast User Switching:

Code: Select all

ln -s /usr/bin/mdmflexiserver /usr/bin/gdmflexiserver
Install your extensions and/or applets the easy way: use a PPA. I really hope these are added to the main repos for the next release of Mint.

Another problem I seem to encounter on every Mint I install is non-working videos, or videos playing at super-fast speeds and no sound. There's an easy fix for this too: disable HDMI audio output if you encounter these symptoms.

Next step: massively speed up Firefox and Chromium!

Re: Install Maya on BTRFS RAID1 - complete guide! No ext4!

Posted: Mon Jul 02, 2012 12:18 am
by fraze
Speeding up Firefox is easy - just force it to cache in RAM instead of on disk.
Open up Firefox, in the location bar type about:config and hit Enter.
Type browser.cache and look at the list that comes up.

Double-click browser.cache.disk.enable to disable it - set to FALSE.
Right-click in an empty area below the list of settings and choose New -> Integer
Give it this name: browser.cache.memory.capacity
Set the value to -1 to enable automatic caching in memory based on the amount of RAM you have available.

Speeding up Chromium is even easier. Install chromium-browser, then find it in the Mint Menu and right-click, Add to Panel.
In the panel you'll see the new Chromium icon. Right-click it, Edit.
Change the Command portion from:

Code: Select all

/usr/bin/chromium-browser %u
(or whatever it is... I'm doing this from memory)
to this:

Code: Select all

/usr/bin/chromium-browser --disk-cache-dir="/tmp/ram/"
That's it! Hope you enjoyed it. Feel free to comment & I'll help sort out any issues I can.