Installing LMDE on BTRFS with lzo compression, almost

Questions about Grub, the liveCD and the installer
Forum rules
Before you post please read this

Installing LMDE on BTRFS with lzo compression, almost

Postby keantoken on Thu Nov 29, 2012 1:28 am

My computer crashed and I got a new SSD drive. I've been trying for ever and ever to get it to install with compression on. I finally discovered a way.

sudo mkdir /target
sudo mount /dev/sdb1 /target -t btrfs -o rw,ssd,space_cache,relatime,discard,compress=lzo
sudo mkdir /target/home
sudo mount /dev/sdb2 /target/home -t btrfs -o rw,ssd,space_cache,relatime,discard,compress=lzo

Change this to reflect the partition system you have on your drive. Above all KNOW what you're doing and WHY, or you'll just screw things up.
This works because the LMDE installer won't remount the partitions and instead writes to the already existing ones I just created with the special mount options. I don't know anyone else who has though of this.

Unfortunately this only partly worked, and I'm not sure if it has anything to do with the premounting.

The installer completes okay, and doesn't give any errors. Not in the UI at least. GTK always spits out some sort of error when opening a UI, I've learned to ignore it because no one seems to have ever fixed it in the last several releases of Gnome-based DM's.

Code: Select all
mint@mint ~ $ sudo mkdir /target
mint@mint ~ $ sudo mkdir /target/home
mint@mint ~ $ sudo mount /dev/sdb5 /target -t btrfs -o ssd,compress=lzo,space_cache,relatime,discard
mint@mint ~ $ sudo mount /dev/sdb4 /target/home -t btrfs -o ssd,compress=lzo,space_cache,relatime,discard
mount: mount point /target/home does not exist
mint@mint ~ $ sudo mkdir /target/home
mint@mint ~ $ sudo mount /dev/sdb4 /target/home -t btrfs -o ssd,compress=lzo,space_cache,relatime,discard
mint@mint ~ $ sudo pluma /etc/mtab

(pluma:5787): Gtk-WARNING **: Attempting to store changes into `/root/.local/share/recently-used.xbel', but failed: Failed to create file '/root/.local/share/recently-used.xbel.265DOW': No such file or directory

(pluma:5787): Gtk-WARNING **: Attempting to set the permissions of `/root/.local/share/recently-used.xbel', but failed: No such file or directory

(pluma:5787): Gtk-WARNING **: Attempting to store changes into `/root/.local/share/recently-used.xbel', but failed: Failed to create file '/root/.local/share/recently-used.xbel.1S2JOW': No such file or directory

(pluma:5787): Gtk-WARNING **: Attempting to set the permissions of `/root/.local/share/recently-used.xbel', but failed: No such file or directory
mint@mint ~ $ sudo /usr/bin/python2.7 -tt /usr/lib/live-installer/main.py $*

** (process:5800): WARNING **: Trying to register gtype 'GMountMountFlags' as enum when in fact it is of type 'GFlags'

** (process:5800): WARNING **: Trying to register gtype 'GDriveStartFlags' as enum when in fact it is of type 'GFlags'

** (process:5800): WARNING **: Trying to register gtype 'GSocketMsgFlags' as enum when in fact it is of type 'GFlags'
 ## INSTALLATION
 --> Installation started
EXECUTING: 'mkfs.btrfs /dev/sdb5'

WARNING! - Btrfs Btrfs v0.19 IS EXPERIMENTAL
WARNING! - see http://btrfs.wiki.kernel.org before using

/dev/sdb5 is mounted
 --> Mounting partitions
 ------ Mounting /live/image/casper/filesystem.squashfs on /source/
EXECUTING: 'mount -o loop -t squashfs /live/image/casper/filesystem.squashfs /source/'
 ------ Mounting /dev/sdb5 on /target/
EXECUTING: 'mount -t btrfs /dev/sdb5 /target'
mount: /dev/sdb5 already mounted or /target busy
mount: according to mtab, /dev/sdb5 is already mounted on /target
 ------ Mounting /dev/sdb3 on /target/boot
EXECUTING: 'mount -t ext4 /dev/sdb3 /target/boot'
 ------ Mounting /dev/sdb6 on /target/var
EXECUTING: 'mount -t reiserfs /dev/sdb6 /target/var'
 ------ Mounting /dev/sdb4 on /target/home
EXECUTING: 'mount -t btrfs /dev/sdb4 /target/home'
mount: /dev/sdb4 already mounted or /target/home busy
mount: according to mtab, /dev/sdb4 is already mounted on /target/home
 --> Indexing files
 --> Copying files
 --> Restoring meta-info
 --> Chrooting
 --> Removing live user
Removing user `mint' ...
Warning: group `mint' has no more members.
Done.
 --> Removing live-initramfs
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Package live-initramfs is not installed, so not removed
The following packages will be REMOVED:
  live-boot* live-boot-initramfs-tools* live-config* live-config-sysvinit*
  live-installer*
0 upgraded, 0 newly installed, 5 to remove and 0 not upgraded.
After this operation, 1,251 kB disk space will be freed.
(Reading database ... 168492 files and directories currently installed.)
Removing live-boot ...
Purging configuration files for live-boot ...
Removing live-boot-initramfs-tools ...
Removing live-config ...
Purging configuration files for live-config ...
Removing live-config-sysvinit ...
Purging configuration files for live-config-sysvinit ...
Removing live-installer ...
Purging configuration files for live-installer ...
Processing triggers for initramfs-tools ...
update-initramfs: Generating /boot/initrd.img-3.2.0-2-amd64
cryptsetup: WARNING: could not determine root device from /etc/fstab
Processing triggers for gnome-menus ...
Processing triggers for desktop-file-utils ...
Processing triggers for menu ...
Error connecting: Error connecting: Connection refused
 --> Adding new user
 --> Writing fstab
 --> Writing hostname
 --> Configuring MDM
 --> Setting the locale
Generating locales (this might take a while)...
  en_US.UTF-8... done
  en_US.UTF-8... done
Generation complete.
 --> Setting the timezone
 --> Localizing Firefox and Thunderbird
 --> Setting the keyboard
 --> Configuring Grub
 --> Running grub-install
Installation finished. No error reported.
 --> Running grub-mkconfig
Generating grub.cfg ...
Found background image: linuxmint.png
Found Debian background: linuxmint.png
Found linux image: /boot/vmlinuz-3.2.0-2-amd64
Found initrd image: /boot/initrd.img-3.2.0-2-amd64
done
 --> Checking Grub configuration
 --> Found Grub theme: if background_image /grub/linuxmint.png; then
 --> Found Grub theme: if background_image /grub/linuxmint.png ; then
 --> Found Grub entry: menuentry 'LinuxMint GNU/Linux, with Linux 3.2.0-2-amd64' --class linuxmint --class gnu-linux --class gnu --class os {
 --> Found Grub entry: menuentry 'LinuxMint GNU/Linux, with Linux 3.2.0-2-amd64 (recovery mode)' --class linuxmint --class gnu-linux --class gnu --class os {
 --> Cleaning APT
 --> Unmounting partitions
EXECUTING: 'umount /target/boot'
EXECUTING: 'umount /target/var'
EXECUTING: 'umount /target/home'
EXECUTING: 'umount /target'
EXECUTING: 'umount /source'
umount: /source: device is busy.
        (In some cases useful info about processes that use
         the device is found by lsof(8) or fuser(1))
 --> All done
 ## INSTALLATION COMPLETE
mint@mint ~ $


This line is particularly disturbing:

cryptsetup: WARNING: could not determine root device from /etc/fstab

When I reset the computer, nothing would boot. So I went into the livecd again, installed and ran boot-repair. It did some sort of repair and posted the log here:

http://paste2.org/p/2537311

I checked the UUIDs of the fstab file and they were legit. I really don't have enough experience to know, but it seems to me somewhere someone got confused about which root they should be running from.

After boot-repair, GRUB loads, gives me normal and recovery kernel options. I select normal kernel. It says something like "loading initial ramdisk image" which I'm not sure is normal. Then it says it /run doesn't exist, and after several complaints asks me to CTRL-D or enter root password. After doing the limited things I know how to do, I find that root is running from sda1, which is the drive GRUB was installed on but not my OS disk.

I would REALLY like to figure out what is wrong here and get compression running. Help?
keantoken
Level 1
Level 1
 
Posts: 14
Joined: Sat Jan 07, 2012 11:14 am

Linux Mint is funded by ads and donations.
 

Re: Installing LMDE on BTRFS with lzo compression, almost

Postby sobrus on Thu Nov 29, 2012 3:01 am

To boot LMDE with BTRFS, change last mount options in fstab of root filesystem to "0 0", because fsck will always fail and it won't boot (because there is no fsck yet).
To do it, you probably will have to use separate working linux system with text editor, probably with root access. Maybe LiveCD will do.

If it doesn't help, just install LMDE on btrfs using installer, and enable compression afterwards, on running system (in fstab).
Most of the system will be re-compressed during UP4->UP5 update OR if you want to be sure - use separate system, move all root directories elsewhere (except boot to avoid grub problems) and copy them back with compression enabled. It may be not very elegant solution, but I've done it this way and it worked.

Also, "compress" mount option can perform poorly. "compress-force" will usually give far better compression ratio, that was at least im my case.
And while using BTRFS remember to always use latest kernel, to improve stability and performance. Stock 3.2 shipped with LMDE, while very stable, is ancient nowadays.
Liquorix, which is 3.6 already (and will be 3.7 soon), will be most probably better for BTRFS system.

Hope this helps.
sobrus
Level 2
Level 2
 
Posts: 87
Joined: Thu Aug 30, 2012 1:36 am

Re: Installing LMDE on BTRFS with lzo compression, almost

Postby keantoken on Thu Nov 29, 2012 12:32 pm

Thanks! I will try that. I am running the LMDE livecd right now.

I've tried the copy back and forth way, but it didn't work. Even with the premounting the same thing happens.

I'm using btrfs for the "transparent compression" feature. I don't care if my compression ratio is not the best as long as I get good performance. If I want more compression I can mount part of the SSD to a new folder with other compression options enabled right?
keantoken
Level 1
Level 1
 
Posts: 14
Joined: Sat Jan 07, 2012 11:14 am

Re: Installing LMDE on BTRFS with lzo compression, almost

Postby keantoken on Thu Nov 29, 2012 1:22 pm

It worked! UDP5 is running right now.

It's a bit jerky, but I need to install graphics drivers and Liquorix before I can compare it to my previous system.
keantoken
Level 1
Level 1
 
Posts: 14
Joined: Sat Jan 07, 2012 11:14 am

Re: Installing LMDE on BTRFS with lzo compression, almost

Postby sobrus on Thu Nov 29, 2012 2:36 pm

Glad to hear that :)

I know about btrfs compression. I've beed using it for about a year or so.
But mounted with standard "compress" mount option it won't compress many (unfortunately compressible) files.
That's why forcing it may be better option, but YMMV.
sobrus
Level 2
Level 2
 
Posts: 87
Joined: Thu Aug 30, 2012 1:36 am

Re: Installing LMDE on BTRFS with lzo compression, almost

Postby keantoken on Thu Nov 29, 2012 3:32 pm

Okay, I have a new problem. UDP5 is taking FOREVER. It usually stops for several seconds while unpacking deb files. This can't be right. It is like the drive hangs for 10 seconds and then give max speed for half a second, then hangs for another 10 seconds.

I have AHCI enabled and the drive is mounted with the discard option. I don't know whether this is a motherboard issue. The bios froze the drive when I first installed it, and won't boot to it which is why GRUB is installed to a different drive, so it can boot there and then manually boot linux on the SSD.

The drive is a PNY Prevail Elite. It has a custom list of SMART attributes which must mean it has its own dedicated firmware so I wouldn't expect TRIM to be broken. If so, PNY's new contract with HP will be an epic fail, and probably all of their SSD's are being RMA'd right now, and will be repeatedly RMA'd until they release new firmware. I would not expect this level of crap from PNY with an enterprise class SSD.
keantoken
Level 1
Level 1
 
Posts: 14
Joined: Sat Jan 07, 2012 11:14 am

Re: Installing LMDE on BTRFS with lzo compression, almost

Postby sobrus on Fri Nov 30, 2012 2:51 am

apt get is doing some really heavy IO. Much more than for example opensuse's zypper.
My Corsair Voyager 3.0 thumbdrive (via USB2.0!) had no problems with zypper and compressed btrfs, yet it was waaay to slow for LMDE and apt-get.
So unfortunately - I am not surprised. LMDE update took some time (half an hour, if not more ) also on my Agility 4 and tuned EXT4 filesystem.

Maybe your drive is faster, but in my opinion btrfs is certainly slower than ext4 (probably more on ancient kernel).
Try adding ssd_spread instead of ssd, and noatime, nodiratime instead of relatime mount options.
But I suppose btrfs is to blame, as your drive should be fast.
sobrus
Level 2
Level 2
 
Posts: 87
Joined: Thu Aug 30, 2012 1:36 am

Re: Installing LMDE on BTRFS with lzo compression, almost

Postby keantoken on Sat Dec 08, 2012 7:32 am

I would have responded sooner, but this site doesn't notify by default.

I've run hdparm -Tt benchmarks on my partitions. Root gets 220Mb/s and /home gets 250Mb/s. I'm not totally sure why the difference as they have the same filesystem and mount options. These speeds are relatively slow since my computer is SATAII instead of III. dd gives me 355Mb/s

It's been running fine so far. What I really need for a benchmark is something that tests the SSD where it usually falls short, for instance during package installation. This is what needs improving.
keantoken
Level 1
Level 1
 
Posts: 14
Joined: Sat Jan 07, 2012 11:14 am

Re: Installing LMDE on BTRFS with lzo compression, almost

Postby sobrus on Mon Dec 10, 2012 3:41 am

My machine is old P35 SATAII too.
From my experience, while just as fast at linear access, btrfs is slower than ext4 during heavy random read/write operations.
That's why I'm not surprised. Moreover, btrfs with compression is copy on write system, that means performance can degrade as drive gets full and garbage collector kicks in.
But putting filesystem aside, make sure that:

- your partitions are properly aligned to 1MB (bad alignment = severe performance degradation and excessive disk wear)
- your I/O scheduler is noop (this can give you slight boost. sometimes even quite noticeable)
sobrus
Level 2
Level 2
 
Posts: 87
Joined: Thu Aug 30, 2012 1:36 am

Re: Installing LMDE on BTRFS with lzo compression, almost

Postby keantoken on Mon Dec 10, 2012 2:26 pm

I am running the Liquorix kernel, so I'm actually using the BFQ scheduler:

http://algo.ing.unimo.it/people/paolo/d ... esults.php

According to

sudo parted /dev/sdb align-check

All my SSD partitions are aligned optimally.

I have my /var partition in ReiserFS, although now I'm wondering why i did it.

So far everything is pretty normal, it's just the unpacking part of installation that causes problems. I could try another scheduler to see if it speeds this up, but I'd lose the responsiveness and productivity of the BFQ scheduler. Perhaps I should make a script to change the scheduler during installations.
keantoken
Level 1
Level 1
 
Posts: 14
Joined: Sat Jan 07, 2012 11:14 am

Re: Installing LMDE on BTRFS with lzo compression, almost

Postby sobrus on Tue Dec 11, 2012 5:08 am

no no, I'm talking about I/O scheduler, not CPU scheduler.
They are different things.
BFQ schedules your CPU time, while CFQ schedules your disk commands (despite similar name they are not related in any way).
And fortunately it is much easier to change :)

for example:
echo noop > /sys/block/sda/queue/scheduler
(if your SSD drive is sda).

LMDE uses tmpfs for various things like /var/run and /tmp - so there is no need for Reiser i think.
sobrus
Level 2
Level 2
 
Posts: 87
Joined: Thu Aug 30, 2012 1:36 am

Re: Installing LMDE on BTRFS with lzo compression, almost

Postby keantoken on Tue Dec 11, 2012 5:40 am

Yes, it's a real disk scheduler:

Code: Select all
$ cat /sys/block/sdb/queue/scheduler
noop deadline cfq [bfq]


I noticed /tmp was a 500MB tmpfs, but nothing else. I wonder if /var/run is supposed to be in tmpfs and I messed something up by partitioning it. The kernel gives me something like "/run/shm does not exist" when starting up. /run is a symbolic link to /var/run/.
keantoken
Level 1
Level 1
 
Posts: 14
Joined: Sat Jan 07, 2012 11:14 am

Re: Installing LMDE on BTRFS with lzo compression, almost

Postby sobrus on Tue Dec 11, 2012 6:03 am

oh, I though BFQ was only for CPU.
Anyway, SSD usually doesn't need any scheduling so selecting noop may help.

On my machine /var/run is certainly tmpfs (although I cannot guarantee I didn't made it. We should check how it is on fresh install, most probably it is this way).
And "/var/run/shm" is posix shared memory. This is used by for example pulseaudio and I bet it should be in RAM.
sobrus
Level 2
Level 2
 
Posts: 87
Joined: Thu Aug 30, 2012 1:36 am

Re: Installing LMDE on BTRFS with lzo compression, almost

Postby keantoken on Tue Dec 11, 2012 1:12 pm

I uninstalled PA and alsa in lieu of OSS4. Can't get youtube sound to work though.

If I create a /run directory as tmpfs, where do I put the commands? Can I ignore the folder contents and mount over it in /etc/fstab? I put a command to create the web cache folder in /tmp in rc.local but that didn't do anything. My /dev/shm folder is missing as well. I guess that could mess with sound perhaps.
keantoken
Level 1
Level 1
 
Posts: 14
Joined: Sat Jan 07, 2012 11:14 am

Re: Installing LMDE on BTRFS with lzo compression, almost

Postby sobrus on Wed Dec 12, 2012 2:32 am

Try installing fresh LMDE on Virtualbox and see how /var/run is mounted.

btw. using tmpfs for internet cache in my opinion defeats the purpose of having ssd.
There is no point of paying for fast disk, if you will be waiting for browser to redownload web pages again and again.
Your browsing experience will actually be worse.
And having few hunderds MB of RAM used by cache (useless once you quit browser) - is also quite pointless.

That said I know that firefox writes huge amount of data. I've disabled disk cache completely, but just because I do have home squid proxy server with its own cache.
sobrus
Level 2
Level 2
 
Posts: 87
Joined: Thu Aug 30, 2012 1:36 am

Re: Installing LMDE on BTRFS with lzo compression, almost

Postby penzoiders on Sun Feb 03, 2013 11:09 am

I've managed to successfully install LMDE on btrfs, check my blog post here:
http://penzoditutto.blogspot.it/2013/02 ... ebian.html

hope it helps.
works flawlessy for me on my SSD.
User avatar
penzoiders
Level 1
Level 1
 
Posts: 17
Joined: Sun Feb 03, 2013 10:17 am

Re: Installing LMDE on BTRFS with lzo compression, almost

Postby keantoken on Sun Feb 03, 2013 7:47 pm

My understanding is that you don't want to use the compress-force option because some files don't compress well and it will lower R/W speed. Is this not true? And I thought lzo compression was what you wanted for improved R/W performance?
keantoken
Level 1
Level 1
 
Posts: 14
Joined: Sat Jan 07, 2012 11:14 am

Re: Installing LMDE on BTRFS with lzo compression, almost

Postby penzoiders on Mon Feb 04, 2013 12:56 pm

keantoken wrote:My understanding is that you don't want to use the compress-force option because some files don't compress well and it will lower R/W speed. Is this not true?


compress-force will force btrfs to TRY to compress all data that has to be written on SSD, it's done by CPU and will not degradate performance in writing non compressible data because it will just not compress that data.

Will not lower performance but let you use more efficiently the chache of your SSD moving the "write barrier" more ahead.
If you notice, when writing a stream of data to a SSD,it will write fast at principle then it will write slower !here's write barrier!.. that's because it first writes on cache then on memory chips. If you compress data you will be able to store up to two times the data in cache: the result is that it will start to write slower much later = move write barrier ahead... sorry for my bad english, maybe I translated an Italian stream of thoughts. :D

keantoken wrote:And I thought lzo compression was what you wanted for improved R/W performance?


zlib is very light and will not weight 1% of total CPU and you will not even notice it (but it will improve your I/O goodness)

I found that the best mount options for SSD (to be put in fstab with btrfs) are these (tested in production enviroment)... I will update my blog too 'cause I forgot to write them all:

Code: Select all
noatime,ssd,compress-force=zlib,ssd_spread,discard,space_cache,recovery


these will improve performance (ssd,space_cache,noatime,compress-force=zlib), mantain performance over time (discard) and reduce fragmentation (ssd_spread) and enable autorecovery upon mount (recovery).

these are intended for use with single ssd setup and is better to set it once and then leave these options like that, required kernel 3.2 or newer to use all these options.
if you use raid btrfs configurations, moving disks and changing and testing, is better to use a more "default" setting.

hope it helps. this is what I know and I like to share with you!

BTRFS RULES!
User avatar
penzoiders
Level 1
Level 1
 
Posts: 17
Joined: Sun Feb 03, 2013 10:17 am

Re: Installing LMDE on BTRFS with lzo compression, almost

Postby keantoken on Mon Feb 04, 2013 5:19 pm

Thanks a lot for the clarification! Your english is better than some native speakers I know...

I will try this and see how it works. Based on all the benchmarks though, Lzo compression resulted in the best R/W speed? Or is Zlib better only because of a smaller write barrier? Is the write barrier more important?

I find I can't just edit /etc/fstab and then do "sudo -a -o remount" to apply the new drive parameters. I have to do the full mount command before the new parameters will show up in /etc/mtab. Maybe I'm doing something wrong?
keantoken
Level 1
Level 1
 
Posts: 14
Joined: Sat Jan 07, 2012 11:14 am

Re: Installing LMDE on BTRFS with lzo compression, almost

Postby keantoken on Mon Feb 04, 2013 5:26 pm

One of the things I was trying to do was to compress the entire linux installation to reduce disk space usage. This is what made it so difficult. You may add in your article that one needs to update the BTRFS mount parameters BEFORE installation so that the OS will install compressed, if that is the aim.
keantoken
Level 1
Level 1
 
Posts: 14
Joined: Sat Jan 07, 2012 11:14 am

Linux Mint is funded by ads and donations.
 
Next

Return to Installation & Boot

Who is online

Users browsing this forum: Google [Bot] and 2 guests