btrfs-transcation at 100% CPU causes processes to hang for several minutes ...Which filesystem do you recommend for SSD?

[damian101]

I have the exact same behavior on my Arch Linux installation, and thanks to your last comment @ilwegesk I figured out that it isn't only the hourly snapshot deletion, but also the hourly snapshot creation that causes the issues for me. Creating or deleting a Btrfs system snapshot can take about 2 minutes on my system. The filesystem is on an SSD. No idea why it takes that long.

Edit: I disabled Btrfs qgroups in the Timeshift settings, and now creating or deleting a Btrfs system snapshot takes almost no time.

Edit 2: While manual snapshot creation and deletion through Timeshift happens instantly now, automatic snapshot creation and deletion seems to take just as long as before, more than 3 minutes combined, locking my filesystem during that period.

[vonMos]

Hi there!

I installed mint on a btrfs partition about six month ago. Every now and then, some processes hang for 1 to 5 minutes and then they all resume simultaneously. That's a bit annoying ... Not all processes do hang, though. My assumption is that processes that do disk IO hang, e.g. when I want to start glances or htop or journalctl, the terminal hangs until everything is fine again.

Recently (I had by chance glances running), I noticed that when this happens the btrfs-transaction process has very high CPU (around 100%).
As soon as this process terminates, all hanging processes resume instantly.
Even though btrfs-cleaner is running afterwards, also with very high CPU (still around 95%).

I have not much experience with btrfs yet, so I wonder if there is anything I could do to either work around or (better) to help fixing this (or both ).

Or would you recommend that I do a re-install with a different FS that is more stable? (ext4? Or which FS would you recommend for a SSD? xfs? zfs? f2fs??)
(as far as I read, btrfs is considered to be quite stable already as long as I don't use some of the fancy features (e.g. some RAIDs) and that it is used for production servers by some large companies ...)

I do enjoy the COW and the snapshot functionality and how it is integrated with timeshift, and wouldn't want to miss this if possible!

Code: Select all

System:    Kernel: 5.8.0-33-generic x86_64 bits: 64 compiler: N/A Desktop: Cinnamon 4.6.7 wm: muffin 4.6.3 dm: LightDM 1.30.0 
           Distro: Linux Mint 20 Ulyana base: Ubuntu 20.04 focal 
Machine:   Type: Convertible System: ASUSTeK product: ZenBook UX562FAC_UX562FA v: 1.0 serial: <filter> 
           Mobo: ASUSTeK model: UX562FAC v: 1.0 serial: <filter> UEFI: American Megatrends v: UX562FAC.304 date: 05/06/2020 
CPU:       Topology: Quad Core model: Intel Core i7-10510U bits: 64 type: MT MCP arch: Kaby Lake rev: C L2 cache: 8192 KiB 
           flags: avx avx2 lm nx pae sse sse2 sse3 sse4_1 sse4_2 ssse3 vmx bogomips: 36799 
           Speed: 1864 MHz min/max: 400/4900 MHz Core speeds (MHz): 1: 1864 2: 2932 3: 1816 4: 2652 5: 2191 6: 1339 7: 1313 
           8: 2514 
Drives:    Local Storage: total: 953.87 GiB used: 344.23 GiB (36.1%) 
           ID-1: /dev/nvme0n1 vendor: Samsung model: MZVLB1T0HALR-00000 size: 953.87 GiB speed: 31.6 Gb/s lanes: 4 
           serial: <filter> rev: EXA7301Q scheme: GPT 
RAID:      Hardware-1: Intel Device driver: ahci v: 3.0 port: 3060 bus ID: 00:17.0 chip ID: 8086.02d7 rev: N/A 
Partition: ID-1: / size: 304.74 GiB used: 125.74 GiB (41.3%) fs: btrfs dev: /dev/nvme0n1p6 
           ID-2: /home size: 304.74 GiB used: 125.74 GiB (41.3%) fs: btrfs dev: /dev/nvme0n1p6 
           ID-3: swap-1 size: 15.26 GiB used: 0 KiB (0.0%) fs: swap dev: /dev/nvme0n1p5 

XFS is the fastest... the only one faster is F2FS but is not available on LM

[LanceM]

XFS is the fastest... the only one faster is F2FS but is not available on LM

It is fully supported. Install from the software manager

Code: Select all

F2fs-tools

[damian101]

XFS is the fastest... the only one faster is F2FS but is not available on LM

I did some benchmark recently (some workload with heavy I/O) on HDDs and btrfs with nodatacow and noatime was consistently a few percent faster than XFS with noatime.

[jrw32982]

Edit 2: While manual snapshot creation and deletion through Timeshift happens instantly now, automatic snapshot creation and deletion seems to take just as long as before, more than 3 minutes combined, locking my filesystem during that period.

I can confirm that this happens for me too. I have turned off qgroups, but when my daily Timeshift happens (it always seems to pick midnight), my entire machine freezes for a couple of minutes. I am also using only SSDs. If I could shift the snapshot time to stick at 4am, I would live with it. Or better yet, it would be nice to fix the problem. I'm not sure if it is the old snapshot deletion that is causing the problem or the new snapshot creation, but it's very annoying. I'm getting to the point of considering turning off Timeshift.

[Larry78723]

@ilwegesk - Why are you doing hourly Timeshift snapshots? Are you making that many changes to the system or are you using it to also make snapshots of /home? If the second, it's the wrong tool for the job. I'd set up a cron job to backup just /home using TAR or rsync. Much less resource intensive that way.

[damian101]

Running

Code: Select all

sudo btrfs quota disable /

fixes the issue for me, at the cost of not being able to see the amount of unshared data a snapshot consumes. Option "Enable BTRFS qgroups" in Timeshift might need to be disabled as well.

[ilwegesk]

Running

Code: Select all

sudo btrfs quota disable /

fixes the issue for me, at the cost of not being able to see the amount of unshared data a snapshot consumes. Option "Enable BTRFS qgroups" in Timeshift might need to be disabled as well.

This seems to work for me: I also had to deselect the option to "enable BTRFS qgroups (recommended)" - otherwise Timeshift did re-enable the quotas again. The down-side is that now the size of the snapshots is no longer displayed. I think that's what Timeshift uses the quotas for (don't understand the internals).

I also disabled hourly snapshots.