Performance boost from two swap slices on separate SSDs

[Busker]

I just installed two (small, cheap) SSDs in my Mint 20.1 box and put a swap slice on the both of them, both with priority set to zero.

It smokes. It's not faster per se but what it does is let me run too many apps with too many browser tabs open at the same time (something I do habitually) without bogging down the system.

I've got my MoBo maxed out at 8GB of RAM, so there's no upgrade to be had there, short of replacing the MoBo. Before, whenever I was below 5-600MB RAM free (which was pretty much my normal state of affairs), responsiveness fell off considerably. And if I got below 400MB free, it would tend to freeze, sometimes permanently. Wouldn't even respond to a <Ctrl><Alt><Backspace> and had to be hard booted.

Yes, I had tried tweaking swappiness and all the memory settings in the browser ad nauseum, and I got no benefit to speak of from my experimenting. Low free RAM still meant laggy performance.

Then I bought a couple of small-ish (64GB), cheap SSDs. I benchmarked them and found their reads/writes were only about 20% faster than my existing HDDs. However, seek times were faster by 100x!!! Huzzah for solid state!

At first I only installed one, initially as the drive to host my swap slice, and then changed to use it as the drive hosting the /boot and root partitions. The latter shortened my boot times considerably but otherwise neither caused any significant improvement in performance when in a state of low free RAM.

So with the first SSD still hosting /boot and root, I thought to try to install the second SSD (which I had bought thinking to put it in a Windows PC but never got around to it) and dedicate the entire drive to a swap slice. That improved the low RAM problem somewhat but it also made me wonder what about two swap slices on two separate SSDs? Root and /boot were only occupying 27GB on /dev/sda so I gave over the remaining 36GB to a second swap slice.

That further improved my low RAM performance but as I was reading up on dual swap slices I found a reference to setting both to the same priority, which would cause the OS to write to the two swap slices in round-robin fashion, improving the net read/write performance. So I set the priority on both to zero and enabled write caching.

And it took off like a herd of turtles!

Now I spend most of my time with less than 500GB RAM free but with NO impact on performance. I don't know how low it can go but I've seen less than 200MB of RAM free and still no noticeable decline in performance. On the other hand I have seen both swap slices with 12GB written to them, so it's obviously that the OS is offloading the excess to the swap slices.



The display is misaligned because I had the terminal window set too narrow but the 12 million something something numbers are the "Used" figure and the priority (off the edge of the screen) is zero for both.

Ordinarily I would be mortified to find 12GB being written to swap, but in this case 12GBx2 just means that the plan is working. The reads and writes are happening so fast (in "human" time) I don't even notice any lag.

When I changed the first SSD from swap slice to hosting /boot and root I also did a clean install of Mint 20.1. Which, as you probably know, defaults the swappiness setting to 60. In the interest of extending the SSD's service life I thought to change swappiness to my customary 10, but after I did I found it wasn't running as smoothly when RAM was ridiculously low. So I put it back to 60 and left it.

Am I driving the SSDs to an early death with constant reading and writing? Don't know don't care. It's fixing a problem, one that's been plaguing me for a long time. If that's the cost of the fix, so be it. Besides, there's no data on either SSD that's irreplaceable (just root, /boot and swap) and I store snapshots of them in /home on the HDD at /dev/sdc anyway. So I see no down-side to working them to death other than some bit of down-time while I source and install a replacement, and the cost of those replacements.

But I'm not so sure I'm shortening their lives that much. First of all, swap on average is read from 7x for each write. On top of which, SSDs, I am given to understand, write in round-robin fashion as a means of wear leveling. And I have allocated swap slice space well in excess of what I ever forsee using specifically to use all that real estate for wear leveling to postpone them dying prematurely.

In the meanwhile, I can keep open 200 browser tabs while I'm watching Netflix if I feel like it, and it still runs like I'm not overloading it. Zero impact on performance.

So if you've got two SSDs lying around not earning their keep, and if you also like to keep a thousand browser tabs open, try the dual swap slice fix.

[Moonstone Man]

... I don't know how low it can go but I've seen less than 200MB of RAM free and still no noticeable decline in performance...

Just be aware that, unlike in Windows, in Linux, physical memory is divided into free memory, available memory, used memory, application memory, buffered memory, cached memory and allocated memory, and each are expressed as a component of total memory. This means that when you see 200MB of RAM free, you need to know which of those allocations subtracted from total memory is being displayed. That is to say, 200MB of free RAM is not necessarily the same as 200MB of free RAM, as difficult as that might be to get your head around. To make matters worse, the sum of used memory, application memory, buffered memory, cached memory and allocated memory can exceed total memory, so you can't place any stock at all in the notion that you have '200MB of RAM free' unless you know explicitly how that memory is divided up and being reported. This is one of the major criticisms I have of Linux Mint using the black box GNU/Linux system monitor; it is under-cooked so it does not provide the user with an accurate representation of what is actually going on.

The kernel will drop cached memory before it goes to swap. You can enhance the kernel's behaviour with memory by putting these lines in /etc/sysctl.conf

Code: Select all

vm.swappiness=10
vm.vfs_cache_pressure=50

... followed by a reboot.

[Busker]

The tweaks you offer (swappiness and cache pressure) are in my "must do" list for a fresh Mint installation. However, I don't get the uninterrupted smooth operation with swappiness set to 10.

[Busker]

I finally so overloaded the twin swap slices that Mint shuddered under the load. I had way, way too many browser tabs open, and was watching a video on a NAS (plus various and sundry other applications; Nemo, Notepadqq, SVNC, and a couple of terminal windows) and it finally bogged down.

This was why:



That's 60GB total written to the two swap slices. I check 'swapon' from time to time and had found as much as 26GB written to both slices, 52GB in total, and Mint still was humming right along like it didn't have a care in the world.

Paging always was a band-aid for too little RAM, but the speed of storage memory (or lack of it) made it a poor second. Thanks to SSDs, it's not so poor any more. It holds up under load like it had a ton more RAM than it does.

[rene]

Fairly interesting --- although I'd explicitly mention for others that if NOT maxed out getting more RAM (or fewer browser tabs) is still significantly better.

However, as the reason for replying: what you are doing is basically RAID0-ing your swap partitions, and in that sense, if you're up for starting new again you could actually RAID the SSDs (if you have BIOS RAID support at least; the Ubuntu/Mint installer works automatically with the usual BIOS raid these days AFAIA). Reason is that you would then also raid your Linux partition(s) alongside for even more sweet goodness.

[Petermint]

For anyone about to buy 2 SSDs, look at the performance of one larger SSD.

Assume you are looking at an SSD with the choice of 64 GB, 128 GB, 256 GB, and 512 GB. The 64 GB might be one storage chip using one channel of the SSD controller. The 128 GB might have two storage chips using two channels and have twice the write speed. One larger SSD, at twice the price of the 64 GB SSD, is often twice the performance. Check the specification pages for the model to find the relative speeds.

The other problem is putting swap on the same SSD as the system. They will slow each other down for many operations. The combined workload might heat up the SSD to the point of switching into slow mode. Check the temp after 5 minutes of intense use.

Using 64 GB on a 64 GB SSD usually means using older chips. Using 64 GB on a newer, larger SSD may mean using better chips and using less electricity for each block written. Your battery lasts longer and the SSD slows down less often from heat overload.

[Moonstone Man]

One larger SSD, at twice the price of the 64 GB SSD, is often twice the performance...

Er, no.

A SATA III SSD can't go any faster than 500MB/s no matter how big it is. The only ways to break that barrier are to go RAID0 SATA III SSD or move to NVMe, and still, a PCIe 3.0 NVMe cannot go faster than the combined speed of 4 x PCIe 3.0 lanes, again no matter how big it is. And the only way to break that barrier is to move to RAID0 PCIe 3.0 NVMe or move to PCIe 4.0 NVMe. Rinse, repeat.

The other problem is putting swap on the same SSD as the system. They will slow each other down for many operations.

Given that SSDs have a seek time of 0ms, and that the kernel is judicious about when it decides to write to swap, that claim is impossible to swallow. It's FUD to invoke imaginary impacts on SSDs that only affect HDDs.

Check the temp after 5 minutes of intense use.

How do you propose that to work on Linux kernel 5.4, which doesn't have the required sensors support? What application should the OP use to pump up the writes sufficiently to perform the test? How many write lives will the SSD lose after performing your proposed test?

Wrong on nearly all counts... including economies of scale:

One larger SSD, at twice the price of the 64 GB SSD

[Petermint]

A SATA III SSD can't go any faster than 500MB/s no matter how big it is.

Ok, name a 64 GB SSD that can maintain writes at 500 MB/s? Tens of thousands of SSD benchmarks prove what I said about writes.

It's FUD to invoke imaginary impacts on SSDs that only affect HDDs.

It's FUD to imply that seek time is the only relevant measurement.

How do you propose that to work on Linux kernel 5.4, which doesn't have the required sensors support?

Open the Disks program and look at the temperature in the SMART data.

economies of scale

A quick look online at the entry size of a popular SSD shows a price of $56. The double size SSD in the line, which is faster for writes, is only $75. That means I can use the $112 budget ( $56 * 2) to buy a larger SSD from a far faster model range. I get two speed wins.