swap not working

[bousouf]

Hello,

the swap on my system is not working and I can't figure out why. I am running some large simulation and my computer can run out of ram, and since the swap is not being used this means a crash. When I run swapon -s, the swap partition is detected correctly, and it shows up in free -m or top. But it never gets used, even when I run out of memory.

Any idea ?
thanks

[Jerry]

Hi. Can you open up a terminal and post the output of

Code: Select all

cat /etc/fstab

there is a line that reads

UUID=(some number is here)

then continue in your terminal and post the output of

Code: Select all

sudo blkid

where the line that reads what dev/sda is (for swap) and is followed by a number which is then followed by TYPE="swap", compare the swap number of both. If one is different, replace the UUID from sudo blkid to the /etc/fstab file.
Ensure that the quotes are not included when replacing, by being in root, the UUID of the swap from sudo blkid to the /etc/fstab file. The /etc/fstab file will then be updated upon next reboot.

By the way, how much RAM does your machine have? If there's a lot of RAM, what happens is that LM will use the RAM as cache.

[all41]

If one is different, replace the UUID from sudo blkid to the /etc/fstab file.

If you modify fstab you can run:

Code: Select all

sudo mount -a

This will check fstab for entry errors. If no errors this will just return to a command prompt.
Then reboot.

[bousouf]

I already checked that the volume identifier was the right one, and just to be sure the file system was not corrupted I also regenerated the swap with mkswap and updated the UUID to no avail. I also tried using a swapfile instead of a partition, but once again it didn't work. I'll still list my fstab and some more info.

The machine has 16GB of ram, but I'm running large simulations and whatever is available will at some point be consumed. I'm hoping the swap would just act as a safeguard that would give me some time to shut down the program or something like that, instead of crash the whole system.


Here is my fstab

• # /etc/fstab: static file system information.
  #
  # Use 'blkid' to print the universally unique identifier for a
  # device; this may be used with UUID= as a more robust way to name devices
  # that works even if disks are added and removed. See fstab(5).
  #
  # <file system> <mount point> <type> <options> <dump> <pass>
  # / was on /dev/sda2 during installation
  UUID=bab45c12-9edc-4b2f-b8dc-a1d8de8b4a69 / ext4 errors=remount-ro 0 1
  # /boot/efi was on /dev/sda1 during installation
  UUID=B7CF-2052 /boot/efi vfat defaults 0 1
  # swap was on /dev/sda3 during installation
  UUID=7eaba7ce-b681-40fb-af1e-a93f509d8f0b none swap sw,pri=1 0 0
  # a swapfile is not a swap partition, no line here
  # use dphys-swapfile swap[on|off] for that

and the blkid

• /dev/sda1: UUID="B7CF-2052" TYPE="vfat"
  /dev/sda2: UUID="bab45c12-9edc-4b2f-b8dc-a1d8de8b4a69" TYPE="ext4"
  /dev/sda3: UUID="7eaba7ce-b681-40fb-af1e-a93f509d8f0b" TYPE="swap"
  /dev/sdb1: LABEL="reykjavik_extern" UUID="c7cc0285-a02f-4a5c-ba3d-473602e40708" TYPE="ext4"

and free -m

• total used free shared buffers cached
  Mem: 16026 7600 8426 140 187 4196
  -/+ buffers/cache: 3216 12809
  Swap: 16361 0 16361

they all indicate the swap should be working fine ...

[niowluka]

Post output of

Code: Select all

cat /proc/sys/vm/swappiness
cat /proc/sys/vm/vfs_cache_pressure

Are you running a custom kernel ?

[bousouf]

I haven't touched the kernel, and I played around with the swappiness which might have been what started the whole thing, I'm not sure. The swap used to work (a few days ago for sure) and in between I tried lowering the swappiness. Since then, I set it back to 60 and even higher values but no luck. I've been calling

Code: Select all

 sysctl vm.swappiness=## 

to do so.

The /proc/sys/vm/swappiness contains

• 60

and the /proc/sys/vm/vfs_cache_pressure

• 100

Note that I also read somewhere online that someone had problems with his swap because the priority in the fstab was default, i.e. -1, and that setting it to 1 fixed his problem. It didn't change anything in my case.

[niowluka]

Priority is only useful when you have more than one swap, it indicates which swap should be preffered by the kernel.

It seems your swap is properly detected and mounted, just kernel decides not to use it. swappiness and cache pressure values look OK, and if you didn't set them up in /etc/sysctl.conf then any changes would not be persistent anyway.

Try removing the pri=1 from fstab. It's not needed when you have only one swap, and it just might be what's confusing the kernel.

[bousouf]

I already tried that, but I'll give it another try.
Also I did update the /etc/sysctl.conf adding a value for the swappiness, but it's 60 now so it should not impact anything. I'll try removing it to see.
At some point when trying to fix my problem I was using a swap file, with or without the swap partition, it did not work either .... could it be that the ram is getting filled to fast for the kernel to react adequately ?

[niowluka]

At some point when trying to fix my problem I was using a swap file, with or without the swap partition, it did not work either .... could it be that the ram is getting filled to fast for the kernel to react adequately ?

If you had some extreme values for swappiness and cache pressure then yes, potentially. With defaults it shouldn't though.

[Petro Dawg]

# swap was on /dev/sda3 during installation
UUID=7eaba7ce-b681-40fb-af1e-a93f509d8f0b none swap sw,pri=1 0 0

I'm by no means a swap expert, but I have fixed a broken swap configuration a time or two on my systems. Have you tried getting rid of the ",pri=1" in the swap line of your fstab file? I doubt it is needed and perhaps it is causing an issue.

Also, using the "System Monitor" application; does Swap usage read "0 bytes (0.0%)" or does it say something like "Not Available"?

[bousouf]

Sorry for the late response, I didn't have access to the computer yesterday.
I tried removing the ",pri=1" again, but same thing ... And the System Monitor (or top, or "free -m", or "swapon -s") all show that the swap is available and should be working properly. My computer is still freezing without trying to use the swap when the ram gets filled up.

[Petro Dawg]

From what I understand the swap space is just for inactive memory pages. Perhaps when the simulator is running there is no inactive memory pages to move to the swap space (all memory is actively being used at one time), or the simulator is trying to use more active RAM than the total RAM installed. Might be time for a memory upgrade if possible.

Does the SWAP partition get used any during normal heavy use, perhaps after streaming video and putting computer to sleep and reawaken? Try to get your RAM usage as high as possible without hitting the 100% mark for a while and then check if it ever uses SWAP.

[kurotsugi]

and the /proc/sys/vm/vfs_cache_pressure

100

it is the culprit.

and free -m

total used free shared buffers cached
Mem: 16026 7600 8426 140 187 4196
-/+ buffers/cache: 3216 12809
Swap: 16361 0 16361

swap in general is slowing down your sytem. you shouldn't use it unless for hibernating. since you have 16gb ram you don't need swap.

[bousouf]

kurotsurgi: as I explained, in this case I would like to use the swap as a safeguard to prevent my computer from freezing when the ram gets filled up, and it does get filled up. And of course I can't avoid it, it's linked to some rather large simulations and the tests I need to do require a lot of memory.
What do you mean by "vfs_cache_pressure is the culprit" ?

Petro Dawg: I thought the swap was used as a (very) slow extension of the ram for extreme cases. At least, I have used it like that in the past for similar simulations on other computers, and on this one too until recently. I tried getting very high ram usage (96%) and the swap was still empty.

[niowluka]

and the /proc/sys/vm/vfs_cache_pressure

100

it is the culprit.

No it isn't. 100 is the default value and should not cause the system to run out of memory and not use swap. Setting 0 could cause the system to run out of memory, but should not prevent from using swap anyway.

Frankly I'm stumped. How quickly does ram fill up ? Is it hours, minutes, seconds ? How much total memory do you need for the simulations ?

[bousouf]

The ram fills up in under a minute, and I'm doing convergence tests on a numerical solver, so whatever memory is available will be the limit (that's why it fills up all the way ... and a little more, I'm using as much as I can).

In the past I have observed simulations going over the maximum available memory and requiring swapping, but this one fills up the memory particularly quickly. Maybe that's what's causing the OS some trouble ? What surprises me is that I'm pretty sure this exact test was working a few days ago (and using swap I assume, I didn't pay attention when running it) and working, and now it doesn't anymore ... the only thing I can think of that I did was play with the swapon / swapoff and swappiness value, but now that everything is back to defaults it should work the same.

[bousouf]

A small update, I maintained the memory usage above 97% for a minute or so by streaming a ton of stuff, and eventually the OS started using the swap. So the swap is working I guess, I still have no idea why the simulation causes a freeze instead of a slow down through swap usage, maybe it's just filling up the memory too fast.

Well thanks for all the help, if you have any more idea that would be great, otherwise I'll just try to run those tests on machines with a larger memory.

[kurotsugi]

taken from kernel documentation

vfs_cache_pressure
------------------

This percentage value controls the tendency of the kernel to reclaim
the memory which is used for caching of directory and inode objects.

At the default value of vfs_cache_pressure=100 the kernel will attempt to
reclaim dentries and inodes at a "fair" rate with respect to pagecache and
swapcache reclaim. Decreasing vfs_cache_pressure causes the kernel to prefer
to retain dentry and inode caches. When vfs_cache_pressure=0, the kernel will
never reclaim dentries and inodes due to memory pressure and this can easily
lead to out-of-memory conditions. Increasing vfs_cache_pressure beyond 100
causes the kernel to prefer to reclaim dentries and inodes.

Increasing vfs_cache_pressure significantly beyond 100 may have negative
performance impact. Reclaim code needs to take various locks to find freeable
directory and inode objects. With vfs_cache_pressure=1000, it will look for
ten times more freeable objects than there are.

I also use some information from here https://rudd-o.com/linux-and-free-softw ... o-fix-that

Values close to 100 provide no gain. Values close to zero can cause huge swap activity during big filesystem scans.

in my test with 100 value as long as there's a space on ram for creating new pagecache then your system will use it instead of using swap. I've tried several value of vm.swappiness but even if i set it on 90 my system won't use the swap partition. I even lowered the value of vm.vfs_cache_pressure to 50 but my swap partition still not used.

instead of using swap I'd recommend using zramswap. it work just like swap. the difference is that the swap partition is created inside your ram. it is lot faster and more HDD friendly.

[niowluka]

A small update, I maintained the memory usage above 97% for a minute or so by streaming a ton of stuff, and eventually the OS started using the swap. So the swap is working I guess, I still have no idea why the simulation causes a freeze instead of a slow down through swap usage, maybe it's just filling up the memory too fast.

That might be the reason then. I don't have swap, so I am not sure how early it gets normally used...

You can try setting swappiness to 100, although that's not a great value for the rest of the time when you don't run your calculations. You can also try set limit to the process.