[SOLVED] Bad memory management

[Menard]

Hi

There I am with 8 GB RAM and it uses only 63% while it is playing to terminate my hard disk by using 100% of the swap file on it

[rene]

A possibility is that you checked usage just after something that took up a large amount of RAM stopped doing so (by exiting usually), i.e., before demand-driven swap-in had lessened swap usage again. Another possibility is that you are misinterpreting some figures somewhere. What is not a possibility is blanket "bad memory management" because Linux is not blanket-bad at managing memory.

If you're just in the first situation try sudo swapoff -a && sudo swapon -a to force swap-in (although, note, you'd want to contemplate why you'd want to).

[spamegg]

What is the value of: cat /proc/sys/vm/swappiness

[karlchen]

There I am with 8 GB RAM and it uses only 63% while it is playing to terminate my hard disk by using 100% of the swap file on it

What is the size of your swapfile?
The swapfile has got a dynamic size. It may start pretty small, thus easily filled by 100%. But as soon as more swap space should be needed, the system would resize it automatically.
Corrected:
The default swapfile will not be dynamic at all. Dynamic swapfiles will only be created and managed, in case the software swapspace has been configured and is in use.

Irrespective of the initially incorrect statement that the swapfile would be resized dynamically by the system, the swapfile may still have been created too small. This is why asking for its current size is still a valid question.
Resizing the (static) swapfile is still less hassle than resizing a swap device may be, even though by default resizing the swapfile cannot be done "on the fly".

Thanks to Rene below, who made me aware that I had been wrong. Made me read (not only) the Ubuntu manpages on swapfile(s) and on the management software swapspace.

[rene]

The swapfile has got a dynamic size. It may start pretty small, thus easily filled by 100%. But as soon as more swap space should be needed, the system would resize it automatically. This is different from the swap device, used by default on older Mint release. Swap devices had a fixed size.

I believe I noticed you once said this before as well in a by us shared thread, but note that this is not true. Linux does not dynamically size swap space be it a swap partition or a swap file.

It's also the kind of thing one would expect someone would only say if literally looking at the behaviour so to make very, very sure that I hadn't missed something quite fundamental I just tested things and, no: size of a swap file is still as static as size of a swap partition. Difference of course is that former can generally be more easily "manually" grown and there's some projects out there to automate adding additional swap files in userspace when under memory pressure --- but the Linux kernel and our Mint systems know not of dynamically-sized swap space; a /swapfile does not grow dynamically.

[karlchen]

Hi, rene.

Then I stand corrected. I had assumed that this were the relevant advantage of swapfile over swap device. The swap device has got a fixed size, whereas the swapfile were dynamic. - But I admit that I still use the good old swap device with its fixed pre-allocated size.

In this case, the fact that the swapfile is filled by 100% does still not indicate poor memory management, but rather it may be a hint that the swapfile has not been sized properly nad needs resizing. Correct?

Regards,
Karl

[rene]

The swap file advantage, if some, is that it's more easy to manually resize/recreate and manually and/or programmatically add additional ones of when under memory pressure but the downside is (possible) fragmentation. Of a single file and as to all of swap space of course certainly in the case of multiple ones.

In essence it's a very fundamental issue: virtual memory management exist at a level much deeper than filesystems; the filesystem subsystem in the kernel uses the virtual-memory subsystem in a very fundamental and unavoidable manner and if now the virtual-memory subsystem also uses the filesystem subsystem we may get into nasty fundamental circular-dependency issues. I actually looked at things a bit deeper a while ago and it did not in fact seem all that bad in practice, and better than I had expected, what with basically all filesystem related setup done away with at the swapon stage already; things after said stage not being fundamentally different between a swap file and swap partition. The mentioned possible fragmentation of a file is an issue but very significantly less in current SSD times than before on HDD (i.e., seek times having gone almost extinct) and it's in any case in practical terms a possibility to make semi-sure that swap files are not or are not significantly fragmented.

All that said --- yes, I also still use swap partitions; swap files just don't make as much technical sense so that without an actual need I'd just rather use the better swap partition technology.

I would say that the swap file/partition being filled 100% may even mean stellar memory management: that above SSD point notwithstanding speed of storage vs. RAM is not even ballpark close yet on the type of machines which I've noticed Menard uses: if he could still halfway sensibly use the machine with enough RAM swapped out to fill a swap file of any conceivable size I'd say Linux was doing a darned fine job there. Because that's sort of the point; it makes outside of the context of hibernation little sense to have more than 1 or 2 GB of swap available on an 8G machine anyway since swapping more simply means you just loaded up the machine to a level which it can not sensibly handle in the first place. Generally at least. I suppose that if you e.g. use VMs in that 8G that you may not necessarily always mind having most of them swapped out if you're not actively using them --- but we're then in specific situations.

That is: rather than an indication of the swap file not having been sized correctly I very much expect it more a matter of the workload not having been sized correctly for the system: given that it's unusably slow anyway with more than at most a few gigs swapped out why bother for allowing for more as a matter of available swap space?

Anyways; spamegg's swappiness query is semi-relevant but also I expect a red herring, with 100% swap usage. As said, swap-in is demand-driven so it's easy to get into posters described situation if something runs that eats up memory and causes swap-out but then stops/exits leaving free memory: the system doesn't swap in that which was swapped out until it in fact has a use for it. It can as said be forced by swapoff/swapon, even if you'd generally need not.

[Da Londo]

https://easylinuxtipsproject.blogspot.c ... html#ID1.6 This might help.

[Menard]

What is the value of: cat /proc/sys/vm/swappiness

Code: Select all

u3@PC1:~$ cat /proc/sys/vm/swappiness
60
u3@PC1:~$ 

[rene]

That's the standard value and while it makes some sense to on a desktop lower it to e.g. 10 or 20 note that by and large /proc/sys/vm/swappiness should be renamed /proc/sys/vm/redherring. It's very unlikely with 100% swap used that you actually experienced a real problem: something that caused swap-out likely just exited and left free RAM. Fully normal.

[Menard]

What is the size of your swapfile?

I didn't set nothing about it, and the system monitor mentions 1.4 GiB
About RAM it is in fact 7 GB left and it is mentionned this : 4.7 GiB (69.3%) in 6.7 GiB | cache 1.7 GiB

[Menard]

To notice is that 2 hours and a half later ... i am still in this session with shown : Swap at 100% and RAM at 69%

[rene]

That would normally mean you didn't in fact use that which is swapped out yet (may as such also not care). As said, try and force things with sudo swapoff -a && sudo swapon -a --- although, note, if you're still in an active-use situation that may cause an immediate kernel OOM-killer trigger and may costs you some big memory consumer(s) so make sure you've saved that which needs saving in your open applications.

[Menard]

No chance I ve just applied this but the swap usage had decreased before at around 50% and now it is again at 100%

[rene]

You then have something's that's actively eating/leaking memory running. I'd reboot.

[Menard]

It is now at 10%

[rene]

Better, but on an 8G desktop machine you should be expecting 0% swap usage if you use it halfway normally. Since any of it is highly usage dependent in that sense it's hard to assist via the internet with memory issues.

In any case, to get the mentioned red herring squared away; it will if it IS a standard desktop machine not hurt to do

Code: Select all

echo vm.swappiness=10 | sudo tee /etc/sysctl.d/swappiness.conf

and activate with sudo sysctl -p /etc/sysctl.d/swappiness.conf (only needed now; after reboot it will be set automatically). This gives the system a less aggressive swapping profile which makes some sense on a desktop and which may help you in some senses --- but as said, memory use, either legitimately or via some buggy/leaking bit of software is the only to be expected actual problem.

[Menard]

OK thanks I ll try this

[Menard]

I suppose that you include in "PC desktop" a mini ATX tower that lie 2 inches up the floor

[rene]

I'd say that, yes, lying computers do tend to be desktops, be they lying 1, 2 or 3 inches, and up whichever surface.

More directly, "desktop" in this sense would be a usage criterion; a system you'd use for e.g. browsing and other day-to-day tasks; not number-crunching, serving, ...