(Solved) VeraCrypt Problem

[felemur]

Lately when using VeraCrypt, my system starts to respond very slowly with long delays after a click, and memory usage goes to max (16Gb), and swap is maxed out as well (according to System Manager).

This happens when transferring files to VeraCrypt containers, not out of them.

Transfer speed into containers is also very slow compared to what it was before this issue started.

I'm wondering if there are settings I can change in VeraCrypt that will correct this. The only way to resolve the problem when it happens is to restart Linux. It has even locked up to the point where I was forced to go nuclear (Raising Elephants Is So Utterly Boring).

There is no other program that causes any issues, so I feel fairly confident this is a VeraCrypt settings issue, possibly a bug, or some unusual situation with my particular setup.

Happy to answer any questions.

Dell Optiplex 7010, Intel i7, 16Gb Ram
Linux Mint 18.2 Cinnamon
VeraCrypt 1.21 (latest)

(This question has been posted in the VeraCrypt forum as well)

[pbear]

That's one hell of a bug! Frankly, makes me glad I stuck with TrueCrypt (my security needs are modest, only worried about casual snoopers).

Anyhoo, I notice the last VeraCrypt update was in early July. Hard to imagine it's been more than three months and you're the only one to notice the issue. Makes me wonder whether some other update has created a conflict. No idea what that might be.

Do please cross-post here if you get a solution on the VC forum.

[felemur]

Yes, I will cross post here with any info from the VeraCrypt forum.

I'm actually surprised that nobody so far has even commented yet on the VeraCrypt forum. I agree I can't be the only one, but you never know.

There are settings in VeraCrypt that I don't understand enough to want to mess with. Two are the "Do not use kernel cryptographic services" option, and the other is the "Do not accelerate AES encryption/decryption by using the AES instructions of the processor" option. Neither are checked by default, and I am worried I could scramble my files if I do check either to see if they help. When this problem is happening, the memory & swap is maxed out, but the i7 is running about 15% on all 8 cores, so its not like the processor is getting stuck into some weird loop. I'm not sure if this is a memory leak issue, but from my little knowledge, it sounds like one.

I've tried shutting off every other program other than "Files" and VeraCrypt to see if that helps, but no.

I've tried other kernels, the latest of the 4.8, 4.10 and 4.11 series, but no change. I even removed VeraCrypt 1.21 and put in an old VeraCrypt 1.18 - but no luck - same problem.

The thing I personally find odd, is this issue just appeared a month or two ago on its own. Before that, VeraCrypt worked flawlessly. I haven't added any software that I can think of in that time frame. I did my latest fresh install when Mint 18.0 came out whenever that was, and have just been doing the suggested updates via the update manager.

PS - I first saw "Time flies like an arrow. Fruit flies like a banana." in a late 1970's OMNI magazine - a sort of science & science fiction magazine. It won some kind of contest for the best funny science word play that was sent in. Most submissions were based around the planet Uranus.

[fungus]

@felemur: Kernel cryptographic services are crypto routines built into the kernel, read more here: https://kernel.readthedocs.io/en/sphinx ... o-API.html Some CPU's have a special AES instruction set which may accelerate encryption quite a lot (sort of a GPU for encryption). Disabling kernel crypto and AES instructions is not a problem, VeraCrypt must have has build in support for encryption to use as a fall back option. If your problem is caused by kernel regression, disabling kernel crypto may help. If your CPU is dying, disabling AES instructions may help. Good luck!

[majpooper]

I am using Veracrypt with a very similar rig i7 16G of RAM and have tried to create your issue but nothing seems slow.
Could it be the kernel - I am running Cinnamon 18 with kernel 4.4.0-97. The other thing that is different I have turned SWAP off.

[felemur]

Sorry for the late reply to your post.

Our rigs do sound similar. We are even running the same kernel, however I have tried others with no help.

It seems that whatever is going wrong with my setup, Veracrypt is going to take up the full amount of memory and swap regardless of what amount that is. I've watched it happen with the System Monitor program that comes with Mint Cinnamon. From using about 2Gb from when I first start transferring the file into the Veracrypt container to full 16Gb used in about 1 minute or less, then moves to swap, and fills that just as fast. After both are full, the entire system basically starts grinding to a halt.

I don't think I made this clear previously: even after I dismount the Veracrypt container, shut down VeraCrypt, the memory usage does not drop. The system will not return to normal until after a restart.

I'm starting to get the feeling the only way I'm going to resolve this is do a fresh install. There must be something I've done along the way that is causing this, as I seem to be the only one with the problem - not a single reply or similar post on the VeraCrypt forum. I sure hope a fresh install works.

[MintBean]

I don't think I made this clear previously: even after I dismount the Veracrypt container, shut down VeraCrypt, the memory usage does not drop. The system will not return to normal until after a restart.

That's unusual because my understanding was that the OS reclaimed the allocated memory on program exit irrespective of whether the code performed the relevant deallocations. Presumably VC either leaks memory through an independent system process it interacts with or it spawned a process that is somehow no destroyed on exit.

Is the missing memory listed against a process after VC exits?

[felemur]

That is a good question - I've looked into that.

According to the "processes" tab on System Monitor, there is nothing using that memory.

[felemur]

@felemur: Kernel cryptographic services are crypto routines built into the kernel, read more here: https://kernel.readthedocs.io/en/sphinx ... o-API.html Some CPU's have a special AES instruction set which may accelerate encryption quite a lot (sort of a GPU for encryption). Disabling kernel crypto and AES instructions is not a problem, VeraCrypt must have has build in support for encryption to use as a fall back option. If your problem is caused by kernel regression, disabling kernel crypto may help. If your CPU is dying, disabling AES instructions may help. Good luck!

Thanks, will try both and report on what happens.

Edit: Tried both options checked. No difference except - CPU usage went from about 15% when they were unchecked to about 30% when checked. (This is when transferring files into VeraCrypt container.)

[pbear]

PS - I first saw "Time flies like an arrow. Fruit flies like a banana." in a late 1970's OMNI magazine - a sort of science & science fiction magazine. It won some kind of contest for the best funny science word play that was sent in. Most submissions were based around the planet Uranus.

Like many classics, the origin of the joke is uncertain, but the most likely correct attribution is to Anthony Oettinger, from a 1966 article in Scientific American, although he didn't state it quite so succinctly (was later paraphrased by others).

Good luck with your quest. Sorry can't be more helpful.

[felemur]

Problem has solved itself in unknown way.

It starts to eat memory, then drops back down again. Does this over and over as it is transferring large amounts of files into an encrypted container.

The only thing I've changed since the last time I used it when the problem was there and now: I've done the updates including the latest 4.4 series kernel. Correlation is not causation, but I'll take it.

[fungus]

Always nice.. Ehrm.. Concerning when problems fixes them selves.. You may want to check up on your disk caching - sound a bit like you are loading a lot of the data being copied into memory before actually writing it to disk. Since disk caching is a kernel feature a kernel update may very well have changed the behaviour of this. Check this article about disk caching: https://lonesysadmin.net/2013/12/22/bet ... rty_ratio/

[Portreve]

No issues here regardless of whether I'm dealing with disk images or physical devices.

Perhaps something else is going on with your system that you haven't yet realized?

[pbear]

The only thing I've changed since the last time I used it when the problem was there and now: I've done the updates including the latest 4.4 series kernel. Correlation is not causation, but I'll take it.

Had a similar pattern several months ago on one of my USB systems. Kernel update knocked out the WiFi connection, which still worked in the host machine running the same kernel from hard drive. Reverted kernel of the USB drive but updated next go round, which worked fine. My guess was that there had been an error installing the earlier update. So, you might have an important unidentified system issue, but not necessarily.

[felemur]

Always nice.. Ehrm.. Concerning when problems fixes them selves.. You may want to check up on your disk caching - sound a bit like you are loading a lot of the data being copied into memory before actually writing it to disk. Since disk caching is a kernel feature a kernel update may very well have changed the behaviour of this. Check this article about disk caching: https://lonesysadmin.net/2013/12/22/bet ... rty_ratio/

Thanks for that, I will read up on that and see what can be done - what you say makes sense. However......

I do remember changing the swappiness to 10 from the default 60 after upgrading to 18.2, but I had forgotten (and you reminded me) I had put less pressure on shrinking the inode cache by adding this line:

Code: Select all

# Improve cache management
vm.vfs_cache_pressure=50

to the /etc/sysctl.conf file as per: https://sites.google.com/site/easylinux ... node-cache

I wonder if that was the root of the issue?

[felemur]

Always nice.. Ehrm.. Concerning when problems fixes them selves.. You may want to check up on your disk caching - sound a bit like you are loading a lot of the data being copied into memory before actually writing it to disk. Since disk caching is a kernel feature a kernel update may very well have changed the behaviour of this. Check this article about disk caching: https://lonesysadmin.net/2013/12/22/bet ... rty_ratio/

In looking into that link you gave, I checked, and found that my dirty background cache ratio was 10%

Code: Select all

vm.dirty_background_bytes = 0
vm.dirty_background_ratio = 10
vm.dirty_bytes = 0
vm.dirty_expire_centisecs = 3000
vm.dirty_ratio = 20
vm.dirty_writeback_centisecs = 500
vm.dirtytime_expire_seconds = 43200

which means with 16Gb of RAM, I would have been loading up the RAM with 1.6GB before it asynchronously flushed the cache to disk. If you add that to the putting less pressure on shrinking the inode cache, I wonder if I caused the problem?

I'm interested in expert opinion before I go ahead and make changes, though with 16Gb RAM, it seems like a conservative change to making the dirty background cache 3 to 5% from the current 10%

[fungus]

I'm interested in expert opinion before I go ahead and make changes, though with 16Gb RAM, it seems like a conservative change to making the dirty background cache 3 to 5% from the current 10%

It is up to you to decide how your caching is to be handled. Experimenting yields experience, so go ahead and fiddle with it and post back with your findings