Completely deleting a file ?

Quick to answer questions about finding your way around Linux Mint as a new user.
Forum rules
There are no such things as "stupid" questions. However if you think your question is a bit stupid, then this is the right place for you to post it. Stick to easy to-the-point questions that you feel people can answer fast. For long and complicated questions use the other forums in the support section.
Before you post read how to get help. Topics in this forum are automatically closed 6 months after creation.
User avatar
all41
Level 19
Level 19
Posts: 9431
Joined: Tue Dec 31, 2013 9:12 am
Location: Computer, Car, Cage

Re: Completely deleting a file ?

Post by all41 »

doubletwo wrote: Mon Nov 27, 2023 2:22 pm So how do countries deal with their no longer wanted secret files? Purely by encryption?
When I worked for a government supplier we destroyed the drives in the presence of security
and one other witness. We all had to sign the traveler which I assume went to the DOD.
This was a hydraulic press with a cutting wedge. They were all hdds at the time.
I'm sure they now have a method for ssd as well
Everything in life was difficult before it became easy.
User avatar
JerryF
Level 16
Level 16
Posts: 6509
Joined: Mon Jun 08, 2015 1:23 pm
Location: Rhode Island, USA

Re: Completely deleting a file ?

Post by JerryF »

doubletwo wrote: Mon Nov 27, 2023 10:27 am I appreciate all the answers, and people taking the time to help.

Not sure however, anyone has given me the answer i seek, or even if the answer i seek exists.

It quite bothers me, that anyone who were to steal my laptop, could possibly recover sensitive files ive deleted.
I think you're asking for a solution that's just not possible. If someone is physically in possession of your files, they could somehow gain access to them one way or another.

If you're so worried about someone stealing your laptop, either don't remove it from your house or save your files on a portable disk that you will take home.
linux-rox
Level 10
Level 10
Posts: 3289
Joined: Sun Jul 19, 2020 9:17 pm

Re: Completely deleting a file ?

Post by linux-rox »

doubletwo wrote: Mon Nov 27, 2023 2:22 pm So how do countries deal with their no longer wanted secret files? Purely by encryption?
You're making a mountain out of a mole hill. Data recovery is fantastically difficult. Only seems easy in the movies because they're pretend. If the stuff you're worrying about actually is sensitive, use encryption. If that sounds like too much trouble, the secret isn't very important. What's not going to happen is someone developing an app to do exactly what you want, just because you want it.
doubletwo
Level 3
Level 3
Posts: 143
Joined: Thu Sep 17, 2015 7:21 am

Re: Completely deleting a file ?

Post by doubletwo »

linux-rox wrote: Mon Nov 27, 2023 8:53 pm
doubletwo wrote: Mon Nov 27, 2023 2:22 pm So how do countries deal with their no longer wanted secret files? Purely by encryption?
You're making a mountain out of a mole hill. Data recovery is fantastically difficult. Only seems easy in the movies because they're pretend. If the stuff you're worrying about actually is sensitive, use encryption. If that sounds like too much trouble, the secret isn't very important. What's not going to happen is someone developing an app to do exactly what you want, just because you want it.
Your replies to me have been aggressive and quite uncalled for. Do you talk to everyone like this?
User avatar
MikeNovember
Level 7
Level 7
Posts: 1782
Joined: Fri Feb 28, 2020 7:37 am
Location: Nice, Paris, France

Re: Completely deleting a file ?

Post by MikeNovember »

Hi,

You can use the shred command, see https://www.freecodecamp.org/news/secur ... and-shred/.

As former said, on a SSD, "sudo fstrim -av" will send to the disk controller the list of abandoned memory cells and, in a few seconds for modern disk, all these cells will be erased.

Regards,

MN
_____________________________
Linux Mint 21.3 Mate host with Ubuntu Pro enabled, VMware Workstation Player with Windows 10 Pro guest, ASUS G74SX (i7-2670QM, 16 GB RAM, GTX560M with 3GB RAM, 1TB SSD).
User avatar
xenopeek
Level 25
Level 25
Posts: 29286
Joined: Wed Jul 06, 2011 3:58 am

Re: Completely deleting a file ?

Post by xenopeek »

shred doesn't work because of wear leveling; each new write to a file goes to a new disk block and leaves the data on the original disk block intact. TRIM doesn't help because what it does is indicate to the SSD firmware that the underlying disk block is not used by the filesystem so that disk block can go back into the pool for the wear leveling algorithm. It does not guarantee the data on it is erased AFAIK.

If you want data erased, after you've already written it in plain text to the SSD, the 2 options are physical destruction of the flash chips or trusting the SSD firmware's security erase command. In both cases it means having to install the OS anew.

But again, you need special tools even to attempt to recover data directly from the flash chips so this may be all theoretical.

There's not good reason not to use disk encryption, assuming responsible computer use where regular backups are made of user files.
Image
billyswong
Level 7
Level 7
Posts: 1875
Joined: Wed Aug 14, 2019 1:02 am

Re: Completely deleting a file ?

Post by billyswong »

doubletwo wrote: Mon Nov 27, 2023 2:22 pm So how do countries deal with their no longer wanted secret files? Purely by encryption?
In the older days, there were only HDDs. When one overwrite a file with zeros or random bits, the file is really overwritten. Nowadays with SSDs, this is no longer true because of wear-leveling.

Government departments that are paranoid about data security crush the retired drives physically as said by @aa41. Without such tools on hand, the only way I can think of for personal paranoid is to overwrite the whole drive in random bits twice. Wear-leveling shall be unable to fake it and the original sector recording your file should get overwritten after the process. And your drive will get aged significantly and closer to failure as well.

So the long term solution is to use disk encryption or home folder encryption right from the beginning.
User avatar
xenopeek
Level 25
Level 25
Posts: 29286
Joined: Wed Jul 06, 2011 3:58 am

Re: Completely deleting a file ?

Post by xenopeek »

billyswong wrote: Tue Nov 28, 2023 8:43 amoverwrite the whole drive in random bits twice. Wear-leveling shall be unable to fake it and the original sector recording your file should get overwritten after the process.
You can't rely on that either. SSDs all have a spare area and if the disk blocks with the sensitive data are approaching end of life they could be swapped out with new disk blocks from the spare area on next write.
Image
User avatar
MikeNovember
Level 7
Level 7
Posts: 1782
Joined: Fri Feb 28, 2020 7:37 am
Location: Nice, Paris, France

Re: Completely deleting a file ?

Post by MikeNovember »

xenopeek wrote: Tue Nov 28, 2023 8:21 am shred doesn't work because of wear leveling; each new write to a file goes to a new disk block and leaves the data on the original disk block intact. TRIM doesn't help because what it does is indicate to the SSD firmware that the underlying disk block is not used by the filesystem so that disk block can go back into the pool for the wear leveling algorithm. It does not guarantee the data on it is erased AFAIK.

If you want data erased, after you've already written it in plain text to the SSD, the 2 options are physical destruction of the flash chips or trusting the SSD firmware's security erase command. In both cases it means having to install the OS anew.

But again, you need special tools even to attempt to recover data directly from the flash chips so this may be all theoretical.

There's not good reason not to use disk encryption, assuming responsible computer use where regular backups are made of user files.
Hi,

Concerning shred
Tthere is effectively a risk that, with journaled file systems, the system does not overwrite data in place. An extract of shred man page:
CAUTION: Note that shred relies on a very important assumption: that
the file system overwrites data in place. This is the traditional way
to do things, but many modern file system designs do not satisfy this
assumption. The following are examples of file systems on which shred
is not effective, or is not guaranteed to be effective in all file sys‐
tem modes:
* log-structured or journaled file systems, such as those supplied with
AIX and Solaris (and JFS, ReiserFS, XFS, Ext3, etc.)
* file systems that write redundant data and carry on even if some
writes fail, such as RAID-based file systems
* file systems that make snapshots, such as Network Appliance's NFS
server
* file systems that cache in temporary locations, such as NFS version 3
clients
* compressed file systems

In the case of ext3 file systems, the above disclaimer applies (and
shred is thus of limited effectiveness) only in data=journal mode,
which journals file data in addition to just metadata. In both the
data=ordered (default) and data=writeback modes, shred works as usual.

Ext3 journaling modes can be changed by adding the data=something op‐
tion to the mount options for a particular file system in the
/etc/fstab file, as documented in the mount man page (man mount).
Note first that, except if you run a server, there is no need to have journaled ext file systems; and, according to the man, with the default ext3 setting, "data=ordered" mode or with "data= writeback" mode, shred works.
With ext4, the default setting is also "data=ordered" mode.

Concerning trim
SSDs have memory cells; each cell has three states: erased, written to 0, written to 1. Before to write on a cell, it should be erased. To improve write performance, unused cells should be erased in the background.
Two mechanisms do this:
- the disk controller "garbage collector", working permanently in the background,
- trim, being done periodically or with a terminal command sudo fstrim -av.
When a trim is done, the file system sends to the disk controller the lists of abandoned/unused memory cells. Then, within a few seconds for modern SSDs (my Crucial one) or within a few minutes for older SSDs, all the abandoned memory cells are erased, preventing any data recovery of deleted files.
There is however a caveat: SSDs have spare cells, to improve their durability; when a set of active cells is replaced by a set of spare ones, the former active cells are no longer seen by the system file, and cannot be trimmed.
They could contain some data, potentially recoverable with forensics tools.

Concerning disk encryption
Full system encryption (operating system and user data) should be used only when there are heavy risks (laptop, used out of office, by a physician, lawyer, activist, journalist, whistleblower, cryptocurrencies owner...).
User home encryption offers almost the same protection, except that, since Linux uses only one temp directory for the system and the user, some user data might be found in "/tmp". The solution is to use a RAM disk for temp files.
Finally, most users have only "some" sensitive files to protect. They can use an encrypted container, using VeraCrypt.

Note that all this has to be taken into account in a true security approach, see viewtopic.php?t=397740.

Regards,

MN
_____________________________
Linux Mint 21.3 Mate host with Ubuntu Pro enabled, VMware Workstation Player with Windows 10 Pro guest, ASUS G74SX (i7-2670QM, 16 GB RAM, GTX560M with 3GB RAM, 1TB SSD).
TaterChip
Level 4
Level 4
Posts: 488
Joined: Sat Apr 22, 2023 12:34 pm
Location: Everywhere USA

Re: Completely deleting a file ?

Post by TaterChip »

after reading the post here, I am now beginning to wander if this is even a viable option anymore. Seems I'm wasting my time.
Attachments
ksnip_20231128-074537.jpg
ksnip_20231128-074537.jpg (9.91 KiB) Viewed 298 times
MSI Steel series GL75 Leopard, i7-10750H, 64GB RAM ... Mint21.1 XFCE
User avatar
MikeNovember
Level 7
Level 7
Posts: 1782
Joined: Fri Feb 28, 2020 7:37 am
Location: Nice, Paris, France

Re: Completely deleting a file ?

Post by MikeNovember »

TaterChip wrote: Tue Nov 28, 2023 9:47 am after reading the post here, I am now beginning to wander if this is even a viable option anymore. Seems I'm wasting my time.
Hi,

File shredder, from flathub or Gnome, is no more than a kind of GUI version of shred, with the same advantages and limitations.

Regards,

MN
_____________________________
Linux Mint 21.3 Mate host with Ubuntu Pro enabled, VMware Workstation Player with Windows 10 Pro guest, ASUS G74SX (i7-2670QM, 16 GB RAM, GTX560M with 3GB RAM, 1TB SSD).
User avatar
xenopeek
Level 25
Level 25
Posts: 29286
Joined: Wed Jul 06, 2011 3:58 am

Re: Completely deleting a file ?

Post by xenopeek »

MikeNovember wrote: Tue Nov 28, 2023 9:21 am Concerning shred
Tthere is effectively a risk that, with journaled file systems, the system does not overwrite data in place.
Doesn't matter which filesystems is being used or which options. On a SSD you can't know which disk block a filesystem block is stored on (that's internal to the firmware) and, because of wear leveling, each write to a filesystem block can use a new disk block and leave the data on the original disk block intact. shred only works if the disk block the filesystem block is stored on is constant and that's not the case for SSDs.
MikeNovember wrote: Tue Nov 28, 2023 9:21 am Concerning trim
SSDs have memory cells; each cell has three states: erased, written to 0, written to 1. Before to write on a cell, it should be erased. To improve write performance, unused cells should be erased in the background.
So my information was outdated. TRIM does guarantee disk blocks get erased as it is part of the GC process. It's communicating to the firmware which filesystem blocks no longer hold relevant information and the firmware can later, in the background at its own pace, erase the underlying disk blocks. This was a good read about the technical bits: https://www.researchgate.net/publicatio ... zation_SSD

Linux Mint runs TRIM automatically every 7 days on devices that support it.
MikeNovember wrote: Tue Nov 28, 2023 9:21 am Concerning disk encryption
Full system encryption (operating system and user data) should be used only when there are heavy risks (laptop, used out of office, by a physician, lawyer, activist, journalist, whistleblower, cryptocurrencies owner...).
Because home directory encryption with ecryptfs has the logout bug and is slower (see the LM 20 and above release notes) we'll have to agree to disagree. Full disk encryption is faster and more reliable.
Image
User avatar
MikeNovember
Level 7
Level 7
Posts: 1782
Joined: Fri Feb 28, 2020 7:37 am
Location: Nice, Paris, France

Re: Completely deleting a file ?

Post by MikeNovember »

xenopeek wrote: Tue Nov 28, 2023 10:10 am
MikeNovember wrote: Tue Nov 28, 2023 9:21 am Concerning shred
Tthere is effectively a risk that, with journaled file systems, the system does not overwrite data in place.
Doesn't matter which filesystems is being used or which options. On a SSD you can't know which disk block a filesystem block is stored on (that's internal to the firmware) and, because of wear leveling, each write to a filesystem block can use a new disk block and leave the data on the original disk block intact. shred only works if the disk block the filesystem block is stored on is constant and that's not the case for SSDs.
OK, shred should so be reserved to HDDs...
MikeNovember wrote: Tue Nov 28, 2023 9:21 am Concerning disk encryption
Full system encryption (operating system and user data) should be used only when there are heavy risks (laptop, used out of office, by a physician, lawyer, activist, journalist, whistleblower, cryptocurrencies owner...).
Because home directory encryption with ecryptfs has the logout bug and is slower (see the LM 20 and above release notes) we'll have to agree to disagree. Full disk encryption is faster and more reliable.
Hi,

I didn't mention ecryptfs but VeraCrypt.

Regards,

MN
_____________________________
Linux Mint 21.3 Mate host with Ubuntu Pro enabled, VMware Workstation Player with Windows 10 Pro guest, ASUS G74SX (i7-2670QM, 16 GB RAM, GTX560M with 3GB RAM, 1TB SSD).
TaterChip
Level 4
Level 4
Posts: 488
Joined: Sat Apr 22, 2023 12:34 pm
Location: Everywhere USA

Re: Completely deleting a file ?

Post by TaterChip »

MikeNovember wrote: Tue Nov 28, 2023 9:58 am
TaterChip wrote: Tue Nov 28, 2023 9:47 am after reading the post here, I am now beginning to wander if this is even a viable option anymore. Seems I'm wasting my time.
Hi,

File shredder, from flathub or Gnome, is no more than a kind of GUI version of shred, with the same advantages and limitations.

Regards,

MN
copy that... time to remove it from my system and workflow
MSI Steel series GL75 Leopard, i7-10750H, 64GB RAM ... Mint21.1 XFCE
linux-rox
Level 10
Level 10
Posts: 3289
Joined: Sun Jul 19, 2020 9:17 pm

Re: Completely deleting a file ?

Post by linux-rox »

doubletwo wrote: Tue Nov 28, 2023 6:22 am Your replies to me have been aggressive and quite uncalled for.
Pointing out you're mistaken is not aggressive. Or are you talking about pointing out you've ghosted two threads after folks went to the trouble of trying to answer your queries? Anyhoo, as a favor to the mods, I won't bother you any more. Good luck.
doubletwo
Level 3
Level 3
Posts: 143
Joined: Thu Sep 17, 2015 7:21 am

Re: Completely deleting a file ?

Post by doubletwo »

linux-rox wrote: Tue Nov 28, 2023 11:56 am
doubletwo wrote: Tue Nov 28, 2023 6:22 am Your replies to me have been aggressive and quite uncalled for.
Pointing out you're mistaken is not aggressive. Or are you talking about pointing out you've ghosted two threads after folks went to the trouble of trying to answer your queries? Anyhoo, as a favor to the mods, I won't bother you any more. Good luck.
Your general tone has been sharp and nasty. Take a look back at your responses.
I'll leave it there
linux-rox
Level 10
Level 10
Posts: 3289
Joined: Sun Jul 19, 2020 9:17 pm

Re: Completely deleting a file ?

Post by linux-rox »

Here are my first to posts two you: viewtopic.php?p=2400007#p2400007; viewtopic.php?p=2400008#p2400008
I'll let others draw their own conclusions. For obvious reasons, though, I won't be answering any follow-up questions.

Have a nice day.
decrepit
Level 5
Level 5
Posts: 841
Joined: Mon Dec 03, 2018 4:07 am
Location: Mandurah Western Australia

Re: Completely deleting a file ?

Post by decrepit »

Out of interest, I've gone back through the whole thread, and see no sign of aggression or nastiness.
But maybe a very frank truthfulness.
I have a feeling linux- rox doesn't suffer fools gladly.
So I understand why you are offended, some times the truth is uncomfortable.
I find the best strategy is to try and learn from it.
I'm sure that was the intention
Post Reply

Return to “Beginner Questions”