Effect of deleting old snapshots in Timeshift?

Questions about applications and software
Forum rules
Before you post please read how to get help
somelurker
Level 3
Level 3
Posts: 173
Joined: Fri Jul 01, 2016 6:10 pm

Effect of deleting old snapshots in Timeshift?

Post by somelurker »

I'm using Timeshift in RSYNC mode. I believe I read somewhere that future snapshots only save changes to older snapshots, so it made me wonder if it is safe to delete older snapshots.

Let's say I make 3 snapshots as follows:
Snapshot A: Oldest snapshot
Snapshot B: Taken between snapshots A and C
Snapshot C: Most recent snapshot

Would it be safe to delete snapshot B, or would doing so wreck snapshot C because snapshot C depends on information from snapshot B? Also, if I delete snapshot A, would both snapshots B and C become useless?
User avatar
Pierre
Level 20
Level 20
Posts: 11223
Joined: Fri Sep 05, 2008 5:33 am
Location: Perth, AU.

Re: Effect of deleting old snapshots in Timeshift?

Post by Pierre »

you would need to check if TS is using either the Incremental backup or the Differential backup method.

what you are describing is the Incremental backup method and TS would use what you specified in the Settings.

http://typesofbackup.com/
gives a description of each type of method to backup your stuff.
8)

what you are looking for is probably the Differential backup,
and of course, there are advantages & disadvantages to each method

some folks prefer the Incremental backup method as it uses less disks, on each backup,
than either a Full backup,or for a Differential backup will use.
and this method also requires All of the Disks, to be kept, for any needed Restore.

whereas a Differential backup uses more disk on each subsequent backup,

it used to depend, if you wanted to save $$$ by recycling some of the backup disks, or not so.

this was back in the days of using Floppy disks, or later on, when using CDs for backing up your stuff.

whereas today, Disks used = Space required . . . .

so if you were using the Differential backup method, then you could indeed save space,
by deleting any backups between the Original Full Backup, and any subsequent Differential backup.
8)
Image
Please edit your original post title to include [SOLVED] - when your problem is solved!
and DO LOOK at those Unanswered Topics - - you may be able to answer some!.
User avatar
slipstick
Level 6
Level 6
Posts: 1048
Joined: Sun Oct 21, 2012 9:56 pm
Location: Somewhere on the /LL0 scale

Re: Effect of deleting old snapshots in Timeshift?

Post by slipstick »

somelurker wrote:
Mon May 28, 2018 8:06 pm
I'm using Timeshift in RSYNC mode. I believe I read somewhere that future snapshots only save changes to older snapshots, so it made me wonder if it is safe to delete older snapshots.

Let's say I make 3 snapshots as follows:
Snapshot A: Oldest snapshot
Snapshot B: Taken between snapshots A and C
Snapshot C: Most recent snapshot

Would it be safe to delete snapshot B, or would doing so wreck snapshot C because snapshot C depends on information from snapshot B? Also, if I delete snapshot A, would both snapshots B and C become useless?
You can delete B without affecting C, or you can delete A without affecting B or C because of the "hard link" method used. Simplifed explanation (skipping over the inodes): the data gets stored on the first snapshot, and the directory entries in the backup are hard-linked to the data files. On subsequent incremental backups, new files are added with their new hard links - for unchanged files, hard links are added in the new snapshots linking to the original data from the first snapshot. A data file which is unchanged over the course of a dozen snapshots will have a dozen hard links to it. If the next snapshot does not contain that file because it has been deleted from your system, then no more hardlinks to that file data will be added, but the data will remain on the backup until there are no more hard links to it from previous snapshots - you would have to delete all snapshots which contained that file before the data would disappear.

Assume A has files 1, 2, and 3. There are hard links from backup A to the data of these each of these files.
A: 1, 2, 3
Now add a file 4 and delete file 2 and make backup B. Now files 1 and 3 have two hard links (A and B), file 2 has one link (A) and file 4 has one link (B).
B: 1, 3, 4
Backups A and B appear to be complete backups at the time they were made. Now modify file 1 to 1' and make backup C. File 1 has two links (A and B), file 1' has one link (C), file 2 has one link (A), file 3 has 3 links (A, B, C), and file 4 has 2 links (B and C).
C: 1', 3, 4
But there is only one copy of the data for each of the 5 files. The data for a file remains until all the hard links to that file have been removed. Delete backup A and file 2 will no longer have any links so file 2 data goes away. But there are still links to files 1, 1', 3, and 4. Backup B is a still a complete backup and so is C.
In theory, theory and practice are the same. In practice, they ain't.
MintBean

Re: Effect of deleting old snapshots in Timeshift?

Post by MintBean »

To state what slipstick said in a slightly less technical manner:
Even if you delete a snapshot, 'behind the scenes' information is retained so long as there are any remaining links to it - so it won't screw up later snapshots.
somelurker
Level 3
Level 3
Posts: 173
Joined: Fri Jul 01, 2016 6:10 pm

Re: Effect of deleting old snapshots in Timeshift?

Post by somelurker »

I mentioned that I was using RSYNC, so this basically forces incremental backup. If Timeshift's description is accurate, there is no option to use differential backup while using RSYNC.

The hard link explanation was quite clear as to the effect of deleting previous snapshots. But there's something I don't understand about the mechanism behind it. I looked at /timeshift/snapshots, and there appears to be one folder for each snapshot. The hardlinks system described sounds much more sophisticated that what appears to be the case on the surface. When I look at those folders without knowledge of how TS works, I just assume that each folder represents one snapshot, and that all the contents of that snapshot are contained in the folder.

Let's go back to your example. In all 3 snapshots, there is a hardlink to file 3. If I delete snapshot A, presumably that just removes the folder corresponding to snapshot A in /timeshift/snapshots. If incremental snapshots are used, the data for file 3 should only be stored in snapshot A because it was unchanged. But you say deleting snapshot A does not get rid of file 3 in snapshots B and C because of the hard link system. If this is true, it seems one of these 3 possibilities must be true:

1. There must be copies of file 3 in both the folders corresponding to snapshots B and C; otherwise, deleting snapshot A would have destroyed any backups of file 3
2. File 3 was moved over to snapshot B and C by Timeshift when deleting snapshot A using Timeshift (this suggests manual deletion of the folders in /timeshift/snapshots is dangerous and that users should exclusively use the Timeshift interface to manage snapshots.
3. File 3 was never stored in any of the folders in /timeshift/snapshots and so it was undisturbed by deleting the folder in /timeshift/snapshots corresponding to snapshot A

What am I missing?

Basically, here was my motivation for asking the question. I feared this scenario: I make one initial snapshot of the system as snapshot A. This snapshot is by far the largest because it has to back up everything. I then make some small change and create snapshot B. Snapshot B only contains the differences made between snapshots A and B. I delete snapshot A. Snapshot B only has the differences and not most of the files that remained the same from A to B, which is 99% of the backup. I therefore only have 1% of the data needed to recover the system on Snapshot B and it fails.
User avatar
chrisuk
Level 5
Level 5
Posts: 592
Joined: Thu Jun 12, 2008 6:16 am

Re: Effect of deleting old snapshots in Timeshift?

Post by chrisuk »

Basically the work is being done by the EXT4 filesystem using rsync and hardlinks... Timeshift is just a frontend for the process. This is basically what Timeshift does - the hard link diagram may help you understand what's happening in the background
Chris

Manjaro MATE - MX Linux - LMDE MATE
User avatar
trytip
Level 13
Level 13
Posts: 4979
Joined: Tue Jul 05, 2016 1:20 pm

Re: Effect of deleting old snapshots in Timeshift?

Post by trytip »

the initial snapshot is the most important. i usually don't like scheduled snapshots since i don't want the extra space so i disable scheduling and try to keep one original snap. when i'm confident that i can use the snapshot for a good restore backup i delete every snapshot and create a new one. i guess with these terabyte harddrives you can just let it do it's thing and then you'll have some more backups, but in the even that something doesn't work you're now less informed about your system because everything was done on auto for you and you have no clue where to start troubleshooting.

today i used timeshift in my linux/mint/mate/17.3 to restore arch linux at least 3 times. something in the new package manager (the most important piece of software in any linux) got updated leaving a whole lot of depression for older packages that haven't been informed of the newest protocol and causing them to all fail miserably.

needless to say everyone has leftover configurations that needed that one extra #uncommented line in a script.
good luck on your quest
Last edited by trytip on Tue May 29, 2018 10:55 pm, edited 1 time in total.
Image
User avatar
slipstick
Level 6
Level 6
Posts: 1048
Joined: Sun Oct 21, 2012 9:56 pm
Location: Somewhere on the /LL0 scale

Re: Effect of deleting old snapshots in Timeshift?

Post by slipstick »

somelurker wrote:
Tue May 29, 2018 3:05 am
I mentioned that I was using RSYNC, so this basically forces incremental backup. If Timeshift's description is accurate, there is no option to use differential backup while using RSYNC.

The hard link explanation was quite clear as to the effect of deleting previous snapshots. But there's something I don't understand about the mechanism behind it. I looked at /timeshift/snapshots, and there appears to be one folder for each snapshot. The hardlinks system described sounds much more sophisticated that what appears to be the case on the surface. When I look at those folders without knowledge of how TS works, I just assume that each folder represents one snapshot, and that all the contents of that snapshot are contained in the folder.
The data is not contained "within" the folder.
somelurker wrote:
Tue May 29, 2018 3:05 am
Let's go back to your example. In all 3 snapshots, there is a hardlink to file 3. If I delete snapshot A, presumably that just removes the folder corresponding to snapshot A in /timeshift/snapshots. If incremental snapshots are used, the data for file 3 should only be stored in snapshot A because it was unchanged. But you say deleting snapshot A does not get rid of file 3 in snapshots B and C because of the hard link system. If this is true, it seems one of these 3 possibilities must be true:

1. There must be copies of file 3 in both the folders corresponding to snapshots B and C; otherwise, deleting snapshot A would have destroyed any backups of file 3
2. File 3 was moved over to snapshot B and C by Timeshift when deleting snapshot A using Timeshift (this suggests manual deletion of the folders in /timeshift/snapshots is dangerous and that users should exclusively use the Timeshift interface to manage snapshots.
3. File 3 was never stored in any of the folders in /timeshift/snapshots and so it was undisturbed by deleting the folder in /timeshift/snapshots corresponding to snapshot A
The answer is 3.
somelurker wrote:
Tue May 29, 2018 3:05 am
What am I missing?
Basically, here was my motivation for asking the question. I feared this scenario: I make one initial snapshot of the system as snapshot A. This snapshot is by far the largest because it has to back up everything. I then make some small change and create snapshot B. Snapshot B only contains the differences made between snapshots A and B. I delete snapshot A. Snapshot B only has the differences and not most of the files that remained the same from A to B, which is 99% of the backup. I therefore only have 1% of the data needed to recover the system on Snapshot B and it fails.
I think what you are missing is that the file data is not contained "within" a particular snapshot folder. Think of it this way: the data is stored separately from the snapshot folder directories. The directories in each snapshot folder contain only hard links to the data. This is true even for the initial snapshot. And remember that the data for a file is only deleted when every hard link to that data is removed. So if you make an initial snapshot, all the file data gets stored with a single hard link to each file's data from the snapshot directory. Now if you immediately make another snapshot without having changed anything, then no more file data is stored, but a new snapshot directory is created with hard links to the file data that was stored with the initial snapshot. Now each file has 2 hard links. Now remove the initial snapshot. No data is deleted - only the number of hard links to each file changes from 2 to 1.
In theory, theory and practice are the same. In practice, they ain't.
somelurker
Level 3
Level 3
Posts: 173
Joined: Fri Jul 01, 2016 6:10 pm

Re: Effect of deleting old snapshots in Timeshift?

Post by somelurker »

Ok, that makes sense. I guess it would be safe to delete most older snapshots then, because if I understood this correctly, each snapshot is a full backup of the system and doesn't depend on any other snapshots.

The resolution of this minor problem is major progress, but it brings up two new problems. The first one is that I've been storing my snapshots on the same partition as the one I'm backing up. I've been using TimeShift to recover from minor problems stemming from failed kernel updates; problems that don't require reinstalling Linux. If I want to back up my TimeShift snapshots on an external drive, how would I go about this? I originally intended to just copy over the folders at /timeshift, but if the backups aren't stored there, how do I do this?

The second problem is about recovery from a total disaster like a dead hard drive. TimeShift states that it can repair "systems which can’t log into a Graphical Environment, are completely formatted or damaged beyond repair." Can I still recover from a complete format of the drive if Mint is installed on a LUKS-encrypted partition? If so, how?
User avatar
slipstick
Level 6
Level 6
Posts: 1048
Joined: Sun Oct 21, 2012 9:56 pm
Location: Somewhere on the /LL0 scale

Re: Effect of deleting old snapshots in Timeshift?

Post by slipstick »

somelurker wrote:
Tue May 29, 2018 8:37 pm
Ok, that makes sense. I guess it would be safe to delete most older snapshots then, because if I understood this correctly, each snapshot is a full backup of the system and doesn't depend on any other snapshots.
Yes, each snapshot is a complete backup of the system files at the time it was made, independent of the other snapshots. But TImeShift only backs up system files, by default it does not back up your personal data files (and is not recommended for that purpose) and by default does not back up hidden configuration files in your home directory (though it can be set to do that). I use TS for my system files and I use "back-in-time" for my data. Back-in-time also uses rsync and takes snapshots in a similar manner to TS, but is more configurable.
somelurker wrote:
Tue May 29, 2018 8:37 pm
The resolution of this minor problem is major progress, but it brings up two new problems. The first one is that I've been storing my snapshots on the same partition as the one I'm backing up. I've been using TimeShift to recover from minor problems stemming from failed kernel updates; problems that don't require reinstalling Linux. If I want to back up my TimeShift snapshots on an external drive, how would I go about this? I originally intended to just copy over the folders at /timeshift, but if the backups aren't stored there, how do I do this?
I have a separate partition on my hard drive for my TS snapshots for easy restoration from bad updates or if I make a mistake and screw up my system. Many people make their TS backups to an external USB drive. The problem with that is that if you have scheduled backups, you have to leave the external drive plugged in all the time, making it susceptible to accidental erasure or other mishap. You could avoid this by only making on-demand backups and just plug in the external drive when you want to take a snapshot. But then you might not have a recent enough backup to fully recover the system. What I do is keep snapshots on my internal HD (monthly, weekly, daily, boot, on-demand, but not hourly). Then every day or two, I plug in my external USB drive and run a script file that uses rsync to back up the snapshots from my internal drive to the USB drive. I'm not entirely satisfied with this as the script file takes a long time to run (20-30 minutes) but seems to work and keep all the links intact. I admit I have never restored my system from one of these externally backed up snapshots, but I have pretty good confidence that it would work, as I cannot tell any difference between the snapshots on the internal HD and the USB drive. Here is the script I use, which I launch from a launcher I made on my desktop:

Code: Select all

#!/bin/bash
#update timeshift backup on external drive from snapshots on internal HD

sudo rsync -aAuHXvis --progress --delete --numeric-ids /media/root/LM18_timeshift/ /media/steve/Z97_LM_Backup/LM18.3_timeshift_backup/timeshift | tee ~/timeshift_backup_log

echo "Finished updating timeshift backup"

sleep 10
exit 0
It starts by deleting all the snapshots from the USB drive that are not on the internal HD, then there is a long pause with no apparent activity (10-15 minutes), then it copies the new snapshots from the internal HD to the USB drive. You would need to change the source and destination in the rsync statement to match your system. It also keeps a log file in my home directory, which is not necessary and takes up a lot of room, but I like to have it to review.
somelurker wrote:
Tue May 29, 2018 8:37 pm
The second problem is about recovery from a total disaster like a dead hard drive. TimeShift states that it can repair "systems which can’t log into a Graphical Environment, are completely formatted or damaged beyond repair." Can I still recover from a complete format of the drive if Mint is installed on a LUKS-encrypted partition? If so, how?
This I can't answer, as I do not use LUKS and know almost nothing about it.
In theory, theory and practice are the same. In practice, they ain't.
pbear
Level 15
Level 15
Posts: 5662
Joined: Wed Jun 21, 2017 12:25 pm
Location: San Francisco

Re: Effect of deleting old snapshots in Timeshift?

Post by pbear »

slipstick wrote:
Mon May 28, 2018 9:49 pm
You can delete B without affecting C, or you can delete A without affecting B or C because of the "hard link" method used. [snip]
FWIW, I found your explanation exceptionally clear and helpful. Had inferred from various sources this was true, but couldn't figure out why. Now I know.
User avatar
slipstick
Level 6
Level 6
Posts: 1048
Joined: Sun Oct 21, 2012 9:56 pm
Location: Somewhere on the /LL0 scale

Re: Effect of deleting old snapshots in Timeshift?

Post by slipstick »

pbear wrote:
Tue May 29, 2018 11:26 pm
slipstick wrote:
Mon May 28, 2018 9:49 pm
You can delete B without affecting C, or you can delete A without affecting B or C because of the "hard link" method used. [snip]
FWIW, I found your explanation exceptionally clear and helpful. Had inferred from various sources this was true, but couldn't figure out why. Now I know.
Glad you found it useful. I tried to keep it as simple as possible and avoid talking about inodes. :)
In theory, theory and practice are the same. In practice, they ain't.
Petermint
Level 6
Level 6
Posts: 1303
Joined: Tue Feb 16, 2016 3:12 am

Re: Effect of deleting old snapshots in Timeshift?

Post by Petermint »

inodes and file storage is easy to explain. Think of a book in a library. You could store them by title but two different books can have the same title. The library adds a label with a unique number. That is their "inode", just like a file in a Linux/Unix file system. The title is the file name. The inode is the unique identifier. Each snapshot can use the same file name and point to the same inode. The inode points to the actual file location.

Use a separate disk for your backup. No mater how many snapshots you make on the same disk, you lose the lot when your disk breaks. You do not need much space on the extra disk. My 1 TB SSD is 901 GB full. The backup disk is 2 TB. 8 snapshots use 1 TB. The first snapshot used 901 GB because it physically copied all the files to the other disk. The remaining snapshots use only about 10 GB because very few files have changed.

Some other points. You backup to the same partition. You clean up your main directories, deleting all the junk. The junk is still in the snapshots which means it is not deleted from disk. You are not freeing up space. When your backup is on another disk, deleting a file from your main disk will free up space.

What happens when file Abc is added between snapshot 1 and 2 then deleted between snapshot 2 and 3. You delete snapshot 2. The file disappears because nothing references the file. You cannot restore file Abc. Keeping older snapshots gives you the option to recover old files you thought you no longer needed. As an example, the MySQL database data is stored in your system files. Each time you update MySQL data, the next snapshot stores new files because they are different. You could recover a lost database from an old backup.
somelurker
Level 3
Level 3
Posts: 173
Joined: Fri Jul 01, 2016 6:10 pm

Re: Effect of deleting old snapshots in Timeshift?

Post by somelurker »

slipstick, what is the difference between what your script does and just copying the entire Timeshift folder to an external drive? For example, it appears that the default storage location for snapshots is simply /timeshift. If I were to copy /timeshift to an external hard drive, would this do the same thing as your script file?
User avatar
AZgl1500
Level 15
Level 15
Posts: 5553
Joined: Thu Dec 31, 2015 3:20 am
Location: Oklahoma where the wind comes sweeping down the plains
Contact:

Re: Effect of deleting old snapshots in Timeshift?

Post by AZgl1500 »

somelurker wrote:
Mon Jun 11, 2018 8:59 pm
slipstick, what is the difference between what your script does and just copying the entire Timeshift folder to an external drive? For example, it appears that the default storage location for snapshots is simply /timeshift. If I were to copy /timeshift to an external hard drive, would this do the same thing as your script file?
I should think that this would work perfectly well, if you copy the entire folder to a "new /Timeshift" folder each time. I would rename the current /Timeshift to /Timeshift.old

that way you have two backups.

I have a separate partition on the laptop's hard drive so that scheduled events can take place anytime.

I have a separate USB HDD that I plug in and then change the "Location" for Timeshift to the extUSB HDD and then run Timeshift in manual mode. this is done when I happen to think about it.

After removing the extUSB drive, If the extUSB HDD location is not reset to the internal HDD's partition, Timeshift won't run like you want it to. so far for me, it works.
Linux Mint 19.3 Cinnamon
User avatar
slipstick
Level 6
Level 6
Posts: 1048
Joined: Sun Oct 21, 2012 9:56 pm
Location: Somewhere on the /LL0 scale

Re: Effect of deleting old snapshots in Timeshift?

Post by slipstick »

somelurker wrote:
Mon Jun 11, 2018 8:59 pm
slipstick, what is the difference between what your script does and just copying the entire Timeshift folder to an external drive? For example, it appears that the default storage location for snapshots is simply /timeshift. If I were to copy /timeshift to an external hard drive, would this do the same thing as your script file?
I've never tried it, but my guess would be that if you copied a Timeshift folder with several snapshots in it without using a method like rsync that preserves the hard link structure, you would end up with multiple copies of most of the data. Each snapshot looks like a complete backup, so I think you would end up with a copy of each complete backup and it would take up a LOT of space.
In theory, theory and practice are the same. In practice, they ain't.
Petermint
Level 6
Level 6
Posts: 1303
Joined: Tue Feb 16, 2016 3:12 am

Re: Effect of deleting old snapshots in Timeshift?

Post by Petermint »

You can simply start a new backup on the external disk. it will be slow because it copies everything. Do your daily or weekly backup to that disk for a while. After a few months, delete the timeshift directory from your current disk.
You could also

Code: Select all

cp --preserve=links
. Read the manual for cp.

My exploration of Timeshift: https://petermoulding.com/timeshift
bigal
Level 4
Level 4
Posts: 348
Joined: Mon Aug 03, 2009 3:26 am
Location: Pembrokeshire, South West Wales

Re: Effect of deleting old snapshots in Timeshift?

Post by bigal »

chrisuk wrote:
Tue May 29, 2018 4:43 am
Basically the work is being done by the EXT4 filesystem using rsync and hardlinks... Timeshift is just a frontend for the process. This is basically what Timeshift does - the hard link diagram may help you understand what's happening in the background
One very helpful link. Thank you.
Lots of Minty machines but there again I like lamb and I do live in Wales!
cday
Level 1
Level 1
Posts: 38
Joined: Mon Dec 02, 2019 12:18 pm
Location: Cheltenham, U.K.

Re: Effect of deleting old snapshots in Timeshift?

Post by cday »

bigal wrote:
Mon Aug 06, 2018 1:12 pm
chrisuk wrote:
Tue May 29, 2018 4:43 am
Basically the work is being done by the EXT4 filesystem using rsync and hardlinks... Timeshift is just a frontend for the process. This is basically what Timeshift does - the hard link diagram may help you understand what's happening in the background
One very helpful link. Thank you.
Unfortunately the linked article seems to be no longer available, is anyone able to make the content available in another way... :?:
User avatar
Pjotr
Level 22
Level 22
Posts: 16190
Joined: Mon Mar 07, 2011 10:18 am
Location: The Netherlands (Holland)
Contact:

Re: Effect of deleting old snapshots in Timeshift?

Post by Pjotr »

cday wrote:
Wed Apr 22, 2020 6:25 am
bigal wrote:
Mon Aug 06, 2018 1:12 pm
chrisuk wrote:
Tue May 29, 2018 4:43 am
Basically the work is being done by the EXT4 filesystem using rsync and hardlinks... Timeshift is just a frontend for the process. This is basically what Timeshift does - the hard link diagram may help you understand what's happening in the background
One very helpful link. Thank you.
Unfortunately the linked article seems to be no longer available, is anyone able to make the content available in another way... :?:
The Wayback internet archive helps:
https://web.archive.org/web/20181008152 ... chine.html

This article of mine might also be helpful:
https://easylinuxtipsproject.blogspot.c ... t.html#ID7
(item 7)
Tip: 10 things to do after installing Linux Mint 20.1 Ulyssa
Keep your Linux Mint healthy: Avoid these 10 fatal mistakes
Twitter: twitter.com/easylinuxtips
All in all, horse sense simply makes sense.
Post Reply

Return to “Software & Applications”