Timeshift development management

[MikeNovember]

Hi,

Here are some remarks about Timeshift development is managed.

1) PPA:
This PPA, https://launchpad.net/~teejee2008/+arch ... /timeshift, used for Timeshift daily builds, still appears to be owned by Tony George, and publications still appear to be done by Tony George, while the PPA is now under Linux Mint control.

Suggestions:
Update the PPA ownership and the publication author.

2) Issues management:
Issues are managed here, https://github.com/linuxmint/timeshift/issues.

178 are open today (2024-02-19).

Latest release notes, "timeshift_24.01.1_amd64.changes", just mention "updated translations". They don't mention any issue fix.

Some of the open issues are very annoying ones:
- No snaps on restore #179, this issue shows that Timeshift is unable to restore the snaps installed on an operating system. Though open on 2023-04-19, there is nobody assigned to this issue. This is a very annoying issue, for it breaks the confidence on Timeshift: since it is proven that Timeshift cannot totally restore a system, how can we use it?

- Snapshots not detected by Timeshift #108, in some cases a created snapshot doesn't appear in the list; I opened this issue on 2022-12-21, found a cause (info.json may contain unreadable characters), others found another cause (info.json missing) and I proposed a workaround. This issue is still unassigned, though Michael Webster did comment this issue. At the moment, I have never seen the problem again since Timeshift 22.11.2. Can this issue be closed?

Suggestions:
- make a list of the most annoying issues among the 178 open ones and assign a developer to treat the issue,
- close the issues that can be (issue not reproduced using latest versions, impossible requests such as "Timeshift as a flatpak", not confirmed issues...),
- in the release notes, mention the fixed issues,
- backport new Timeshift releases to Linux Mint 20.x.

Regards,

MN

[Pjotr]

Sensible suggestions for an important tool like Timeshift.

[SMG]

1) PPA:
This PPA, https://launchpad.net/~teejee2008/+arch ... /timeshift, used for Timeshift daily builds, still appears to be owned by Tony George, and publications still appear to be done by Tony George, while the PPA is now under Linux Mint control.

Where did you read that the PPA ownership changed? Just because the source code is under Linux Mint control does not mean the PPA is under Linux Mint control. The blog post describing how Timeshift is now an Xapp and maintenance is being done by the Linux Mint team does not mention taking control of the PPA.

Tony George can still make PPAs from the latest code if he wants (and apparently he does). Do not confuse the source code with PPAs which provide the code. They are two separate things.

- Snapshots not detected by Timeshift #108, in some cases a created snapshot doesn't appear in the list; I opened this issue on 2022-12-21, found a cause (timeshift.json may contain unreadable characters), others found another cause (timeshift.json missing) and I proposed a workaround. This issue is still unassigned, though Michael Webster did comment this issue. At the moment, I have never seen the problem again since Timeshift 22.11.2. Can this issue be closed?

If you opened it, give an update saying you have not seen the problem since then and then close the issue. I've seen others do that many times.

[MikeNovember]

- Snapshots not detected by Timeshift #108, in some cases a created snapshot doesn't appear in the list; I opened this issue on 2022-12-21, found a cause (info.json may contain unreadable characters), others found another cause (info.json missing) and I proposed a workaround. This issue is still unassigned, though Michael Webster did comment this issue. At the moment, I have never seen the problem again since Timeshift 22.11.2. Can this issue be closed?

If you opened it, give an update saying you have not seen the problem since then and then close the issue. I've seen others do that many times.

Hi,

I have written in the issue that I had not seen the unexpected characters error since version 22.11.2.
But others have mentioned another cause, missing info.json.
Michael Webster had told he would try to reproduce this issue.
My hypothesis was people using write caching on removable disks and removing disk before the writing was really finished.
But, since I haven't seen this info.json missing error, I can't conclude on it.
Note this is not the most annoying error, since there is a workaround.
Here is the complete workaround:
- Disable write caching on the disk where you save your Timeshift snapshots (this removes a potential cause for info.json missing, by removing an external disk too early).
- Disable write caching on the disk that could be restored with Timeshift (this prevents a system restart while restore is not finished).
- After a snapshot creation, check the presence of info.json file, and open it with a text reader; if it is OK, copy it and save it somewhere in your home and close Timeshift; if it is KO, relaunch the Timeshift snapshot creation.
- When you want to restore the system from a snapshot, if the snapshot is found, OK, proceed; if the snapshot is not found, close Timeshift, copy the saved info.json (over an existing one, if one exists), launch Timeshift and proceed.

Today, the most annoying issue is Timeshift unable to restore snaps. The cause is not known, and since it is not known there are doubts on Timeshift capability to correctly restore a system: is the unknown cause able to disturb other part of the restore?
From the description of the problem in the issue, it appears that snaps are saved in the snapshot, but Timeshift is not able to restore them correctly: "the /snap folder does have most of the needed files, but there is no /snap/bin/ directory and all the current symlinks are missing". Since the problem is in the restore phase, one could suspect write permissions.
Note that, though the issue is open since 2023-04-19 (ten months), no Linux Mint developer has commented it (read it?).
Today, Timeshift is not reliable. I still use it, but using also Foxclone to have reliable backups in the case Timeshift would not work correctly.

Linux Mint has decided to take the control on Timeshift development, and presents Timeshift as one of the main programs. But Timeshift legacy is full of issues, and not treating these issues will have as consequence an abandon of Timeshift by users.

Regards,

MN

[AndyMH]

Are you seriously suggesting that the mint devs fix a bug in timeshift relating to snaps - which are deliberately disabled by default in mint, in my view for very good reasons?

[MikeNovember]

Are you seriously suggesting that the mint devs fix a bug in timeshift relating to snaps - which are deliberately disabled by default in mint, in my view for very good reasons?

Yes, I am strongly suggesting that Mint developers fix this bug, for several reasons:
- Though disabled by default, users can enable snaps in Linux Mint.
- Timeshift is also available for Debian, Ubuntu and several other distros; Tony George did not create Timeshift for Linux Mint only.
- Until the cause of this failure to restore snaps is not identified and fixed, there is a doubt on Timeshift capability to correctly and fully restore an operating system from a snapshot; the same cause might disturb restore process for something else than snaps.

A backup and restore program that is not able to correctly restore an operating system from its backup is not a reliable tool.

Regards,

MN

[Hoser Rob]

Are you seriously suggesting that the mint devs fix a bug in timeshift relating to snaps - which are deliberately disabled by default in mint, in my view for very good reasons?

Of course. Mint users have the option of installing snap packages and have every right to do so.

[MikeNovember]

Hi,

Googling for topics related to Timeshift and snaps restore, I have found this old closed issue on Timeshift GitHub, at the time it was developed by Tony George:
https://github.com/teejee2008/timeshift/issues/701.
In the discussion, Tony George says:

/snap is a read-only mount. Actual snap files are kept in the /var/lib/snapd/ directory.
it is excluded on purpose. Closing this issue.

About snaps directories:
"/snap": virtual directories, read-only, containing snaps mounted while running; should not be backed up.
"/snap/bin": contains environment wrappers for the installed snaps, or, in recent versions, symlinks to "/usr/bin/snap" where are found the wrappers; should be backed up, since these wrappers or symlinks are created when a snap is installed.
"/usr/bin/snap": should be backed up.
"/var/snap/": data and configuration files of running snaps; should not be backed up.
"/var/lib/snapd": contains the installed snaps; should be backed up and restored by Timeshift.
"~/snap": contains the snaps configurations, on user home; should be backed up and restored by a data backup program (like FreeFileSync), though Timeshift could do it.

So, comparing this with the old closed issue, it appears that Timeshift doesn't backup "/snap" directory at all, while it should at least backup the "/snap/bin" subdirectory.

This is probably the cause to Timeshift not able to correctly back up and restore snaps.

To fix it by a Timeshift maintainer seems easy.

Regards,

MN

PS: I have added this comments to the "No snaps on restore #179" issue, https://github.com/linuxmint/timeshift/issues/179

[MikeNovember]

Hi,

I confirm the preceding analysis. Here is the list of excluded directories, from Timeshift sources, https://github.com/linuxmint/timeshift/ ... /Main.vala:

exclude_list_default.add("/dev/*");
exclude_list_default.add("/proc/*");
exclude_list_default.add("/sys/*");
exclude_list_default.add("/media/*");
exclude_list_default.add("/mnt/*");
exclude_list_default.add("/tmp/*");
exclude_list_default.add("/run/*");
exclude_list_default.add("/var/run/*");
exclude_list_default.add("/var/lock/*");
//exclude_list_default.add("/var/spool/*");
exclude_list_default.add("/var/lib/dhcpcd/*");
exclude_list_default.add("/var/lib/docker/*");
exclude_list_default.add("/var/lib/schroot/*");
exclude_list_default.add("/lost+found");
exclude_list_default.add("/timeshift/*");
exclude_list_default.add("/timeshift-btrfs/*");
exclude_list_default.add("/data/*");
exclude_list_default.add("/DATA/*");
exclude_list_default.add("/cdrom/*");
exclude_list_default.add("/sdcard/*");
exclude_list_default.add("/system/*");
exclude_list_default.add("/etc/timeshift.json");
exclude_list_default.add("/var/log/timeshift/*");
exclude_list_default.add("/var/log/timeshift-btrfs/*");
exclude_list_default.add("/swapfile");
exclude_list_default.add("/snap/*");

Here is the content of a "standard" (default) "exclude.list" file found in a Timeshift snapshot:

/dev/*
/proc/*
/sys/*
/media/*
/mnt/*
/tmp/*
/run/*
/var/run/*
/var/lock/*
/var/lib/dhcpcd/*
/var/lib/docker/*
/var/lib/schroot/*
/lost+found
/timeshift/*
/timeshift-btrfs/*
/data/*
/DATA/*
/cdrom/*
/sdcard/*
/system/*
/etc/timeshift.json
/var/log/timeshift/*
/var/log/timeshift-btrfs/*
/swapfile
/snap/*
/root/.thumbnails
/root/.cache
/root/.dbus
/root/.gvfs
/root/.local/share/[Tt]rash
/home/*/.thumbnails
/home/*/.cache
/home/*/.dbus
/home/*/.gvfs
/home/*/.local/share/[Tt]rash
/root/.mozilla/firefox/*.default/Cache
/root/.mozilla/firefox/*.default/OfflineCache
/root/.opera/cache
/root/.kde/share/apps/kio_http/cache
/root/.kde/share/cache/http
/home/*/.mozilla/firefox/*.default/Cache
/home/*/.mozilla/firefox/*.default/OfflineCache
/home/*/.opera/cache
/home/*/.kde/share/apps/kio_http/cache
/home/*/.kde/share/cache/http
/var/cache/apt/archives/*
/var/cache/pacman/pkg/*
/var/cache/yum/*
/var/cache/dnf/*
/var/cache/eopkg/*
/var/cache/xbps/*
/var/cache/zypp/*
/var/cache/edb/*
/home/username/**
/root/**
/home/*/**

(bold and underline are mine).

Since"/snap" is excluded, "/snap/bin" is not backed up, and when Timeshift is restored, the content of "/snap/bin" is considered as files having changed since the last snapshot and are deleted.

Regards,

MN

[AndyMH]

Since"/snap" is excluded, "/snap/bin" is not backed up, and when Timeshift is restored, the content of "/snap/bin" is considered as files having changed since the last snapshot and are deleted.

Have you tested that or is it what you think will happen. Apply the same logic to any other exclusion, e.g. /timeshift, it would delete your snapshots, it doesn't happen.

[MikeNovember]

Since"/snap" is excluded, "/snap/bin" is not backed up, and when Timeshift is restored, the content of "/snap/bin" is considered as files having changed since the last snapshot and are deleted.

Have you tested that or is it what you think will happen. Apply the same logic to any other exclusion, e.g. /timeshift, it would delete your snapshots, it doesn't happen.

Hi,

It is an analysis, of open issue 179, of old closed issue by Tony George, and of Timeshift source.

So, it is what I think and should be considered as a potential bug cause, since it matches and explains what people have described in issue 179: no symlinks in "/snap/bin".

I understand what you say; I have not read completely the source code, just tried to find a confirmation of Tony George saying that "/snap" was not backed up.

In the code there may be several exclude lists, concerning backup or restore. There could be special instructions concerning Timeshift snapshots.

When you look at code, I am not shocked by other exclusions such as "/dev", "/mnt", "/tmp".

Note that, in another part of sources, some files are excluded on their suffix, such as ".old" and others.

Timeshift is far to make snapshots of all system files. This might be revised, see if it is accurate and up-to-date and eventually changed.

Regards,

MN

[MikeNovember]

Hi,

Antonio Petricca has issued a Problem Report with a proposed fix "SNAP restore fixup #279", https://github.com/linuxmint/timeshift/pull/279

The fix is based on not excluding "/snap", and having it saved / restored when creating / restoring snapshots.

Regards,

MN