Page 1 of 1

[SOLVED] incremental backup question

Posted: Thu Jul 16, 2020 3:12 am
by afora
This may be a misguided question, but I was wondering if it's possible to run an 'incremental' backup (e.g. with BackInTime or anything else for that matter) such that the base files ARE NOT copied when the very first snapshot is created? I struggle to find any references to this, so I'm thinking there may be another solution to this.

Use case: I have a large data store which is fully backed up weekly to an off-site storage, the frequency is so low due to logistical limitations. In the meantime, I wanted to run incremental backups on the production data (say every 15 mins, 1 hour, 1 day) locally without using any external devices, and then reset it once the next main backup has been carried out. Due to the size of the production data I really do not need yet another complete backup, only copies of changed/added/deleted files which are snapshotted at those time intervals.

Back in the Windows times, we used a solution which had a tracking database which basically would be used to compare the current FS to and then make incremental copies without a complete base snapshot. Is there a similar approach in Linux Mint?

EDIT 27 July 2020
-------------------------------

Despite some great comments in this threat, in the end I decided to stick to the normal incremental backup routine -> copy the originals once as a necessary evil. The main reason is that I do not have time to pursue the suggested directions which are a way over my head. YMMV.

Re: incremental backup question

Posted: Fri Jul 17, 2020 2:54 am
by deepakdeshp

Re: incremental backup question

Posted: Fri Jul 17, 2020 7:04 am
by afora
This is a a very general summary of different backup types - and nothing about what I was asking. You did not seem to have read my question...

Re: incremental backup question

Posted: Fri Jul 17, 2020 9:13 am
by mr_raider
Yes. The solution you seek is implemented by the btrfs and snapshot function.

With that setup timeshift can create a snapshot which will dynamically increase in size as the origin files start to change.

Implementing this retroactively on a running system is difficult. If you ever need to reinstall your OS it can be considered.

Re: incre2mental backup question

Posted: Fri Jul 17, 2020 10:24 am
by deepakdeshp
afora wrote:
Fri Jul 17, 2020 7:04 am
This is a a very general summary of different backup types - and nothing about what I was asking. You did not seem to have read my question...
The backup solutions listed support incremental backup. You have to choose one of them.

Re: incremental backup question

Posted: Fri Jul 17, 2020 11:08 am
by antikythera
Did you set your machine to use btrfs for the data partition or just go with standard ext4 when you installed Linux Mint? btrfs offers the ability to take snapshots of the file system (copy-on-write) as you desire without the need for backing everything up first.

https://en.wikipedia.org/wiki/Btrfs#Sub ... _snapshots

Re: incremental backup question

Posted: Fri Jul 17, 2020 8:23 pm
by afora
Regarding btrfs vs zfs - I've been reading about zfs and snapshoting for some time already and I was mulling over implementing it at some stage when I have a few weeks to spare on deploying it (as everything new it takes me in Linux).

So the question is - the functionality of an incremental backup implemented in backintime without copying over a base is it also available in zfs, or is it a feature of btrfs only? It feels as if zfs has a more active community and potentially a bit more stable than btrfs?

This's a fantastic reference, thank you guys!

Re: incremental backup question

Posted: Fri Jul 17, 2020 9:32 pm
by mr_raider
afora wrote:
Fri Jul 17, 2020 8:23 pm
Regarding btrfs vs zfs - I've been reading about zfs and snapshoting for some time already and I was mulling over implementing it at some stage when I have a few weeks to spare on deploying it (as everything new it takes me in Linux).

So the question is - the functionality of an incremental backup implemented in backintime without copying over a base is it also available in zfs, or is it a feature of btrfs only? It feels as if zfs has a more active community and potentially a bit more stable than btrfs?

This's a fantastic reference, thank you guys!
zfs and btrfs both have snap shotting ability. For btrfs the tools are implemented directly in the kernel and have been for some time. Zfs is the older system, but due to licensing limitations was not part of the kernel.

I don't know jack squat about zfs, but I've been using btrfs as my default FS on *buntus/Mints since kernel 2.6. The snap shotting ability is intrinsic to btrfs, and does not require an external program. Timeshift just leverages the ability and presents it in a clean format.

The Mint/Ubuntu installer will deploy btrfs automatically if you set the install partition to btrfs and mount it at /. The installer will create two subvolumes for /home and /, and time shft will then allow you do snapshot either or both subvolume. The deployment is transparent to teh user. But if you want to know more about btrfs and snapshots/sub volumes you can read more. Mint/Ubuntu has extremely poor documentation on the subject, so I often use btrfs wiki or the arch wiki.

https://btrfs.wiki.kernel.org/index.php ... Subvolumes

Re: incremental backup question

Posted: Fri Jul 17, 2020 10:47 pm
by afora
Thank you.

Also, will this be true for backintime?
mr_raider wrote:
Fri Jul 17, 2020 9:32 pm

Timeshift just leverages the ability and presents it in a clean format.

Re: incremental backup question

Posted: Fri Jul 17, 2020 10:52 pm
by mr_raider
afora wrote:
Fri Jul 17, 2020 10:47 pm
Thank you.

Also, will this be true for backintime?
mr_raider wrote:
Fri Jul 17, 2020 9:32 pm

Timeshift just leverages the ability and presents it in a clean format.
Never used it.