[SOLVED] incremental backup question

[afora]

This may be a misguided question, but I was wondering if it's possible to run an 'incremental' backup (e.g. with BackInTime or anything else for that matter) such that the base files ARE NOT copied when the very first snapshot is created? I struggle to find any references to this, so I'm thinking there may be another solution to this.

Use case: I have a large data store which is fully backed up weekly to an off-site storage, the frequency is so low due to logistical limitations. In the meantime, I wanted to run incremental backups on the production data (say every 15 mins, 1 hour, 1 day) locally without using any external devices, and then reset it once the next main backup has been carried out. Due to the size of the production data I really do not need yet another complete backup, only copies of changed/added/deleted files which are snapshotted at those time intervals.

Back in the Windows times, we used a solution which had a tracking database which basically would be used to compare the current FS to and then make incremental copies without a complete base snapshot. Is there a similar approach in Linux Mint?

EDIT 27 July 2020
-------------------------------

Despite some great comments in this threat, in the end I decided to stick to the normal incremental backup routine -> copy the originals once as a necessary evil. The main reason is that I do not have time to pursue the suggested directions which are a way over my head. YMMV.

[deepakdeshp]

https://help.ubuntu.com/community/Backu ... l%20backup.

[afora]

This is a a very general summary of different backup types - and nothing about what I was asking. You did not seem to have read my question...

[mr_raider]

Yes. The solution you seek is implemented by the btrfs and snapshot function.

With that setup timeshift can create a snapshot which will dynamically increase in size as the origin files start to change.

Implementing this retroactively on a running system is difficult. If you ever need to reinstall your OS it can be considered.

[deepakdeshp]

This is a a very general summary of different backup types - and nothing about what I was asking. You did not seem to have read my question...

The backup solutions listed support incremental backup. You have to choose one of them.

[antikythera]

Did you set your machine to use btrfs for the data partition or just go with standard ext4 when you installed Linux Mint? btrfs offers the ability to take snapshots of the file system (copy-on-write) as you desire without the need for backing everything up first.

https://en.wikipedia.org/wiki/Btrfs#Sub ... _snapshots

[afora]

Regarding btrfs vs zfs - I've been reading about zfs and snapshoting for some time already and I was mulling over implementing it at some stage when I have a few weeks to spare on deploying it (as everything new it takes me in Linux).

So the question is - the functionality of an incremental backup implemented in backintime without copying over a base is it also available in zfs, or is it a feature of btrfs only? It feels as if zfs has a more active community and potentially a bit more stable than btrfs?

This's a fantastic reference, thank you guys!

[mr_raider]

Regarding btrfs vs zfs - I've been reading about zfs and snapshoting for some time already and I was mulling over implementing it at some stage when I have a few weeks to spare on deploying it (as everything new it takes me in Linux).

So the question is - the functionality of an incremental backup implemented in backintime without copying over a base is it also available in zfs, or is it a feature of btrfs only? It feels as if zfs has a more active community and potentially a bit more stable than btrfs?

This's a fantastic reference, thank you guys!

zfs and btrfs both have snap shotting ability. For btrfs the tools are implemented directly in the kernel and have been for some time. Zfs is the older system, but due to licensing limitations was not part of the kernel.

I don't know jack squat about zfs, but I've been using btrfs as my default FS on *buntus/Mints since kernel 2.6. The snap shotting ability is intrinsic to btrfs, and does not require an external program. Timeshift just leverages the ability and presents it in a clean format.

The Mint/Ubuntu installer will deploy btrfs automatically if you set the install partition to btrfs and mount it at /. The installer will create two subvolumes for /home and /, and time shft will then allow you do snapshot either or both subvolume. The deployment is transparent to teh user. But if you want to know more about btrfs and snapshots/sub volumes you can read more. Mint/Ubuntu has extremely poor documentation on the subject, so I often use btrfs wiki or the arch wiki.

https://btrfs.wiki.kernel.org/index.php ... Subvolumes

[afora]

Thank you.

Also, will this be true for backintime?

Timeshift just leverages the ability and presents it in a clean format.

[mr_raider]

Thank you.

Also, will this be true for backintime?

Timeshift just leverages the ability and presents it in a clean format.

Never used it.