[SOLVED] incremental backup question

All Gurus once were Newbies
Forum rules
There are no such things as "stupid" questions. However if you think your question is a bit stupid, then this is the right place for you to post it. Please stick to easy to-the-point questions that you feel people can answer fast. For long and complicated questions prefer the other forums within the support section.
Before you post please read how to get help
Post Reply
afora
Level 4
Level 4
Posts: 203
Joined: Mon Aug 26, 2019 7:35 pm

[SOLVED] incremental backup question

Post by afora »

This may be a misguided question, but I was wondering if it's possible to run an 'incremental' backup (e.g. with BackInTime or anything else for that matter) such that the base files ARE NOT copied when the very first snapshot is created? I struggle to find any references to this, so I'm thinking there may be another solution to this.

Use case: I have a large data store which is fully backed up weekly to an off-site storage, the frequency is so low due to logistical limitations. In the meantime, I wanted to run incremental backups on the production data (say every 15 mins, 1 hour, 1 day) locally without using any external devices, and then reset it once the next main backup has been carried out. Due to the size of the production data I really do not need yet another complete backup, only copies of changed/added/deleted files which are snapshotted at those time intervals.

Back in the Windows times, we used a solution which had a tracking database which basically would be used to compare the current FS to and then make incremental copies without a complete base snapshot. Is there a similar approach in Linux Mint?

EDIT 27 July 2020
-------------------------------

Despite some great comments in this threat, in the end I decided to stick to the normal incremental backup routine -> copy the originals once as a necessary evil. The main reason is that I do not have time to pursue the suggested directions which are a way over my head. YMMV.
Last edited by afora on Mon Jul 27, 2020 4:05 am, edited 1 time in total.
deepakdeshp
Level 18
Level 18
Posts: 8444
Joined: Sun Aug 09, 2015 10:00 am

Re: incremental backup question

Post by deepakdeshp »

If I have helped you solve a problem, please add [SOLVED] to your first post title, it helps other users looking for help, and keeps the forum clean.
Regards,
Deepak

I am using Mint 20 Cinnamon 64 bit with AMD A8/7410 / 8GB
Mint 20 Cinnamon AMD Ryzen3500U
afora
Level 4
Level 4
Posts: 203
Joined: Mon Aug 26, 2019 7:35 pm

Re: incremental backup question

Post by afora »

This is a a very general summary of different backup types - and nothing about what I was asking. You did not seem to have read my question...
mr_raider
Level 7
Level 7
Posts: 1889
Joined: Sun Jun 20, 2010 9:50 am
Location: Montreal, QC

Re: incremental backup question

Post by mr_raider »

Yes. The solution you seek is implemented by the btrfs and snapshot function.

With that setup timeshift can create a snapshot which will dynamically increase in size as the origin files start to change.

Implementing this retroactively on a running system is difficult. If you ever need to reinstall your OS it can be considered.
Image
deepakdeshp
Level 18
Level 18
Posts: 8444
Joined: Sun Aug 09, 2015 10:00 am

Re: incre2mental backup question

Post by deepakdeshp »

afora wrote:
Fri Jul 17, 2020 7:04 am
This is a a very general summary of different backup types - and nothing about what I was asking. You did not seem to have read my question...
The backup solutions listed support incremental backup. You have to choose one of them.
If I have helped you solve a problem, please add [SOLVED] to your first post title, it helps other users looking for help, and keeps the forum clean.
Regards,
Deepak

I am using Mint 20 Cinnamon 64 bit with AMD A8/7410 / 8GB
Mint 20 Cinnamon AMD Ryzen3500U
User avatar
antikythera
Level 9
Level 9
Posts: 2633
Joined: Thu Jul 02, 2020 12:52 pm

Re: incremental backup question

Post by antikythera »

Did you set your machine to use btrfs for the data partition or just go with standard ext4 when you installed Linux Mint? btrfs offers the ability to take snapshots of the file system (copy-on-write) as you desire without the need for backing everything up first.

https://en.wikipedia.org/wiki/Btrfs#Sub ... _snapshots
Don't take life so seriously, nobody gets out alive anyway!
AMSTRAD CPC6128 - 128KB RAM, 3" Hitachi Floppy Diskette Drive, External Sony Cassette Recorder, Locomotive BASIC 1.1, CTM-644 Monitor
afora
Level 4
Level 4
Posts: 203
Joined: Mon Aug 26, 2019 7:35 pm

Re: incremental backup question

Post by afora »

Regarding btrfs vs zfs - I've been reading about zfs and snapshoting for some time already and I was mulling over implementing it at some stage when I have a few weeks to spare on deploying it (as everything new it takes me in Linux).

So the question is - the functionality of an incremental backup implemented in backintime without copying over a base is it also available in zfs, or is it a feature of btrfs only? It feels as if zfs has a more active community and potentially a bit more stable than btrfs?

This's a fantastic reference, thank you guys!
mr_raider
Level 7
Level 7
Posts: 1889
Joined: Sun Jun 20, 2010 9:50 am
Location: Montreal, QC

Re: incremental backup question

Post by mr_raider »

afora wrote:
Fri Jul 17, 2020 8:23 pm
Regarding btrfs vs zfs - I've been reading about zfs and snapshoting for some time already and I was mulling over implementing it at some stage when I have a few weeks to spare on deploying it (as everything new it takes me in Linux).

So the question is - the functionality of an incremental backup implemented in backintime without copying over a base is it also available in zfs, or is it a feature of btrfs only? It feels as if zfs has a more active community and potentially a bit more stable than btrfs?

This's a fantastic reference, thank you guys!
zfs and btrfs both have snap shotting ability. For btrfs the tools are implemented directly in the kernel and have been for some time. Zfs is the older system, but due to licensing limitations was not part of the kernel.

I don't know jack squat about zfs, but I've been using btrfs as my default FS on *buntus/Mints since kernel 2.6. The snap shotting ability is intrinsic to btrfs, and does not require an external program. Timeshift just leverages the ability and presents it in a clean format.

The Mint/Ubuntu installer will deploy btrfs automatically if you set the install partition to btrfs and mount it at /. The installer will create two subvolumes for /home and /, and time shft will then allow you do snapshot either or both subvolume. The deployment is transparent to teh user. But if you want to know more about btrfs and snapshots/sub volumes you can read more. Mint/Ubuntu has extremely poor documentation on the subject, so I often use btrfs wiki or the arch wiki.

https://btrfs.wiki.kernel.org/index.php ... Subvolumes
Image
afora
Level 4
Level 4
Posts: 203
Joined: Mon Aug 26, 2019 7:35 pm

Re: incremental backup question

Post by afora »

Thank you.

Also, will this be true for backintime?
mr_raider wrote:
Fri Jul 17, 2020 9:32 pm

Timeshift just leverages the ability and presents it in a clean format.
mr_raider
Level 7
Level 7
Posts: 1889
Joined: Sun Jun 20, 2010 9:50 am
Location: Montreal, QC

Re: incremental backup question

Post by mr_raider »

afora wrote:
Fri Jul 17, 2020 10:47 pm
Thank you.

Also, will this be true for backintime?
mr_raider wrote:
Fri Jul 17, 2020 9:32 pm

Timeshift just leverages the ability and presents it in a clean format.
Never used it.
Image
Post Reply

Return to “Newbie Questions”