Fear of Slow Backup Update [SOLVED]

[cecilieaux]

I would like to refresh a backup I did a few months ago to upgrade the OS via clean install ( 17.3 Rosa to 18 Sarah); since it went very slow (32 hours to back up 500gb of data via a USB 3.0 port). The hardware I used is a Seagate Backup Plus 4TB Portable External Hard Drive USB 3.0 (STDR4000901) I bought in July; the software was Linux Mint's own Backup Tool.

Before backing up I partitioned the drive into 4 partitions of roughly the same size: one NTFS (intended for transfers from Windows machines); three EXT4 (intended for transfers of data from my Linux desktop, my Linux laptop and another to store downloaded apps and other extras). At present only the first and second partition have data.

The backup data is not encoded or compressed; it's in its natural state.

PROBLEM: Is there a better way, either hardware or software, to merely copy new files to the external drive while deleting old files I have replaced or updated? I have considered the following but I lack the expertise to tackle either one:

1. As to the hardware, I posted a low customer review for the drive and Seagate said I should have checked my Windows drivers (!!!). I went to the site and they have no Linux drivers, only Windows and Mac. Is there a substitute?

2. If it is a software problem,

(a) is there a set of command line instructions to copy updated or new files to the external drive and to delete files in the external drive that are not present in the original? or

(b) is there something better than the Backup Tool? (I downloaded Aptik to back up downloaded apps, but I am afraid to use it for refreshing the backup and force a long second process).

Thanks, I hope I laid out the problem and solutions sought clearly.

Cecilieaux

[Flemur]

PROBLEM: Is there a better way, either hardware or software, to merely copy new files to the external drive while deleting old files I have replaced or updated?

rsync works great, especially with the GUI, grsync (easy preview & scrolling thru changes).

I never understood why anyone uses more specialized backup tools, other than to make an image to be distributed/installed on multiple machines, as in an office environment.

Edit:

2. If it is a software problem,

??? I couldn't see that you mentioned any actual problem, either hardware or software.

[WharfRat]

cecilieaux,

If the files are in in its natural state you should look into rsync to sync your files and for backups.

I use it exclusively for those purposes

[Termy]

It sounds like you're just wanting to clone the drive. You might want to consider using something like CloneZilla, to do a full image backup; the downside is that the files won't* be accessible without restoring the image.

* There are ways around it, but they're awkward as hell.

Recommend doing research first, if this route interests you.

[cecilieaux]

PROBLEM: Is there a better way, either hardware or software, to merely copy new files to the external drive while deleting old files I have replaced or updated?

rsync works great, especially with the GUI, grsync (easy preview & scrolling thru changes).

I never understood why anyone uses more specialized backup tools, other than to make an image to be distributed/installed on multiple machines, as in an office environment.

Edit:

2. If it is a software problem,

??? I couldn't see that you mentioned any actual problem, either hardware or software.

The slowness was the problem.

But as to rsync and grsync. I download and install both, right? Rsync first, then grcsync?

[Pierre]

you should be able to Install Both at the same time, from Package Manager AKA synaptic.

[WharfRat]

But as to rsync and grsync. I download and install both, right? Rsync first, then grcsync?

I'm pretty sure rsync comes standard with most distros so you only have to install grcsync.

grsync has a rsync depends so just installing grsync will pull rsync if it isn't installed

[pbear]

(a) is there a set of command line instructions to copy updated or new files to the external drive and to delete files in the external drive that are not present in the original?

rsync will do this. You should read the man page, of course. Notice local backup differs slightly from that on a network, in that the default is to copy whole files if modified since last backup rather than just the changes. (This is a good thing, imho.) Then, there's a switch for deleting files on the backup no longer appearing at source. Finally, I like to see what a backup is going to do before running it. This also is supported.

So, the terminal command I use is in the form of rsync -avn --delete /home/user/folder/subfolder/ /media/user/folder/subfolder/. If you have spaces in file or folder names, enclose in quote marks (e.g., "/home/user/text docs/subfolder/") -a means several things (see man page), including backing up sub-directories and leaving timestamps unchanged. As usual, -v means verbose. -n does a dry run; if it looks good (haven't had a glitch yet, actually), run again without. No doubt you can guess what --delete does.

There are sophisticated ways to do this with scripts, chrons and such. If you want to pursue those, I'm sure someone here can help. My approach is simply to maintain a text file with the half-dozen commands I need, which I run manually depending on how fast any particular folder is changing.

All this, by the way, is independent of full system image backups with Clonezilla Live. I consider these complementary strategies.

[cecilieaux]

Thank you all, this is very useful.