Backing up files to USB drive

[Netherprovinc3]

Even though there are more sophisticated ways to backup files, I am just manually copying files to a USB thumb drive and having that serve as a backup. This might also serve as a learning for me, in understanding some commands better.

I used

Code: Select all

cp -a /source/of/data/. /destination/for/data/ 

Here is my source of the information:
https://askubuntu.com/questions/86822/h ... nt-directo

As I understand, the -a flag causes
-hidden files to be copied (this is good, I want that)
-file permissions to be preserved (this one has me concerned)
With file permissions, I am unsure as to what that means for the future when I am copying the files back. When I am copying the files onto another computer, how will that be looked at in these situation:
-to a user's directory. Will it need to be the same username as the "source" computer?
-to the home directory, not part of the users directory

I guess my question is, do I want the permissions to look exactly the same (source vs files on the USB thumb drive) so that the files can be successfully transferred to another computer?

Both the source install and the destination install will be ext4 file format.

[WharfRat]

A much better alternative to cp for backups is rsync

As per your example rsync -aviuh --progress --stats --delete-after /source/of/data/ /destination/for/data/

Once you sync the folders only updated files will be transferred to the destination

[Netherprovinc3]

As per your example rsync -aviuh --progress --stats --delete-after /source/of/data/ /destination/for/data/

Interesting. I am reading up on the syntax.
Now I'm curious about Timeshift's source code. Specifically wondering if they use the same command at the very core of the program. Investigating...

[deck_luck]

Indeed Time Shift uses rsync on the back end. Most desktop sync application are GUI wrappers using rsync on the back end process. The rsync utility is a very powerful tool for managing directory/file changes.

Also, I use rsync on MAC OS to do custom snapshots of in progress projects instead of waiting for Time Machine to run a full snapshot. Also, I keep a "near online" storage on a USB drive. When I make enough changes to my near online storage I propagate the changes to my daily, weekly, and monthly backup storage devices using custom rsync scripts.

The grsync application may be of interest to you as well. It as a gui for the rsync utility. You can install it from the Linux Mint Software Manager by searching for grsync.

I think you will find the rsync utility to be very useful.

[Netherprovinc3]

A much better alternative to cp for backups is rsync
As per your example rsync -aviuh --progress --stats --delete-after /source/of/data/ /destination/for/data/

I am a little bit concerned about the -u
u: forces rsync to skip any files for which the destination file already exists and has a date later than the source file.
I am assuming that the file path and the time stamp is all that's being used.

Let me describe an unintended consequence.
First let me point out that changing the name of a text file does not change the last modified date (I just tested this in Nemo).
On to our example. Let's say I have the files, "stuff to do this week" (modified on 7/1) "stuff to do next week" (modified on 6-30) I run the command, to back up each of these.
I then delete "stuff to do this week" and rename "stuff to do next week" to "stuff to do this week"
Now when we run the command, "stuff to do this week" (6-30) doesn't overwrite "stuff to this week" (7-1)
Might not be the best example but it illustrates a point.

So, I think you should get rid of the -u
That way, rsync will use a "quick check" that (by default) checks if each file's size and time of last modification match between the sender and receiver.
according to https://ss64.com/bash/rsync_options.html in the section under
-c, --checksum
Granted, your initial proposal should run faster.

[all41]

I am not seeing a progress display.
My simple script is:
rsync -r -t -v --progress -s /mnt/storage/canon/t6 /media/uno/Backup
where the source is on a mounted hd and the destination is on a mounted sd.
The directory is copied *.* but no progress display

[jglen490]

What kind of progress do you expect?

Here's one of my rsync commands:

Code: Select all

sudo rsync -auv /home /media/john/Backup1

When I kick this off, the target disk starts flashing (it's a drive inside an enclosure) and the files being copied scroll down the terminal page. One thing to remember with the "u" option, is if you move a file from one directory to another between backups, that file ends up twice on the target drive. One for the original location and one for the moved to location at the source.

I have four different drives: Backup1, Backup2, Backup3, and Backup4. Each with a similar command structure. And I rotate them on a schedule. Every once in a while, it's necessary to just clean up a target drive; so I choose three rotations and use the following command:

Code: Select all

sudo rm -rf /media/john/Backup1/*

Again one for each drive.

With the "U" option, a backup typically takes a few minutes in the second and later cycles. The first can take a long time, depending on your data.

Like with any backup solution, it will take some time to manage. But once you have a process, it just works.

[WharfRat]

I am not seeing a progress display.
My simple script is:
rsync -r -t -v --progress -s /mnt/storage/canon/t6 /media/uno/Backup
where the source is on a mounted hd and the destination is on a mounted sd.
The directory is copied *.* but no progress display

Add the -i option

From the man page

-i, --itemize-changes output a change-summary for all updates

[Netherprovinc3]

I am confused about the resulting file permissions.
I just formatted part of an ssd, using gparted
The owner and the group for the partition both seem to be root. So now to use rsync, I would have to run the command as sudo (I am guessing). I vaguely remember reading that running some copy/backup commands can change the file permissions. Maybe my memory is off on that, though.
Maybe just best to just keep the source owner and group the same as destination owner and group?

I appreciate learning some of the commands that drive backup programs. Maybe some day I will look at source code instead of searching the interwebs

[deck_luck]

You can always create a sub directory under the mount point and set the owner and permissions to whatever you desire. Also, if the whole file system is only used by one user you can change the root of the file system permissions to only allow that user access. It is up to you to decide the permissions. Remember, Linux based operating system runs in enterprise multi-user environments, and permissions are crucial to keep individual home directories private. Likewise to protect the operating system files from accidental or malware intentional changes, key area of the operating system have strict permissions only allowing root write access. The user, group, and other permissions are very necessary part of Linux.

20191028 edit: Removed a confusing last paragraph meant for another thread. I aplogized for any confusion it may have caused.

[jglen490]

I am confused about the resulting file permissions.
I just formatted part of an ssd, using gparted
The owner and the group for the partition both seem to be root. So now to use rsync, I would have to run the command as sudo (I am guessing). I vaguely remember reading that running some copy/backup commands can change the file permissions. Maybe my memory is off on that, though.
Maybe just best to just keep the source owner and group the same as destination owner and group?

I appreciate learning some of the commands that drive backup programs. Maybe some day I will look at source code instead of searching the interwebs

I ALWAYS use sudo with rsync because I backup the entire /home, and I've observed NO file or directory permissions being changed at the target drive. There are cases where using sudo with some GUI programs can result in permissions changes.