Backing up files to USB drive

Questions about other topics - please check if your question fits better in another category before posting here
Forum rules
Before you post please read how to get help
Post Reply
Netherprovinc3
Level 3
Level 3
Posts: 137
Joined: Mon Feb 04, 2019 9:29 pm

Backing up files to USB drive

Post by Netherprovinc3 » Fri Aug 09, 2019 10:14 pm

Even though there are more sophisticated ways to backup files, I am just manually copying files to a USB thumb drive and having that serve as a backup. This might also serve as a learning for me, in understanding some commands better.

I used

Code: Select all

cp -a /source/of/data/. /destination/for/data/ 
Here is my source of the information:
https://askubuntu.com/questions/86822/h ... nt-directo

As I understand, the -a flag causes
-hidden files to be copied (this is good, I want that)
-file permissions to be preserved (this one has me concerned)
With file permissions, I am unsure as to what that means for the future when I am copying the files back. When I am copying the files onto another computer, how will that be looked at in these situation:
-to a user's directory. Will it need to be the same username as the "source" computer?
-to the home directory, not part of the users directory

I guess my question is, do I want the permissions to look exactly the same (source vs files on the USB thumb drive) so that the files can be successfully transferred to another computer?

Both the source install and the destination install will be ext4 file format.
Linux Mint 19.1 Cinnamon

User avatar
WharfRat
Level 21
Level 21
Posts: 13022
Joined: Thu Apr 07, 2011 8:15 pm

Re: Backing up files to USB drive

Post by WharfRat » Sat Aug 10, 2019 3:12 am

A much better alternative to cp for backups is rsync

As per your example rsync -aviuh --progress --stats --delete-after /source/of/data/ /destination/for/data/

Once you sync the folders only updated files will be transferred to the destination :wink:
ImageImage

Netherprovinc3
Level 3
Level 3
Posts: 137
Joined: Mon Feb 04, 2019 9:29 pm

Re: Backing up files to USB drive

Post by Netherprovinc3 » Sat Aug 10, 2019 3:31 pm

WharfRat wrote:
Sat Aug 10, 2019 3:12 am
As per your example rsync -aviuh --progress --stats --delete-after /source/of/data/ /destination/for/data/
Interesting. I am reading up on the syntax.
Now I'm curious about Timeshift's source code. Specifically wondering if they use the same command at the very core of the program. Investigating...
Linux Mint 19.1 Cinnamon

deck_luck
Level 3
Level 3
Posts: 150
Joined: Mon May 27, 2019 6:57 pm
Location: here

Re: Backing up files to USB drive

Post by deck_luck » Sat Aug 10, 2019 4:19 pm

Indeed Time Shift uses rsync on the back end. Most desktop sync application are GUI wrappers using rsync on the back end process. The rsync utility is a very powerful tool for managing directory/file changes.

Also, I use rsync on MAC OS to do custom snapshots of in progress projects instead of waiting for Time Machine to run a full snapshot. Also, I keep a "near online" storage on a USB drive. When I make enough changes to my near online storage I propagate the changes to my daily, weekly, and monthly backup storage devices using custom rsync scripts.

The grsync application may be of interest to you as well. It as a gui for the rsync utility. You can install it from the Linux Mint Software Manager by searching for grsync.

I think you will find the rsync utility to be very useful.
💡Give a man a fish, and you feed him for a day. Teach a man to fish, and you feed him for a lifetime. ✝️

Netherprovinc3
Level 3
Level 3
Posts: 137
Joined: Mon Feb 04, 2019 9:29 pm

Re: Backing up files to USB drive

Post by Netherprovinc3 » Sat Aug 10, 2019 10:20 pm

WharfRat wrote:
Sat Aug 10, 2019 3:12 am
A much better alternative to cp for backups is rsync
As per your example rsync -aviuh --progress --stats --delete-after /source/of/data/ /destination/for/data/
I am a little bit concerned about the -u
u: forces rsync to skip any files for which the destination file already exists and has a date later than the source file.
I am assuming that the file path and the time stamp is all that's being used.

Let me describe an unintended consequence.
First let me point out that changing the name of a text file does not change the last modified date (I just tested this in Nemo).
On to our example. Let's say I have the files, "stuff to do this week" (modified on 7/1) "stuff to do next week" (modified on 6-30) I run the command, to back up each of these.
I then delete "stuff to do this week" and rename "stuff to do next week" to "stuff to do this week"
Now when we run the command, "stuff to do this week" (6-30) doesn't overwrite "stuff to this week" (7-1)
Might not be the best example but it illustrates a point.

So, I think you should get rid of the -u
That way, rsync will use a "quick check" that (by default) checks if each file's size and time of last modification match between the sender and receiver.
according to https://ss64.com/bash/rsync_options.html in the section under
-c, --checksum
Granted, your initial proposal should run faster.
Last edited by Netherprovinc3 on Sat Aug 10, 2019 10:46 pm, edited 2 times in total.
Linux Mint 19.1 Cinnamon

all41
Level 14
Level 14
Posts: 5473
Joined: Tue Dec 31, 2013 9:12 am
Location: Computer, Car, Cage

Re: Backing up files to USB drive

Post by all41 » Sat Aug 10, 2019 10:36 pm

I am not seeing a progress display.
My simple script is:
rsync -r -t -v --progress -s /mnt/storage/canon/t6 /media/uno/Backup
where the source is on a mounted hd and the destination is on a mounted sd.
The directory is copied *.* but no progress display

jglen490
Level 5
Level 5
Posts: 585
Joined: Sat Jul 15, 2017 9:57 pm

Re: Backing up files to USB drive

Post by jglen490 » Sat Aug 10, 2019 11:11 pm

What kind of progress do you expect?

Here's one of my rsync commands:

Code: Select all

sudo rsync -auv /home /media/john/Backup1
When I kick this off, the target disk starts flashing (it's a drive inside an enclosure) and the files being copied scroll down the terminal page. One thing to remember with the "u" option, is if you move a file from one directory to another between backups, that file ends up twice on the target drive. One for the original location and one for the moved to location at the source.

I have four different drives: Backup1, Backup2, Backup3, and Backup4. Each with a similar command structure. And I rotate them on a schedule. Every once in a while, it's necessary to just clean up a target drive; so I choose three rotations and use the following command:

Code: Select all

sudo rm -rf /media/john/Backup1/*
Again one for each drive.

With the "U" option, a backup typically takes a few minutes in the second and later cycles. The first can take a long time, depending on your data.

Like with any backup solution, it will take some time to manage. But once you have a process, it just works.
I feel more like I do than I did when I got here.
Toshiba A135-S2386, Intel T2080, ATI Radeon® Xpress 200M Chipset, 2GB RAM, 500GB

User avatar
WharfRat
Level 21
Level 21
Posts: 13022
Joined: Thu Apr 07, 2011 8:15 pm

Re: Backing up files to USB drive

Post by WharfRat » Sun Aug 11, 2019 8:25 am

all41 wrote:
Sat Aug 10, 2019 10:36 pm
I am not seeing a progress display.
My simple script is:
rsync -r -t -v --progress -s /mnt/storage/canon/t6 /media/uno/Backup
where the source is on a mounted hd and the destination is on a mounted sd.
The directory is copied *.* but no progress display
Add the -i option

From the man page
-i, --itemize-changes output a change-summary for all updates
ImageImage

Netherprovinc3
Level 3
Level 3
Posts: 137
Joined: Mon Feb 04, 2019 9:29 pm

Re: Backing up files to USB drive

Post by Netherprovinc3 » Mon Aug 12, 2019 8:06 pm

I am confused about the resulting file permissions.
I just formatted part of an ssd, using gparted
The owner and the group for the partition both seem to be root. So now to use rsync, I would have to run the command as sudo (I am guessing). I vaguely remember reading that running some copy/backup commands can change the file permissions. Maybe my memory is off on that, though.
Maybe just best to just keep the source owner and group the same as destination owner and group?

I appreciate learning some of the commands that drive backup programs. Maybe some day I will look at source code instead of searching the interwebs :)
Linux Mint 19.1 Cinnamon

deck_luck
Level 3
Level 3
Posts: 150
Joined: Mon May 27, 2019 6:57 pm
Location: here

Re: Backing up files to USB drive

Post by deck_luck » Tue Aug 13, 2019 5:50 pm

You can always create a sub directory under the mount point and set the owner and permissions to whatever you desire. Also, if the whole file system is only used by one user you can change the root of the file system permissions to only allow that user access. It is up to you to decide the permissions. Remember, Linux based operating system runs in enterprise multi-user environments, and permissions are crucial to keep individual home directories private. Likewise to protect the operating system files from accidental or malware intentional changes, key area of the operating system have strict permissions only allowing root write access. The user, group, and other permissions are very necessary part of Linux.

Ok what do you want? If you want a way to easily share files between different computers on a USB hard drive you could use FAT32. It will be read and write accessible on Windows or Linux, and you will not have deal with user permissions requiring root to change the permissions. This is a good method for sharing picture, videos, or music without any user permission concerns.
💡Give a man a fish, and you feed him for a day. Teach a man to fish, and you feed him for a lifetime. ✝️

jglen490
Level 5
Level 5
Posts: 585
Joined: Sat Jul 15, 2017 9:57 pm

Re: Backing up files to USB drive

Post by jglen490 » Wed Aug 14, 2019 10:33 pm

Netherprovinc3 wrote:
Mon Aug 12, 2019 8:06 pm
I am confused about the resulting file permissions.
I just formatted part of an ssd, using gparted
The owner and the group for the partition both seem to be root. So now to use rsync, I would have to run the command as sudo (I am guessing). I vaguely remember reading that running some copy/backup commands can change the file permissions. Maybe my memory is off on that, though.
Maybe just best to just keep the source owner and group the same as destination owner and group?

I appreciate learning some of the commands that drive backup programs. Maybe some day I will look at source code instead of searching the interwebs :)
I ALWAYS use sudo with rsync because I backup the entire /home, and I've observed NO file or directory permissions being changed at the target drive. There are cases where using sudo with some GUI programs can result in permissions changes.
I feel more like I do than I did when I got here.
Toshiba A135-S2386, Intel T2080, ATI Radeon® Xpress 200M Chipset, 2GB RAM, 500GB

Post Reply

Return to “Other topics”