[solved] A curious backup issue

Forum rules
Before you post please read how to get help

[solved] A curious backup issue

Postby Cammo on Fri May 23, 2014 12:06 am

Hi forum,

I've made a backup bash script using rsync to sync files between my laptop and USB. Basically, I use rsync -uai x y to backup x to y. Now, the files transfer fine, but interestingly the script doesn't update old files on y with the newer versions from x. It just skips them. Isn't the -u switch supposed to over-write the old versions with the new one?

The script runs without any errors and the log files confirm that new files are being written, but old ones are being skipped rather than overwritten.

Any help is much appreciated.

Thanks,

Cammo

. The full source code is attached below, but

Code: Select all
#!/bin/bash
# Purpose:      For syncing files between two drives, where changes might occur
#              on either. (I use it to backup documents between a PC and USB)
#
#      Rsyncs $OLD-PATH to $NEW-PATH, ignoring newer files on $NEW-PATH. Then, rsyncs $NEW-PATH onto $OLD-PATH, ignoring newer files.
#      The output is redirected to $logLocation, so you can grep through what's been copied later at your leisure.

# Usage:      ./ backup-script.sh
# Author:      Cammo
# Date:        Mon 12 May 2014
# Version:      0.5
# Disclaimer:   Feel free to use it, but don't blame me if your computer conks out.

#=====================================================================
#              Enter your required params here...
OldPath='/mnt/Storage/Uni/'
mountPoint='/media/cameron/UUI'
NewPath="$mountPoint/Uni/"
myName='cameron'

now=$(date +"%F")
logLocation="/home/$myName/Documents/BackupLogs/backup-output-$now.txt"
#=====================================================================

chown $myName: $mountPoint; chown $myName: $logLocation
mount '/dev/sdb1' $mountPoint>>$logLocation
mkdir "$mountPoint/Uni">>$logLocation
echo "">>$logLocation
echo "">>$logLocation
echo $(date +"%c")>>$logLocation
echo ''>>$logLocation
echo 'Hard drive path = ' $OldPath>>$logLocation
echo 'USB path = ' $NewPath>>$logLocation
echo ''>>$logLocation
echo "Starting  backup from $OldPath to $NewPath" >>$logLocation
echo ''>>$logLocation


rsync -uai $OldPath $NewPath>>$logLocation
echo 'Syncing Old->New done'>>$logLocation
rsync -uai $NewPath $OldPath>>$logLocation
echo 'Syncing New->Old done'>>$logLocation
chown -R $myName: $NewPath>>$logLocation
echo 'taking ownership of new files -> done'>>$logLocation
Last edited by Cammo on Sat May 31, 2014 2:06 am, edited 1 time in total.
Cammo
Level 2
Level 2
 
Posts: 60
Joined: Tue Apr 08, 2014 8:19 pm

Linux Mint is funded by ads and donations.
 

Re: A curious backup issue

Postby WharfRat on Fri May 23, 2014 12:52 am

Try this

Create a file in an empty new folder

Code: Select all
echo "I was created on $(date)" > afile


rsync the file to the flash drive then update afile

Code: Select all
echo "I was updated on $(date)" > afile


Add -n to the rsync options to see if it is detected to get synced e.g., rsync -naui

-n is a trial run, nothing will actually get updated. If it is listed then remove the -n, let it run then diff the files.

Good luck :wink:
Image
User avatar
WharfRat
Level 12
Level 12
 
Posts: 4450
Joined: Thu Apr 07, 2011 8:15 pm

Re: A curious backup issue

Postby Cammo on Mon May 26, 2014 2:07 am

Sorry for the late reply WharfRat, I've been bogged down in assignments over the last few days. Getting ready for exam season, and all the joys that entails!

Ok, I ran WharfRat's suggestion, here's the code I used:

Code: Select all
cameron@cameron-ThinkPad-X220 /mnt/Storage/Test $ echo  "I was created on $(date)" > test.txt
cameron@cameron-ThinkPad-X220 /mnt/Storage/Test $ rsync -a /mnt/Storage/Test/ /media/cameron/sdcard/Test/
rsync: failed to set times on "/media/cameron/sdcard/Test/.": Operation not permitted (1)
rsync: mkstemp "/media/cameron/sdcard/Test/.test.txt.OORhUA" failed: Permission denied (13)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1070) [sender=3.0.9]
cameron@cameron-ThinkPad-X220 /mnt/Storage/Test $ sudo !!
sudo rsync -a /mnt/Storage/Test/ /media/cameron/sdcard/Test/
[sudo] password for cameron:
cameron@cameron-ThinkPad-X220 /mnt/Storage/Test $ echo  "I was modified on $(date)" > test.txt
cameron@cameron-ThinkPad-X220 /mnt/Storage/Test $ sudo rsync -uain /mnt/Storage/Test/ /media/cameron/sdcard/Test/
.d...p..... ./
>f.stp..... test.txt
cameron@cameron-ThinkPad-X220 /mnt/Storage/Test $ sudo rsync -uai /mnt/Storage/Test/ /media/cameron/sdcard/Test/
.d...p..... ./
>f.stp..... test.txt


When I opened up the backed up copy of test.txt, it was "I was modified..." etc as expected. But when I ran the same command from my script, it did the same thing it did before - copied new files over, but didn't overwrite modifications.

I note, however that my tests only ran when I sudo'd rsync. I'm guessing this is because /media/cameron/sdcard is owned by root, so my normal user can't touch it. I thought that root privileges was the problem, so I tried running my backup script as root - both by sudo'ing it and even (out of desperation) su'ing and entering in the code line by line. Close, but still no cigar.

So, I tried chown'ing the mount point to cameron (my normal user), but got the following error:

Code: Select all
cameron@cameron-ThinkPad-X220 /mnt/Storage/Test $ sudo chown -R cameron: /media/cameron/sdcard
chown: changing ownership of ‘/media/cameron/sdcard/Test’: Operation not permitted


And the same error when I logged in as root actual (sudo -i) (which is strange, because that account let me fix grub up when it died, and I can't imagine my backup point needing higher privileges than that!)

Thanks as always,

Cammo
Cammo
Level 2
Level 2
 
Posts: 60
Joined: Tue Apr 08, 2014 8:19 pm

Re: A curious backup issue

Postby WharfRat on Mon May 26, 2014 10:30 am

Cammo,

Yes, the problems you're experiencing is all permissions, i.e., not being able to update the file's timestamp.

I use rsync -aviuh and never have a problem as long as the permissions are set correctly.

What's the file system of the sdcard :?: If it's not a linux file system then that's most likely the problem.
Image
User avatar
WharfRat
Level 12
Level 12
 
Posts: 4450
Joined: Thu Apr 07, 2011 8:15 pm

Re: A curious backup issue

Postby Cammo on Tue May 27, 2014 4:22 am

Hi WharfRat,

My SD card is a 32G SD-HC, which registers as vfat. I've checked that it isn't write-protected and I can write to it by sudo'ing - except, oddly enough, with rsync --update.

Do you have any ideas on how to fix the problem? I'd prefer to avoid reformatting my sdcard to ext-n (because I also use it on a Win system) if at all possible.

I've tried using the chown command to take over the mount point, but as mentioned in my last post, I get permission denied errors, even when running it as root.
Cammo
Level 2
Level 2
 
Posts: 60
Joined: Tue Apr 08, 2014 8:19 pm

Re: A curious backup issue

Postby WharfRat on Tue May 27, 2014 12:08 pm

Cammo,

The FAT filesystems do not understand Linux permissions at all. If you backed-up anything that's permission critical like su or umount in /bin that has the setuid bit set it's not going to restore with those permissions.

You can try mounting the partition with the options uid=1000,gid=1000 for access while backing-up - that's about it.

Good luck :wink:
Image
User avatar
WharfRat
Level 12
Level 12
 
Posts: 4450
Joined: Thu Apr 07, 2011 8:15 pm

Re: A curious backup issue

Postby Cammo on Sat May 31, 2014 2:06 am

Thanks, that fixed it for me.

Much appreciated :-)
Cammo
Level 2
Level 2
 
Posts: 60
Joined: Tue Apr 08, 2014 8:19 pm

Linux Mint is funded by ads and donations.
 

Return to Scripts & Bash

Who is online

Users browsing this forum: No registered users and 2 guests

cron