rsync

Questions about other topics - please check if your question fits better in another category before posting here
Forum rules
Before you post please read this

rsync

Postby Odipides on Mon Jun 25, 2012 11:47 am

I do this exercise twice a year. I read a review of rsync and how great it is for backing stuff up and resolve to create an rsync crontab job to sync my local home directory with a backup dir on the file server.

I always give up about an hour later.

The problem I always hit is that, while it's easy to get rsync to copy, I can never get it to NOT copy if a file is already on the remote drive and has the same timestamp, size and checksum as the one on my local folder. This is a problem because, the first time you run the sync, rsync always copies EVERYTHING! So, if your backup is 250GB, it takes about 24 hours to do the first sync (over a wifi network), despite the fact that the files are already on the remote and completely up to date (having copied them across using a USB 3.0 drive in anticipation of having partial file syncing)!

This is the rsync command I'm using:
Code: Select all
rsync -az -i -u -E --checksum -p -e ssh  /home/myname  myname@192.168.1.72:/home/myname/myname-backups/myname-home-dir

--checksum (allegedly) makes rsync just look at the checksum of each file and -u (allegedly) indicates it should skip the copy if the remote file is up to date.

But it still copies the lot.

I've checked up on a load of stuff about rsync and it seems a lot of people have the same problem. I wondered if anyone had managed to get this working OR if someone could recommend a syncing tool that actually does what it says on the tin.
Odipides
Level 1
Level 1
 
Posts: 19
Joined: Fri May 25, 2012 7:13 am

Linux Mint is funded by ads and donations.
 

Re: rsync

Postby bimsebasse on Mon Jun 25, 2012 1:02 pm

Try the frontend Grsync. It does what you describe without fail for me.
Thank you for this thread. That’s all I can say. You most definitely have made this forum into something special. You clearly know what you are doing, you’ve covered so many bases. Thanks!
User avatar
bimsebasse
Level 7
Level 7
 
Posts: 1709
Joined: Fri Nov 11, 2011 10:21 am
Location: Scandinavia

Re: rsync

Postby Odipides on Mon Jun 25, 2012 1:23 pm

Tried it, still seems to want to copy all the time.

Maybe it's a file system or time sync problem.
Odipides
Level 1
Level 1
 
Posts: 19
Joined: Fri May 25, 2012 7:13 am

Re: rsync

Postby mintybits on Mon Jun 25, 2012 3:05 pm

What filesystem is it?
Once you've done rsync once (at least on the first file before nyou give up) if you then do it again presumably it doesn't rewrite the files?

I've never found any problems with rsync, in fact it is one of my favourite command line tools. What method are you using to copy the files to the 3GB drive and then to copy them to the target? Is the filesystem on the 3GB the same?
You might try rsync'ing a single file and then comparing all its attributes with the same file that you copied via the 3GB drive (in different directories, obviously).

Your -e option isn't necessary as rsync defaults to ssh. -a includes -p. I haven't used the --checksum myself but bear in mind that this causes rsync to calculate the checksum of the files on both sides and this can take some time. I usually just use rsync -vax.
mintybits
Level 5
Level 5
 
Posts: 827
Joined: Fri Jan 27, 2012 5:09 pm

Re: rsync

Postby Odipides on Tue Jun 26, 2012 6:00 am

Hi,
Thanks for the help. Got it working (didn't give up after an hour this time - lol)

Problem was in fact the missing trailing '/' - so it was, in fact, copying everything into a subdirectory in the target. However, because there are so many files in the destination root sub-dir I hadn't noticed; dim of me really.

Here's the final command line:
Code: Select all
rsync -e ssh -r -t -p -g -v --progress --delete --ignore-existing -u --modify-window=1 -c -l -z -s /home/<username>/    <username>@192.168.20.72:/home/<username>/<username>-backups/<username>-home-dir/


(I've left the -e ssh in in case someone is using a different ssh client)
Odipides
Level 1
Level 1
 
Posts: 19
Joined: Fri May 25, 2012 7:13 am


Return to Other topics

Who is online

Users browsing this forum: No registered users and 4 guests