rsync

Questions about other topics - please check if your question fits better in another category before posting here
Forum rules
Before you post read how to get help. Topics in this forum are automatically closed 6 months after creation.
Locked
Odipides
Level 1
Level 1
Posts: 19
Joined: Fri May 25, 2012 7:13 am

rsync

Post by Odipides »

I do this exercise twice a year. I read a review of rsync and how great it is for backing stuff up and resolve to create an rsync crontab job to sync my local home directory with a backup dir on the file server.

I always give up about an hour later.

The problem I always hit is that, while it's easy to get rsync to copy, I can never get it to NOT copy if a file is already on the remote drive and has the same timestamp, size and checksum as the one on my local folder. This is a problem because, the first time you run the sync, rsync always copies EVERYTHING! So, if your backup is 250GB, it takes about 24 hours to do the first sync (over a wifi network), despite the fact that the files are already on the remote and completely up to date (having copied them across using a USB 3.0 drive in anticipation of having partial file syncing)!

This is the rsync command I'm using:

Code: Select all

rsync -az -i -u -E --checksum -p -e ssh  /home/myname  myname@192.168.1.72:/home/myname/myname-backups/myname-home-dir
--checksum (allegedly) makes rsync just look at the checksum of each file and -u (allegedly) indicates it should skip the copy if the remote file is up to date.

But it still copies the lot.

I've checked up on a load of stuff about rsync and it seems a lot of people have the same problem. I wondered if anyone had managed to get this working OR if someone could recommend a syncing tool that actually does what it says on the tin.
Last edited by LockBot on Wed Dec 28, 2022 7:16 am, edited 1 time in total.
Reason: Topic automatically closed 6 months after creation. New replies are no longer allowed.
bimsebasse

Re: rsync

Post by bimsebasse »

Try the frontend Grsync. It does what you describe without fail for me.
Odipides
Level 1
Level 1
Posts: 19
Joined: Fri May 25, 2012 7:13 am

Re: rsync

Post by Odipides »

Tried it, still seems to want to copy all the time.

Maybe it's a file system or time sync problem.
mintybits

Re: rsync

Post by mintybits »

What filesystem is it?
Once you've done rsync once (at least on the first file before nyou give up) if you then do it again presumably it doesn't rewrite the files?

I've never found any problems with rsync, in fact it is one of my favourite command line tools. What method are you using to copy the files to the 3GB drive and then to copy them to the target? Is the filesystem on the 3GB the same?
You might try rsync'ing a single file and then comparing all its attributes with the same file that you copied via the 3GB drive (in different directories, obviously).

Your -e option isn't necessary as rsync defaults to ssh. -a includes -p. I haven't used the --checksum myself but bear in mind that this causes rsync to calculate the checksum of the files on both sides and this can take some time. I usually just use rsync -vax.
Odipides
Level 1
Level 1
Posts: 19
Joined: Fri May 25, 2012 7:13 am

Re: rsync

Post by Odipides »

Hi,
Thanks for the help. Got it working (didn't give up after an hour this time - lol)

Problem was in fact the missing trailing '/' - so it was, in fact, copying everything into a subdirectory in the target. However, because there are so many files in the destination root sub-dir I hadn't noticed; dim of me really.

Here's the final command line:

Code: Select all

rsync -e ssh -r -t -p -g -v --progress --delete --ignore-existing -u --modify-window=1 -c -l -z -s /home/<username>/    <username>@192.168.20.72:/home/<username>/<username>-backups/<username>-home-dir/
(I've left the -e ssh in in case someone is using a different ssh client)
Locked

Return to “Other topics”