[Solved] Making a backup script

[Cammo]

Hi all, I'm trying to make a backup script that runs when I plug my USB in, following this guide: http://stackoverflow.com/questions/4189 ... th-a-direc

My script works fine and does what I want it to do when I call it from the console. But, for completeness, here it is:

Code: Select all

#!/bin/bash
# Purpose:   	For syncing files between two drives, where changes might occur 
#	     	on either. (I use it to backup documents between a PC and USB)
#
#		Rsyncs $OLD-PATH to $NEW-PATH, ignoring newer files on $NEW-PATH. Then, rsyncs $NEW-PATH onto $OLD-PATH, ignoring newer files.
#
# Usage:   	./ backup-script.sh
# Author:   	Cammo
# Date:    	Thur 8 May 2014
# Version:   	0.1
# Disclaimer:   Feel free to use it, but don't blame me if your computer conks out.


OldPath='/mnt/Storage/Uni/'
NewPath='/media/cameron/UUI/UniStuff/Uni/'

echo 'Hard drive path = ' $OldPath 
echo 'USB path = ' $NewPath 

rsync -uax --progress $OldPath $NewPath
rsync -uax --progress $NewPath $OldPath

echo ''
echo 'All done!'

Now, I've made a file called 90-usb-backup.rules in /etc/udev/rules.d/
It contains:

Code: Select all

ACTION=="add", SUBSYSTEM=="usb_device", SYSFS{idVendor}=="0781", SYSFS{idProduct}=="5576", RUN+="/mnt/Data/Documents/backup-script.sh"

This is where I suspect the problem is because my backup program runs fine from the command line, so long as the USB is plugged in. Does anyone have any ideas on what I'm doing wrong? Thanks in advance!

EDIT: And I've checked the idVendor and idProduct match up with that shown for my thumbdrive in lsusb (confirmed by running lsusb twice, once with my thumb drive plugged in and once with it removed).

[WharfRat]

Cammo,

I haven't seen SYSFS{idVendor} with the output of udevadm

Code: Select all

bill@mint ~ $ udevadm info  --attribute-walk /dev/sdg|grep 154b
    ATTRS{idVendor}=="154b"
bill@mint ~ $ 

Try ATTRS{idVendor}=="0781", ATTRS{idProduct}=="5576"

[Cammo]

Hi WharfRat,

Thanks for your help! You were right, I ran a version of your script and it returned ATTRS instead of SYSID. I've tried amending my udev rule, restarting my computer, but it still doesn't run my script. I've double checked that it is executable and in the location the rule looks in. All that happens when I plug the USB in question in is the default behaviour - opening another instance of nautilus. I've had a look in my processes, and don't see an instance of the script running (being a backup script, it takes a while to execute).

Do you have any other troubleshooting advice? I'm frankly stumped.

PS: my amended udev rule is: 90-usb-backup.rules

Code: Select all

ACTION=="add", SUBSYSTEM=="usb_device", ATTRS{idVendor}=="0781", ATTRS{idProduct}=="5576", RUN+="/mnt/Data/Documents/backup-script.sh"

And running your script shows that 0781 and 5576 are the right numbers for those fields (my USB is on sdb, with only the default vfat partitioning)

Thanks again,
Cammo

[WharfRat]

Cammo,

I'm not sure the ACTION attribute is doing you any good there. I just did a simple rule for a usb stick to use as an example mounted at /dev/sdg1. This is the rule in /etc/udev/rules.d/88-dongle.rules

Code: Select all

DRIVERS=="usb",ATTRS{idVendor}=="1d6b",ATTRS{idProduct}=="0002",RUN+="/home/bill/script/88-dongle.sh"

Plug the drive in and check where it's mounted then run

Code: Select all

sudo udevadm trigger --verbose --dry-run|grep sdg

Use the first line which will look something like /sys/devices/pci0000:00/0000:00:12.2/usb1/1-5/1-5:1.0/host7/target7:0:0/7:0:0:0/block/sdg and run

Code: Select all

sudo udevadm test /sys/devices/pci0000:00/0000:00:12.2/usb1/1-5/1-5:1.0/host7/target7:0:0/7:0:0:0/block/sdg/

There's going to be a lot of output, but what you're looking for is lines that let you know if your rules were included as follows:

read rules file: /etc/udev/rules.d/88-dongle.rules
RUN '/home/bill/script/88-dongle.sh' /etc/udev/rules.d/88-dongle.rules:1

And at the end of the output a summary of the actions

run: '/lib/udev/hdparm'
run: '/home/bill/script/88-dongle.sh'
run: '/sbin/modprobe -qba dm-multipath'

Good luck

[Cammo]

Ok, so far you're right - running the udevadm test code with the original file gives an error saying invalid use of ACTION. I've made the changes you recommended - I changed 90-usb-backup.rules to say:

Code: Select all

DRIVERS=="usb",ATTRS{idVendor}=="0781",ATTRS{idProduct}=="5576",RUN+="/mnt/Data/Documents/backup-script.sh"

Running your test once again, I did the following: (usb mounts to sdb1)

Code: Select all

su
udevadm trigger --verbose --dry-run|grep sdb
/* Returns 2 results - the top result is the one ending in /sdb, so */
udevadm trigger --verbose --dry-run|grep -m1 sdb
udevadm test $(!!)
exit

which returns: (among many other things)

Code: Select all

read rules file: /etc/udev/rules.d/90-usb-backup.rules
...
RUN '/mnt/Data/Documents/backup-script.sh' /etc/udev/rules.d/90-usb-backup.rules:1
...
run: '/mnt/Data/Documents/backup-script.sh'

So, it seems that the script SHOULD execute on device plug in, right? Now, restarting and re-plugging my usb only opens the normal nemo/nautilus box with the files in it, but there's no sign of the script running.

Just to check it's all working, I ran

Code: Select all

sudo udevadm trigger

which set off my file backup as expected. Or at least, I think it did because it didn't open up a shell terminal with the copy progress as I expected (the --progress option normally gives a running commentary of how rsync's going), but the files started appearing on my USB and my HDD light went nuts for a while.

It's certainly a lot better than what I had before, but still not quite there. I tried experimenting with the ACTION="add", but every time I put that into my rule and re-ran your diagnostic it gave me an invalid-ACTION error.

Any suggestions on where to from here?

Thanks again for your help,

Cammo

[WharfRat]

Cammo,

I didn't see a mount in your script You most likely mounted it before you ran the script from the cl.

This is what I put in mine

Code: Select all

#!/bin/bash
#
sleep 5s
sudo mount /dev/sdg1 /media/temp/
exit 0

and it did mount. However I should point out that I overrode the need to enter a password for sudo so that's probably the reason it worked for me.

Another thing is you have echo and rsync --progress. I 'm pretty sure that the udev environment is much like the cron environment, meaning there are no X DISPLAY, SESSION_MANAGER etc., etc.

To check change the script to

Code: Select all

#!/bin/bash
env >/home/yourname/udev.txt
exit

to see what you're working with.

Good luck

UPDATE: Out of curiosity I checked the udev environment and it seems I was wrong. It's even much different than a cron environment:

cron:

Code: Select all

bill@mint ~/script $ cat cronenv.txt 
SHELL=/bin/sh
PATH=/usr/bin:/bin
PWD=/home/bill
LANG=en_US.UTF-8
SHLVL=1
HOME=/home/bill
LOGNAME=bill
_=/usr/bin/env

udev:

Code: Select all

bill@mint ~ $ cat udev.txt 
ID_PART_ENTRY_SIZE=27074434
ID_USB_DRIVER=usb-storage
ID_MODEL=Cruzer_Glide
ID_PATH_TAG=pci-0000_00_12_2-usb-0_5_1_0-scsi-0_0_0_0
ID_MODEL_ENC=Cruzer\x20Glide\x20\x20\x20\x20
ID_REVISION=1.26
DEVTYPE=partition
ID_BUS=usb
SUBSYSTEM=block
ID_SERIAL=SanDisk_Cruzer_Glide_200517384213F9434809-0:0
ID_FS_UUID=bd67ddb9-f9d7-4b04-8b55-dfe054066b29
UDISKS_PARTITION_OFFSET=64512
DEVPATH=/devices/pci0000:00/0000:00:12.2/usb1/1-5/1-5:1.0/host7/target7:0:0/7:0:0:0/block/sdg/sdg1
ID_MODEL_ID=5575
ID_VENDOR_ENC=SanDisk\x20
UDISKS_PARTITION_NUMBER=1
ID_PART_ENTRY_FLAGS=0x80
ID_PART_ENTRY_DISK=8:96
ID_FS_VERSION=1.0
MINOR=97
ACTION=add
PWD=/
UDISKS_PARTITION_ALIGNMENT_OFFSET=0
ID_PART_ENTRY_SCHEME=dos
ID_PART_ENTRY_TYPE=0x83
ID_FS_TYPE=ext3
USEC_INITIALIZED=48340446
MAJOR=8
ID_DRIVE_THUMB=1
DEVLINKS=/dev/disk/by-id/usb-SanDisk_Cruzer_Glide_200517384213F9434809-0:0-part1 /dev/disk/by-path/pci-0000:00:12.2-usb-0:5:1.0-scsi-0:0:0:0-part1 /dev/disk/by-uuid/bd67ddb9-f9d7-4b04-8b55-dfe054066b29
ID_VENDOR_ID=0781
DEVNAME=/dev/sdg1
SHLVL=1
ID_FS_USAGE=filesystem
ID_TYPE=disk
ID_PART_TABLE_TYPE=dos
UDISKS_PRESENTATION_NOPOLICY=0
ID_PART_ENTRY_NUMBER=1
UDISKS_PARTITION_SIZE=13862110208
ID_FS_UUID_ENC=bd67ddb9-f9d7-4b04-8b55-dfe054066b29
UDISKS_PARTITION_SCHEME=mbr
UDISKS_PARTITION_TYPE=0x83
UDISKS_PARTITION=1
ID_PART_ENTRY_OFFSET=126
ID_INSTANCE=0:0
ID_VENDOR=SanDisk
ID_USB_INTERFACE_NUM=00
ID_SERIAL_SHORT=200517384213F9434809
UDISKS_PARTITION_SLAVE=/sys/devices/pci0000:00/0000:00:12.2/usb1/1-5/1-5:1.0/host7/target7:0:0/7:0:0:0/block/sdg
UDISKS_PARTITION_FLAGS=boot
SEQNUM=2244
ID_USB_INTERFACES=:080650:
ID_PATH=pci-0000:00:12.2-usb-0:5:1.0-scsi-0:0:0:0
_=/usr/bin/env

Bottom line is you're not running in an interactive shell when you setup a script to execute from a cron or udev script.

Good luck

[Cammo]

Hi Wharfrat,

That's it - I can't believe I overlooked mounting the USB before trying to write to it! :facepalm: I guess I'm so used to HAL (I think?) doing the hard yards of mounting it when I plug it in.

Anyway, that's solved it for me. Here's my solution in case it helps anyone else:

• 1. Run

  Code: Select all

  lsusb

  in your terminal[/b]
  
  2. Find your USB's idVendor and idProduct from the list. For example, my one says:

  Bus 001 Device 006: ID 0781:5576 SanDisk Corp.

  This means that my idVendor = 0781 and my idProduct = 5576.
  
  3. Make a file in /etc/udev/rules.d/ called 90-usb-backup.rules (or whatever)
  
  4. Edit that file so that it reads:

  Code: Select all

      DRIVERS=="usb",ATTRS{idVendor}=="0781",ATTRS{idProduct}=="5576",RUN+="/mnt/Data/Documents/backup-script.sh"

  where 0781 and 5576 are the numbers you got in step 2, and the dir after RUN+= is where your desired script is.
  
  5. Save and close the file, make sure the script exists and is marked executable, and that it mounts your usb device (or you won't be able to access it in your script). For example, I included a line like

  Code: Select all

  mount /dev/sdb1 /media/cameron/UUI

  
  in my script
  
  6. Then run:

  Code: Select all

  sudo udevadm control --reload-rules

  (it resets your udev rules)
  
  7. Plug your device in, and the script should run.
  NB: You won't see a console or anything, so if you want to see an output, redirect the commands to a logfile (for an example, see my attached backup-script.sh below)

And here's my backup script, in case it helps anyone:

Code: Select all

#!/bin/bash
# Purpose:      For syncing files between two drives, where changes might occur
#           	on either. (I use it to backup documents between a PC and USB)
#
#		Rsyncs $OLD-PATH to $NEW-PATH, ignoring newer files on $NEW-PATH. Then, rsyncs $NEW-PATH onto $OLD-PATH, ignoring newer files.
#		The output is redirected to $logLocation, so you can grep through what's been copied later at your leisure.

# Usage:      ./ backup-script.sh
# Author:      Cammo
# Date:        Mon 12 May 2014
# Version:      0.5
# Disclaimer:   Feel free to use it, but don't blame me if your computer conks out.

#=====================================================================
#              Enter your required params here...
OldPath='/mnt/Storage/Uni/'
NewPath='/media/cameron/UUI/UniStuff/Uni/'
mountPoint='/media/cameron/UUI'
myName='cameron'

now=$(date +"%F")
logLocation="/home/$myName/Documents/BackupLogs/backup-output-$now.txt"
#=====================================================================

chown $myName: $mountPoint; chown $myName: $logLocation
mount '/dev/sdb1' $mountPoint

echo 'Hard drive path = ' $OldPath
echo 'USB path = ' $NewPath

echo "">>$logLocation
echo "">>$logLocation
echo $(date +"%c")>>$logLocation
echo ''>>$logLocation
echo 'Hard drive path = ' $OldPath>>$logLocation
echo 'USB path = ' $NewPath>>$logLocation
echo ''>>$logLocation
echo "Starting  backup from $OldPath to $NewPath" >>$logLocation
echo ''>>$logLocation


rsync -uaih $OldPath $NewPath>>$logLocation
echo 'Syncing Old->New done'
rsync -uaih $NewPath $OldPath>>$logLocation
echo 'Syncing New->Old done'
chown -R $myName: $NewPath>>$logLocation
echo 'taking ownership of new files -> done'

(A summary of what I have learned so far, with grateful thanks to Wharfrat for his patience and kind assistance)

And by the way, thanks Wharfrat for the update about how different udev is to cron - it helped me solve my mount problem. Turns out there's no need to edit sudoers and make mount x y an exception (as I was originally planning on doing), or even to use the sudo command in my udev-called script. My reasoning was that if udev runs as root (like rc.local), then there's no point su'ing - it's already root! My above script works ok without the sudo command, so I guess I'm right. Also, thanks for your idea about redirecting env to a txt file - it gave me the idea of making a logfile so that no interactive shell is necessary.

Thanks again Wharfrat,

Cammo

[WharfRat]

That's it - I can't believe I overlooked mounting the USB before trying to write to it! :facepalm:

I have permanent hand prints imprinted on my face from similar experiences

Turns out there's no need to edit sudoers and make mount x y an exception (as I was originally planning on doing), or even to use the sudo command in my udev-called script.

Yes, that makes sense as udev would have to be able to create block/character files and manipulate /dev/ and /sys/. Also I later realized that the env >/home/yourname/udev.txt line created the file as root.

Thanks again Wharfrat

No problem