when run from cron, script fails with "broken pipe"

[patrick.mooney]

Hello,

I'm using a script to automate a backup of some of my data to a remote server. It compresses the selected folders, encrypts them with GPG, and ships them off with scp. When run from the command line, the script works fine ... when run as a cron job, though, the script fails, and the log file that it creates just ends with:

Code: Select all

tar: -: Cannot write: Broken pipe
tar: Error is not recoverable: exiting now

Here's the script:

Code: Select all

# !/bin/bash
tar c -PSvv --to-stdout --exclude=".thumbnails" --exclude-tag-under=".SkipMe" /home/patrick/Documents/school /home/patrick/Documents/writing 2> /home/patrick/Desktop/remote-backup.log | bzip2 -z 2>> /home/patrick/Desktop/remote-backup.log | gpg2 -r 505AB18E --batch -o "/tmp/Backup.tar.bz2.gpg" -e - > /home/patrick/Desktop/remote-backup.log && scp /tmp/Backup.tar.bz2.gpg patrickmooney@ustorage.ucsb.edu:backups 2>> /home/patrick/Desktop/remote-backup.log
if [ -d /media/Externa/backups ]; then
  cp /tmp/Backup.tar.bz2.gpg /media/Externa/backups/
fi
rm /tmp/Backup.tar.bz2.gpg
chown patrick:patrick /home/patrick/Desktop/remote-backup.log

The script is run as root (i.e., installed with "sudo crontab -e"). Here's the cron line:

Code: Select all

30 6 * * * /home/patrick/.scripts/backup-remote.sh

I've tried searching the Mint and Ubuntu forums, plus Google, but nothing useful has turned up. I've been tearing my hair out over this for a month now. Sure, I have a lot of hair, but still. Any help is appreciated.

Currently running Linux Mint Julia on an HP Pavilion dv6000, 2GB RAM, 2 GHz Intel Core Duo.

[Pilosopong Tasyo]

The script is run as root...

I can't seem to find the logic why the script has to be run as root. I don't see any sudo commands anywhere in that script. The script runs fine under the user account, so why not just set it under the user's cron table instead of root's? The script will run at the designated time anyway even if the user account is not logged in.

[thaimann]

As I said in a previous post, cron provides very few global variables and this can cause your script to fail.

But if you run it like the following, it should get your user globals and should work;

58 * * * * /bin/su -c /home/user/myscript

[thaimann]

Sorry I said that wrong. It will run with root privileges.

[xenopeek]

Probably this is cron run as root has restrictive $PATH setting. Try setting at the top of your script:

Code: Select all

PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

[Pilosopong Tasyo]

cron provides very few global variables...

I looked at the code fragment again and I don't see any "$VARIABLEs" being used anywhere. Unless if you're referring to environment variables.

Anyway, I suggest the OP break up this portion of the code:

Code: Select all

tar c -PSvv --to-stdout --exclude=".thumbnails" --exclude-tag-under=".SkipMe" /home/patrick/Documents/school /home/patrick/Documents/writing 2> /home/patrick/Desktop/remote-backup.log | bzip2 -z 2>> /home/patrick/Desktop/remote-backup.log | gpg2 -r 505AB18E --batch -o "/tmp/Backup.tar.bz2.gpg" -e - > /home/patrick/Desktop/remote-backup.log && scp /tmp/Backup.tar.bz2.gpg patrickmooney@ustorage.ucsb.edu:backups 2>> /home/patrick/Desktop/remote-backup.log

into individual commands instead of putting everything as a single line of code, relying solely on piping "|" and redirection ">" as input to the next command.

You'd be amazed how easier it is to debug [programming] problems if you simplify code.

[Habitual]

add this to your code and see what happens to the log file?

Code: Select all

# !/bin/bash
set -x
...


[patrick.mooney]

Thanks for all the replies! I've modified the cron entry to try thaimann's idea (run with su -c) and the script should start again in a few minutes. I'll report back then.

[patrick.mooney]

Sorry for the delay getting back to you. Internet connectivity in the grad student dorms here seems to be highly intermittent over a holiday break.

None of the suggested fixes actually resolves the problem. Running the script with su -c in the cron job produces the same problem ... same with prefixing the cron command with sudo. Adding set -x to the beginning of the script either produces no change to the log, or adds a few hundred bytes of zeros to the beginning of it.

I could certainly break up the primary command into subcommands, but (a) this would require several hundred MB of extra scratch space, plus requiring that commands be run sequentially instead of concurrently, and (b) what is this, DOS? everyone keeps telling me that one of the advantages of the Linux command line/scripting system is its flexibility. I'd like to take advantage of this flexibility when writing a script. Ideally, I'd just like to get the piping working.

any help is much appreciated.

[patrick.mooney]

... and I notice now that executing the script from a user crontab, instead of the root crontab, seems to fix the problem. I'm scratching my head over why this is, but it seems to be working. Still, if anyone can explain why it would fail from the root user's account, I'd be interested in hearing. (=

[riffraff]

Hi there,

you should definetly change your first line to a shebang!

change this

Code: Select all

# !/bin/bash

to that

Code: Select all

#!/bin/bash

This is a big difference - just run

Code: Select all

file <scriptname>

on both variants and you will see that yours is a "ASCII-FILE" and, after correcting it, it will be a "Bourne-Again Shell Script".

Maybe that is your problem here at all