when run from cron, script fails with "broken pipe"

About writing shell scripts and making the most of your shell
Forum rules
Topics in this forum are automatically closed 6 months after creation.
Locked
patrick.mooney

when run from cron, script fails with "broken pipe"

Post by patrick.mooney »

Hello,

I'm using a script to automate a backup of some of my data to a remote server. It compresses the selected folders, encrypts them with GPG, and ships them off with scp. When run from the command line, the script works fine ... when run as a cron job, though, the script fails, and the log file that it creates just ends with:

Code: Select all

tar: -: Cannot write: Broken pipe
tar: Error is not recoverable: exiting now
Here's the script:

Code: Select all

# !/bin/bash
tar c -PSvv --to-stdout --exclude=".thumbnails" --exclude-tag-under=".SkipMe" /home/patrick/Documents/school /home/patrick/Documents/writing 2> /home/patrick/Desktop/remote-backup.log | bzip2 -z 2>> /home/patrick/Desktop/remote-backup.log | gpg2 -r 505AB18E --batch -o "/tmp/Backup.tar.bz2.gpg" -e - > /home/patrick/Desktop/remote-backup.log && scp /tmp/Backup.tar.bz2.gpg patrickmooney@ustorage.ucsb.edu:backups 2>> /home/patrick/Desktop/remote-backup.log
if [ -d /media/Externa/backups ]; then
  cp /tmp/Backup.tar.bz2.gpg /media/Externa/backups/
fi
rm /tmp/Backup.tar.bz2.gpg
chown patrick:patrick /home/patrick/Desktop/remote-backup.log
The script is run as root (i.e., installed with "sudo crontab -e"). Here's the cron line:

Code: Select all

30 6 * * * /home/patrick/.scripts/backup-remote.sh
I've tried searching the Mint and Ubuntu forums, plus Google, but nothing useful has turned up. I've been tearing my hair out over this for a month now. Sure, I have a lot of hair, but still. Any help is appreciated.

Currently running Linux Mint Julia on an HP Pavilion dv6000, 2GB RAM, 2 GHz Intel Core Duo.
Last edited by LockBot on Wed Dec 28, 2022 7:16 am, edited 1 time in total.
Reason: Topic automatically closed 6 months after creation. New replies are no longer allowed.
User avatar
Pilosopong Tasyo
Level 6
Level 6
Posts: 1432
Joined: Mon Jun 22, 2009 3:26 am
Location: Philippines

Re: when run from cron, script fails with "broken pipe"

Post by Pilosopong Tasyo »

patrick.mooney wrote:The script is run as root...
I can't seem to find the logic why the script has to be run as root. I don't see any sudo commands anywhere in that script. The script runs fine under the user account, so why not just set it under the user's cron table instead of root's? The script will run at the designated time anyway even if the user account is not logged in.
o Give a man a fish and he will eat for a day. Teach him how to fish and he will eat for a lifetime!
o If an issue has been fixed, please edit your first post and add the word [SOLVED].
thaimann

Re: when run from cron, script fails with "broken pipe"

Post by thaimann »

As I said in a previous post, cron provides very few global variables and this can cause your script to fail.

But if you run it like the following, it should get your user globals and should work;

58 * * * * /bin/su -c /home/user/myscript
thaimann

Re: when run from cron, script fails with "broken pipe"

Post by thaimann »

Sorry I said that wrong. It will run with root privileges.
User avatar
xenopeek
Level 25
Level 25
Posts: 29507
Joined: Wed Jul 06, 2011 3:58 am

Re: when run from cron, script fails with "broken pipe"

Post by xenopeek »

Probably this is cron run as root has restrictive $PATH setting. Try setting at the top of your script:

Code: Select all

PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
Image
User avatar
Pilosopong Tasyo
Level 6
Level 6
Posts: 1432
Joined: Mon Jun 22, 2009 3:26 am
Location: Philippines

Re: when run from cron, script fails with "broken pipe"

Post by Pilosopong Tasyo »

thaimann wrote:cron provides very few global variables...
I looked at the code fragment again and I don't see any "$VARIABLEs" being used anywhere. Unless if you're referring to environment variables.

Anyway, I suggest the OP break up this portion of the code:

Code: Select all

tar c -PSvv --to-stdout --exclude=".thumbnails" --exclude-tag-under=".SkipMe" /home/patrick/Documents/school /home/patrick/Documents/writing 2> /home/patrick/Desktop/remote-backup.log | bzip2 -z 2>> /home/patrick/Desktop/remote-backup.log | gpg2 -r 505AB18E --batch -o "/tmp/Backup.tar.bz2.gpg" -e - > /home/patrick/Desktop/remote-backup.log && scp /tmp/Backup.tar.bz2.gpg patrickmooney@ustorage.ucsb.edu:backups 2>> /home/patrick/Desktop/remote-backup.log
into individual commands instead of putting everything as a single line of code, relying solely on piping "|" and redirection ">" as input to the next command.

You'd be amazed how easier it is to debug [programming] problems if you simplify code.
o Give a man a fish and he will eat for a day. Teach him how to fish and he will eat for a lifetime!
o If an issue has been fixed, please edit your first post and add the word [SOLVED].
Habitual

Re: when run from cron, script fails with "broken pipe"

Post by Habitual »

add this to your code and see what happens to the log file?

Code: Select all

# !/bin/bash
set -x
...
patrick.mooney

Re: when run from cron, script fails with "broken pipe"

Post by patrick.mooney »

Thanks for all the replies! I've modified the cron entry to try thaimann's idea (run with su -c) and the script should start again in a few minutes. I'll report back then.
patrick.mooney

Re: when run from cron, script fails with "broken pipe"

Post by patrick.mooney »

Sorry for the delay getting back to you. Internet connectivity in the grad student dorms here seems to be highly intermittent over a holiday break.

None of the suggested fixes actually resolves the problem. Running the script with su -c in the cron job produces the same problem ... same with prefixing the cron command with sudo. Adding set -x to the beginning of the script either produces no change to the log, or adds a few hundred bytes of zeros to the beginning of it.

I could certainly break up the primary command into subcommands, but (a) this would require several hundred MB of extra scratch space, plus requiring that commands be run sequentially instead of concurrently, and (b) what is this, DOS? everyone keeps telling me that one of the advantages of the Linux command line/scripting system is its flexibility. I'd like to take advantage of this flexibility when writing a script. Ideally, I'd just like to get the piping working.

any help is much appreciated.
patrick.mooney

Re: when run from cron, script fails with "broken pipe"

Post by patrick.mooney »

... and I notice now that executing the script from a user crontab, instead of the root crontab, seems to fix the problem. I'm scratching my head over why this is, but it seems to be working. Still, if anyone can explain why it would fail from the root user's account, I'd be interested in hearing. (=
riffraff

Re: when run from cron, script fails with "broken pipe"

Post by riffraff »

Hi there,

you should definetly change your first line to a shebang!

change this

Code: Select all

# !/bin/bash
to that

Code: Select all

#!/bin/bash
This is a big difference - just run

Code: Select all

file <scriptname>
on both variants and you will see that yours is a "ASCII-FILE" and, after correcting it, it will be a "Bourne-Again Shell Script".

Maybe that is your problem here at all ;-)
Locked

Return to “Scripts & Bash”