A question on scripts and cron

About writing shell scripts and making the most of your shell
Forum rules
Topics in this forum are automatically closed 6 months after creation.
Locked
lisapisa2
Level 3
Level 3
Posts: 124
Joined: Thu Jan 30, 2014 11:36 am

A question on scripts and cron

Post by lisapisa2 »

I have working two scripts, one that does a backup and another script that does a rootkit search. Both work fine. The next piece was to put them into a cron daily job. How can I see if the cron job has run? I've tried to poke around and read man but what I dont know is for a daily job must it execute at the time specified in cron or is it a run once even if it is past the time specified. What determines when a cron daily job will run?

Code: Select all

/usr/local/bin/rkhunter -c --cronjob 2>&1 
Last edited by LockBot on Wed Dec 28, 2022 7:16 am, edited 1 time in total.
Reason: Topic automatically closed 6 months after creation. New replies are no longer allowed.
User avatar
Pilosopong Tasyo
Level 6
Level 6
Posts: 1432
Joined: Mon Jun 22, 2009 3:26 am
Location: Philippines

Re: A question on scripts and cron

Post by Pilosopong Tasyo »

I moved your thread in the Scripts & Bash section. You might get more replies here.
o Give a man a fish and he will eat for a day. Teach him how to fish and he will eat for a lifetime!
o If an issue has been fixed, please edit your first post and add the word [SOLVED].
User avatar
xenopeek
Level 25
Level 25
Posts: 29459
Joined: Wed Jul 06, 2011 3:58 am

Re: A question on scripts and cron

Post by xenopeek »

lisapisa2 wrote:2>&1
Where does the output go? There's no terminal associated with cron, so it can't log there, and you've not redirected output to a file... Best way is to create a bash script to launch your job(s) and put that in /etc/cron.daily. Then have that script start with redirecting all output to a logfile, so you can see what's going on. More info + example here: http://forums.linuxmint.com/viewtopic.p ... =0#p481766

You can add echo statements to your bash script, for example to log time & date it was started.
Image
lisapisa2
Level 3
Level 3
Posts: 124
Joined: Thu Jan 30, 2014 11:36 am

Re: A question on scripts and cron

Post by lisapisa2 »

Okay I got it. Maybe I am closer. So if I put the following script in the cron.daily directory I am assuming it will generate a file dailybackup.log and put the backup information into it

Code: Select all

#!/bin/bash
#
exec >>/var/log/dailybackup.log 2>&1
#backup files (do I need sudo?)
/usr/bin/rsync -avz  --exclude={/proc,/dev,/media,/mnt,/sys,/tmp,/home/xxxxx/images,/home/xxxxx/ctest,/home/xxxxx/Downloads,/home/xxxxx/"VirtualBox VMs"} // /media/xxxxx/Backup 
exit
What I dont know is how to check cron to see if the scccript ran and what the results were. I dont want to use a MTA. I would prefer to have a logfile to check on cron results.
User avatar
xenopeek
Level 25
Level 25
Posts: 29459
Joined: Wed Jul 06, 2011 3:58 am

Re: A question on scripts and cron

Post by xenopeek »

cron logs to /var/log/syslog. For example, I see an entry "Job `cron.daily' started" appear there. You don't need to use sudo in cron scripts, as scripts in cron.daily will be run as root.

I'd perhaps expand your script to include a timestamp and message to the logfile announcing starting the daily backup:

Code: Select all

#!/bin/bash
#
exec >>/var/log/dailybackup.log 2>&1

# announce starting
echo "$(date +'%b %d %X') $(uname -n) $(basename $0): starting daily backup"

#backup files (do I need sudo?)
/usr/bin/rsync -avz  --exclude={/proc,/dev,/media,/mnt,/sys,/tmp,/home/xxxxx/images,/home/xxxxx/ctest,/home/xxxxx/Downloads,/home/xxxxx/"VirtualBox VMs"} // /media/xxxxx/Backup
You don't need the exit statement either, unless you need to explicitly exit the script earlier or need to return a specific error code.
Image
lisapisa2
Level 3
Level 3
Posts: 124
Joined: Thu Jan 30, 2014 11:36 am

Re: A question on scripts and cron

Post by lisapisa2 »

Thank you xenopeek. I am learnign lots about scripting. I know a lot about network analysis, security and know 3G/4G inside out so feel free to PM me if you have nay questions.

I have not put it in cron yet as I want the script rock solid before. This is so strange because I excluded the directory

/home/xxx/"VirtualBox VMs"

and in the log I see

home/xxx/VirtualBox VMs/
home/xxx/VirtualBox VMs/MintKDE32/
home/xxx/VirtualBox VMs/MintKDE32/MintKDE32.vbox
home/xxx/VirtualBox VMs/MintKDE32/MintKDE32.vbox-prev
home/xxx/VirtualBox VMs/MintKDE32/MintKDE32.vhd
home/xxx/VirtualBox VMs/MintKDE32/Logs/
home/xxx/VirtualBox VMs/MintKDE32/Logs/VBox.log
home/xxx/VirtualBox VMs/MintKDE32/Logs/VBox.log.1
home/xxx/VirtualBox VMs/MintKDE32/Logs/VBox.log.2
home/xxx/VirtualBox VMs/MintKDE32/Logs/VBox.log.3
home/xxx/VirtualBox VMs/MintKDE32/Logs/VBox.png.2
There is no way that it should be copying those directories unless I missed something?

Its not fatal - jsut kind of curious as to why it would behave like this and I have checked and doublechecked spelling and other common areas. For that reason I use cut and past rather than typing.

Now the final question. If I just put the script in /etc/cron.daily it will run. Is this correct?
lisapisa2
Level 3
Level 3
Posts: 124
Joined: Thu Jan 30, 2014 11:36 am

Re: A question on scripts and cron

Post by lisapisa2 »

Wow - it just got really ugly. I ran the backup script luckily not as a cron job and it did not ignore the directories I had excluded. The source is a SSD drive and the target is a mechanical HD. When I started the backup my SSD was using 60 out of a possible 110 GB of space and my HD was 500 GB all empty. The rsync command died on the large files but when it did it consumed all the storage space on the SSD. The machine has 16 GB RAM and the file it choked on was 4 GB in size. The net result was that I saw the no disk space result and then the OS blew up. I restarted and of course could not get in because the SSD was full. I had to boot off USB and manually delete files. This is really concernign because I am trying to get a reliable backup to avoid exactly these types of problems and there is something wrong with rsync, large files and SSD drives. I have plenty of RAM and the transfer is all in HW SSD->MHD (where MHD is mechanical hard drive). I have tried some backup utilities but things like the backup tool miscalculate the size of the SSD and then say there is not enough space on the target. What is really strange is that if I run the bare rsync command in my script as a standalone command in a bash shell everything executes OK. For some reason when if is in the script it blows up.
Habitual

Re: A question on scripts and cron

Post by Habitual »

lisapisa:

You can have more than one --exclude statement in your command/script.

Code: Select all

--exclude={/proc,/dev,/media,/mnt,/sys,/tmp,/home/xxxxx/images,/home/xxxxx/ctest,/home/xxxxx/Downloads}  --exclude=/home/xxxxx/"VirtualBox VMs"
Also you utilize a file exclusion "list" that may provide the result (omitting /home/xxxxx/"VirtualBox VMs") you are seeking.

-exclude-from=/path/to/exclusion.list

AND don't forget about dry running the command before "take off" using

Code: Select all

/usr/bin/rsync -avzn
or

Code: Select all

/usr/bin/rsync -avz --dry-run
so.......

Code: Select all

/usr/bin/rsync -avz  --exclude={/proc,/dev,/media,/mnt,/sys,/tmp,/home/xxxxx/images,/home/xxxxx/ctest,/home/xxxxx/Downloads,} --exclude=/home/xxxxx/"VirtualBox VMs" // /media/xxxxx/Backup
could "solve" the included "VirtualBox VMs" directory in your command.
This is exactly why I renamed mine to VboxVMs/ ;)

Is that double forwards slash intentional?

Hope that's helpful. ;)
lisapisa2
Level 3
Level 3
Posts: 124
Joined: Thu Jan 30, 2014 11:36 am

Re: A question on scripts and cron

Post by lisapisa2 »

I am convinced there is a rsync bug. The problem is that to duplicate it takes down the file system. There is something funny about having a SSD drive as the source, a MHD as the target and large files. I even upped my RAM to 16 GB which did make it somewhat better but the problem still rears its head and when it does it compltely kills the system. I never could find any problem with the SSD drive and had to resort to a new destructive install. I'm still looking at a better backup solution. The backup tool doesn't work and rsync blows up. WHat I dont understand about rsync is that if you run it from the command line it only copies files that have changed. When you put it in a script it copies everything and ignores the exclude parameter. That is what led to the crash and the logs show it. If I knew it would just copy files that have changed and not copy the excluded directories I could use it. That raises a question. In the code I was supplied there was this line

Code: Select all

exec >>/var/log/dailybackup.log 2>&
I dont know if that played a role or if is the script in general.
Habitual

Re: A question on scripts and cron

Post by Habitual »

I backup my stuff all the time with exclude included in my script(s).
Works every time.

What are you doing with

Code: Select all

exec >>/var/log/dailybackup.log 2>&1
?
User avatar
xenopeek
Level 25
Level 25
Posts: 29459
Joined: Wed Jul 06, 2011 3:58 am

Re: A question on scripts and cron

Post by xenopeek »

Habitual wrote:What are you doing with

Code: Select all

exec >>/var/log/dailybackup.log 2>&1
?
That was my recommendation to add to the start of any scripts run from cron. It redirects stdout and stderr to a log file, so any output from the script or the commands it runs (or error messages from the shell) are written to a log file. I use this on all my Bash scripts run from cron, and it has helped me pin down some programming mistakes.
Image
Habitual

Re: A question on scripts and cron

Post by Habitual »

xenopeek wrote:
Habitual wrote:What are you doing with

Code: Select all

exec >>/var/log/dailybackup.log 2>&1
?
That was my recommendation to add to the start of any scripts run from cron.
That's all I need to know about 'that'. :)
User avatar
xenopeek
Level 25
Level 25
Posts: 29459
Joined: Wed Jul 06, 2011 3:58 am

Re: A question on scripts and cron

Post by xenopeek »

I know, but for any other readers :wink:
Image
Locked

Return to “Scripts & Bash”