A question on scripts and cron

Forum rules
Before you post please read how to get help

A question on scripts and cron

Postby lisapisa2 on Sun Feb 09, 2014 7:11 pm

I have working two scripts, one that does a backup and another script that does a rootkit search. Both work fine. The next piece was to put them into a cron daily job. How can I see if the cron job has run? I've tried to poke around and read man but what I dont know is for a daily job must it execute at the time specified in cron or is it a run once even if it is past the time specified. What determines when a cron daily job will run?

Code: Select all
/usr/local/bin/rkhunter -c --cronjob 2>&1
lisapisa2
Level 1
Level 1
 
Posts: 42
Joined: Thu Jan 30, 2014 11:36 am

Linux Mint is funded by ads and donations.
 

Re: A question on scripts and cron

Postby Pilosopong Tasyo on Mon Feb 10, 2014 2:20 am

I moved your thread in the Scripts & Bash section. You might get more replies here.
Image
o Give a man a fish and he will eat for a day. Teach him how to fish and he will eat for a lifetime!
o If an issue has been fixed, please edit your first post and add the word [SOLVED].
User avatar
Pilosopong Tasyo
Level 6
Level 6
 
Posts: 1188
Joined: Mon Jun 22, 2009 3:26 am
Location: Philippines

Re: A question on scripts and cron

Postby xenopeek on Mon Feb 10, 2014 7:57 am

lisapisa2 wrote:2>&1

Where does the output go? There's no terminal associated with cron, so it can't log there, and you've not redirected output to a file... Best way is to create a bash script to launch your job(s) and put that in /etc/cron.daily. Then have that script start with redirecting all output to a logfile, so you can see what's going on. More info + example here: viewtopic.php?f=90&t=83116&start=0#p481766

You can add echo statements to your bash script, for example to log time & date it was started.
User avatar
xenopeek
Level 21
Level 21
 
Posts: 15256
Joined: Wed Jul 06, 2011 3:58 am
Location: The Netherlands

Re: A question on scripts and cron

Postby lisapisa2 on Mon Feb 10, 2014 6:41 pm

Okay I got it. Maybe I am closer. So if I put the following script in the cron.daily directory I am assuming it will generate a file dailybackup.log and put the backup information into it

Code: Select all
#!/bin/bash
#
exec >>/var/log/dailybackup.log 2>&1
#backup files (do I need sudo?)
/usr/bin/rsync -avz  --exclude={/proc,/dev,/media,/mnt,/sys,/tmp,/home/xxxxx/images,/home/xxxxx/ctest,/home/xxxxx/Downloads,/home/xxxxx/"VirtualBox VMs"} // /media/xxxxx/Backup
exit


What I dont know is how to check cron to see if the scccript ran and what the results were. I dont want to use a MTA. I would prefer to have a logfile to check on cron results.
lisapisa2
Level 1
Level 1
 
Posts: 42
Joined: Thu Jan 30, 2014 11:36 am

Re: A question on scripts and cron

Postby xenopeek on Tue Feb 11, 2014 2:56 am

cron logs to /var/log/syslog. For example, I see an entry "Job `cron.daily' started" appear there. You don't need to use sudo in cron scripts, as scripts in cron.daily will be run as root.

I'd perhaps expand your script to include a timestamp and message to the logfile announcing starting the daily backup:
Code: Select all
#!/bin/bash
#
exec >>/var/log/dailybackup.log 2>&1

# announce starting
echo "$(date +'%b %d %X') $(uname -n) $(basename $0): starting daily backup"

#backup files (do I need sudo?)
/usr/bin/rsync -avz  --exclude={/proc,/dev,/media,/mnt,/sys,/tmp,/home/xxxxx/images,/home/xxxxx/ctest,/home/xxxxx/Downloads,/home/xxxxx/"VirtualBox VMs"} // /media/xxxxx/Backup

You don't need the exit statement either, unless you need to explicitly exit the script earlier or need to return a specific error code.
User avatar
xenopeek
Level 21
Level 21
 
Posts: 15256
Joined: Wed Jul 06, 2011 3:58 am
Location: The Netherlands

Re: A question on scripts and cron

Postby lisapisa2 on Tue Feb 11, 2014 4:32 pm

Thank you xenopeek. I am learnign lots about scripting. I know a lot about network analysis, security and know 3G/4G inside out so feel free to PM me if you have nay questions.

I have not put it in cron yet as I want the script rock solid before. This is so strange because I excluded the directory

/home/xxx/"VirtualBox VMs"

and in the log I see

home/xxx/VirtualBox VMs/
home/xxx/VirtualBox VMs/MintKDE32/
home/xxx/VirtualBox VMs/MintKDE32/MintKDE32.vbox
home/xxx/VirtualBox VMs/MintKDE32/MintKDE32.vbox-prev
home/xxx/VirtualBox VMs/MintKDE32/MintKDE32.vhd
home/xxx/VirtualBox VMs/MintKDE32/Logs/
home/xxx/VirtualBox VMs/MintKDE32/Logs/VBox.log
home/xxx/VirtualBox VMs/MintKDE32/Logs/VBox.log.1
home/xxx/VirtualBox VMs/MintKDE32/Logs/VBox.log.2
home/xxx/VirtualBox VMs/MintKDE32/Logs/VBox.log.3
home/xxx/VirtualBox VMs/MintKDE32/Logs/VBox.png.2
There is no way that it should be copying those directories unless I missed something?

Its not fatal - jsut kind of curious as to why it would behave like this and I have checked and doublechecked spelling and other common areas. For that reason I use cut and past rather than typing.

Now the final question. If I just put the script in /etc/cron.daily it will run. Is this correct?
lisapisa2
Level 1
Level 1
 
Posts: 42
Joined: Thu Jan 30, 2014 11:36 am

Re: A question on scripts and cron

Postby lisapisa2 on Tue Feb 11, 2014 6:06 pm

Wow - it just got really ugly. I ran the backup script luckily not as a cron job and it did not ignore the directories I had excluded. The source is a SSD drive and the target is a mechanical HD. When I started the backup my SSD was using 60 out of a possible 110 GB of space and my HD was 500 GB all empty. The rsync command died on the large files but when it did it consumed all the storage space on the SSD. The machine has 16 GB RAM and the file it choked on was 4 GB in size. The net result was that I saw the no disk space result and then the OS blew up. I restarted and of course could not get in because the SSD was full. I had to boot off USB and manually delete files. This is really concernign because I am trying to get a reliable backup to avoid exactly these types of problems and there is something wrong with rsync, large files and SSD drives. I have plenty of RAM and the transfer is all in HW SSD->MHD (where MHD is mechanical hard drive). I have tried some backup utilities but things like the backup tool miscalculate the size of the SSD and then say there is not enough space on the target. What is really strange is that if I run the bare rsync command in my script as a standalone command in a bash shell everything executes OK. For some reason when if is in the script it blows up.
lisapisa2
Level 1
Level 1
 
Posts: 42
Joined: Thu Jan 30, 2014 11:36 am

Re: A question on scripts and cron

Postby Habitual on Tue Feb 11, 2014 6:35 pm

lisapisa:

You can have more than one --exclude statement in your command/script.
Code: Select all
--exclude={/proc,/dev,/media,/mnt,/sys,/tmp,/home/xxxxx/images,/home/xxxxx/ctest,/home/xxxxx/Downloads}  --exclude=/home/xxxxx/"VirtualBox VMs"

Also you utilize a file exclusion "list" that may provide the result (omitting /home/xxxxx/"VirtualBox VMs") you are seeking.

-exclude-from=/path/to/exclusion.list

AND don't forget about dry running the command before "take off" using
Code: Select all
/usr/bin/rsync -avzn
or
Code: Select all
/usr/bin/rsync -avz --dry-run


so.......
Code: Select all
/usr/bin/rsync -avz  --exclude={/proc,/dev,/media,/mnt,/sys,/tmp,/home/xxxxx/images,/home/xxxxx/ctest,/home/xxxxx/Downloads,} --exclude=/home/xxxxx/"VirtualBox VMs" // /media/xxxxx/Backup
could "solve" the included "VirtualBox VMs" directory in your command.
This is exactly why I renamed mine to VboxVMs/ ;)

Is that double forwards slash intentional?

Hope that's helpful. ;)
What's a landing but a take off in reverse?
User avatar
Habitual
Level 8
Level 8
 
Posts: 2246
Joined: Sun Nov 21, 2010 8:31 pm
Location: LM17Q-Xfce

Re: A question on scripts and cron

Postby lisapisa2 on Wed Feb 12, 2014 11:24 pm

I am convinced there is a rsync bug. The problem is that to duplicate it takes down the file system. There is something funny about having a SSD drive as the source, a MHD as the target and large files. I even upped my RAM to 16 GB which did make it somewhat better but the problem still rears its head and when it does it compltely kills the system. I never could find any problem with the SSD drive and had to resort to a new destructive install. I'm still looking at a better backup solution. The backup tool doesn't work and rsync blows up. WHat I dont understand about rsync is that if you run it from the command line it only copies files that have changed. When you put it in a script it copies everything and ignores the exclude parameter. That is what led to the crash and the logs show it. If I knew it would just copy files that have changed and not copy the excluded directories I could use it. That raises a question. In the code I was supplied there was this line

Code: Select all
exec >>/var/log/dailybackup.log 2>&


I dont know if that played a role or if is the script in general.
lisapisa2
Level 1
Level 1
 
Posts: 42
Joined: Thu Jan 30, 2014 11:36 am

Re: A question on scripts and cron

Postby Habitual on Tue Feb 18, 2014 4:44 pm

I backup my stuff all the time with exclude included in my script(s).
Works every time.

What are you doing with
Code: Select all
exec >>/var/log/dailybackup.log 2>&1
?
What's a landing but a take off in reverse?
User avatar
Habitual
Level 8
Level 8
 
Posts: 2246
Joined: Sun Nov 21, 2010 8:31 pm
Location: LM17Q-Xfce

Re: A question on scripts and cron

Postby xenopeek on Wed Feb 19, 2014 2:37 am

Habitual wrote:What are you doing with
Code: Select all
exec >>/var/log/dailybackup.log 2>&1
?

That was my recommendation to add to the start of any scripts run from cron. It redirects stdout and stderr to a log file, so any output from the script or the commands it runs (or error messages from the shell) are written to a log file. I use this on all my Bash scripts run from cron, and it has helped me pin down some programming mistakes.
User avatar
xenopeek
Level 21
Level 21
 
Posts: 15256
Joined: Wed Jul 06, 2011 3:58 am
Location: The Netherlands

Re: A question on scripts and cron

Postby Habitual on Fri Feb 21, 2014 11:26 am

xenopeek wrote:
Habitual wrote:What are you doing with
Code: Select all
exec >>/var/log/dailybackup.log 2>&1
?

That was my recommendation to add to the start of any scripts run from cron.
That's all I need to know about 'that'. :)
What's a landing but a take off in reverse?
User avatar
Habitual
Level 8
Level 8
 
Posts: 2246
Joined: Sun Nov 21, 2010 8:31 pm
Location: LM17Q-Xfce

Re: A question on scripts and cron

Postby xenopeek on Fri Feb 21, 2014 11:33 am

I know, but for any other readers :wink:
User avatar
xenopeek
Level 21
Level 21
 
Posts: 15256
Joined: Wed Jul 06, 2011 3:58 am
Location: The Netherlands

Linux Mint is funded by ads and donations.
 

Return to Scripts & Bash

Who is online

Users browsing this forum: No registered users and 2 guests