Batch script to delete then move all files

Forum rules
Before you post please read how to get help
Post Reply
badams_ios
Level 2
Level 2
Posts: 71
Joined: Tue Jun 18, 2013 7:55 pm

Batch script to delete then move all files

Post by badams_ios »

I know absolutely NOTHING about scripting. I'm running Linux Mint 17.3

I need a bash script that will run as a cron job.

Let's say I have 2 folders we'll call A and a sub-folder, B. On a day specified by cron, I want to delete all files from B. No errors if none exist, just keep going to step 2. Step 2 should MOVE all files from A to B. No errors if nothing exists in A.

Here's my scenario. On the 1st Monday of each month at 6:00 am, a full backup is made to a shared folder. On the 2nd, 3rd and 4th week, it'll make a differential backup to the same location. Those I already have working. THANKS to the support team at terabyteunlimited. What I need now is to delete all the files in B and moves all of A to B. (this'll run at 2 am because the move will take a few hours)

I'd like to ADD to the scripts that make the backups, a warning that the system is going to reboot and give me 5 minutes to save any work. Just a simple eye catching box in the middle of the screen that says "GOING DOWN!" It doesn't need to say "In 5 minutes" and it doesn't need to give a count down. (Unless you feel like it :lol: It'd be nice if I could minimize that message, but no big deal if I can't.
Clewless

Lenovo ThinkServer
Intel(R) Core(TM) i3-4130 CPU @ 3.40GHz
20 GB RAM
3 TB WD Black, hardware raid 5 - /home
Nvidia GeForce GT 220
32" TV
In the Man Cave :)


Mint Mate 17.1

Media Center, image and video editing

User avatar
Flemur
Level 18
Level 18
Posts: 8024
Joined: Mon Aug 20, 2012 9:41 pm
Location: Potemkin Village

Re: Batch script to delete then move all files

Post by Flemur »

What I need now is to delete all the files in B and moves all of A to B.
Assuming the files aren't directories and their names don't start with ".":

Code: Select all

rm /path/to/A/B/*
mv /path/to/A/* /path/to/A/B/.
You'll get an error about "cannot move 'A/B' to a subdirectory of itself, ..." but it works.
(this'll run at 2 am because the move will take a few hours)
If B is a subdirectory of A, as you specified, the move will probably be almost instantaneous because the files aren't actually moved on the disk, it just changes which directory contains them.
Please edit your original post title to include [SOLVED] if/when it is solved!
Your data and OS are backed up....right?

badams_ios
Level 2
Level 2
Posts: 71
Joined: Tue Jun 18, 2013 7:55 pm

Re: Batch script to delete then move all files

Post by badams_ios »

I'll give that a try. The files are actually on a shared drive, so I expect it to take forever. And a day.
Clewless

Lenovo ThinkServer
Intel(R) Core(TM) i3-4130 CPU @ 3.40GHz
20 GB RAM
3 TB WD Black, hardware raid 5 - /home
Nvidia GeForce GT 220
32" TV
In the Man Cave :)


Mint Mate 17.1

Media Center, image and video editing

lmuserx4849

Re: Batch script to delete then move all files

Post by lmuserx4849 »

badams_ios wrote:I know absolutely NOTHING about scripting.
imho, this is the best resource for learning bash:
http://linuxcommand.org/lc3_writing_shell_scripts.php
He covers topics many do not. Uses best practices, and he's even got a script that you can run to create a template.
Great way to be consistent.

After that, check out greg's bash FAQ:
http://mywiki.wooledge.org/BashFAQ

Most commands have a man page (i.e., man bash). Bash also has a shell builtin called help (i.e., help if). A number of commands
have info pages (i.e., info bash). Most commands have a help option (i.e., date --help). If you want to know where
something is in your path (i.e., type -a date). If you want to know the file type (i.e., file /bin/date)

Here's gnu's core utility manual in html format:
http://www.gnu.org/software/coreutils/m ... utils.html

In the future you might want to look at rsync, rsnapshot, etc. for your backup/maintenance procedure.

Here's an educational snippet to experiment with and test, test, test:

Code: Select all

#!/bin/bash
set -u

declare -r  dirA="$HOME/A"
declare -r  dirB="$HOME/B"
declare -r  windowManager='kwin'        # if you don't know run: wmctrl -m
declare -r  scriptName="${0##*/}"
declare -r  sleepTime='5m'
declare -ra cmds=(pgrep grep cut notify-send sleep rm mv mkdir)
declare --  msg='' timestamp='' pid='' dbus='' cmd=''

function finish() 
{
  local -- msg="${1:?Missing message text}"
  local -- rc="${2:?return code}"

  printf -- '%s %s: %s\n' "[$('date' +'%F %H:%M %Z')]" "${scriptName}" "${msg}"
  exit "${rc}"
}

# make sure all necessary commands are available
for cmd in "${cmds[@]}"; do
  type -a "${cmd}" &>/dev/null || finish "Command ${cmd} not installed" 1
done

# validate existence of directories
if [[ ! -d "${dirA}" ]]; then
  finish "Directory does not exist: ${dirA}" 1
elif [[ ! -d "${dirB}" ]]; then
  finish "Directory does not exist: ${dirB}" 1
fi

# currently required by notify-send w/in crontab. 
# I don't know if there is a better way. Some people save $DBUS_SESSION_BUS_ADDRESS from their
# interactive session at logon in a known file and then just source that file.
pid="$(pgrep -u${LOGNAME} ${windowManager})"
dbus="$(grep -z DBUS_SESSION_BUS_ADDRESS /proc/${pid}/environ|cut -d= -f2-)"
export DBUS_SESSION_BUS_ADDRESS="${dbus}"

# display warning message 
timestamp="[$('date' +'%F %H:%M %Z')]"
msg="${scriptName}: System is going to reboot in ${sleepTime}."
printf -- '%s %s\n' "${timestamp}" "${msg}"
msg+="\nSave your work."
notify-send --icon=dialog-information "${timestamp}" "${msg}"
# check out: logger, wall, dialog, kde only: kdialog)
# kde example: DISPLAY=:0 kdialog --caption "${timestamp}" --msgbox "${scriptName}\n${msg}"

# wait
sleep "${sleepTime}" 

# maintenance details
rm -r "${dirB}"        || finish "rm ${dirB} failed" 1
mv "${dirA}" "${dirB}" || finish "mv ${dirA} ${dirB} failed" 1
mkdir "${dirA}"        || finish "mkdir ${dirA} failed" 1

finish "Done" 0

badams_ios
Level 2
Level 2
Posts: 71
Joined: Tue Jun 18, 2013 7:55 pm

[SOLVED] Re: Batch script to delete then move all files

Post by badams_ios »

lmuserx4849

Thanks. I'll play with that after I setup a virtual machine before I delve that far. I tried to learn gwbasic back in the day. I was pretty happy when others started writing programs. Then came that BBS scene. I ran a Fidonet ASP BBS from '88 to '94. when the Internet ruined all my fun. But google made up for that.

For now I simply run:
mv /media/syncovery/Backups/*.tbi /media/syncovery/Backups/Rollover
then
rm /media/syncovery/Backups/*.tbi from a cron job.

That share is setup in fstab. Since there's no way to lock files in Linux, I have to take / offline to run terabyte's imaging software.

Will rsync create some kind of backup or image that I can use to restore the system completely should a hard drive crash? With Terabyte I can do a restore of the entire drive in about 15 minutes. REAL simple! I can do a backup or restore with terabyte from a cd very simply and fast, but who wants to do it manually when it can be automated?

/media/syncovery is on a Qnap NAS with raid 5 with a hot failover.
Clewless

Lenovo ThinkServer
Intel(R) Core(TM) i3-4130 CPU @ 3.40GHz
20 GB RAM
3 TB WD Black, hardware raid 5 - /home
Nvidia GeForce GT 220
32" TV
In the Man Cave :)


Mint Mate 17.1

Media Center, image and video editing

lmuserx4849

Re: Batch script to delete then move all files

Post by lmuserx4849 »

I remember CompuServe :-)

Bash scripts are more like DOS Batch files. QWBasic is probably more like Python or Perl.

Entire system and incremental. Yes. Netgear ReadyNAS uses rsync.

I'm not familar with Terabyte. rsync is a command and you can write your own scripts. rsnapshot is written in perl and based on rsync. There are other more sophisticated/gui options in the linux world like amanda, bacula...

User avatar
LinuxJim
Level 5
Level 5
Posts: 659
Joined: Tue Jan 26, 2016 8:01 pm
Location: Oregon, USA

Re: [SOLVED] Re: Batch script to delete then move all files

Post by LinuxJim »

badams_ios wrote:lThat share is setup in fstab. Since there's no way to lock files in Linux, I have to take / offline to run terabyte's imaging software.
Who told you that? Linux has both mandatory and cooperative file locking.

Filesystems are mounted by default without the mandatory flag, to avoid race conditions - but if you have a need for it, you can certainly turn it on:

Code: Select all

man mount 8

badams_ios
Level 2
Level 2
Posts: 71
Joined: Tue Jun 18, 2013 7:55 pm

Re: Batch script to delete then move all files

Post by badams_ios »

I was told by Terabyte Unlimited that the linux drive has no way of using VSS or Phylock. That files might change during a backup.

Regardless, I own Terabytes Imaging software. Have since 2008 but always used it for Windows which is extremely simple. I have been using it for Linux, you boot from a CD, very easy to map a drive and make a backup, but I'm tired of doing it manually. What I love about Terabyte's system. A drive crashes, I boot from a CD, connect to the network share, tell it to restore to a brand new drive and in 15 minutes I'm back in business.

At this point, I can make a full backup automatically, but I'm having a hard time with their differential backup. In 8 years I've never needed their support. Now I do and it SUCKS!!! Their prompt, I usually get a reply within minutes, but it's cryptic B.S.!

I ask for help and I get "I would assume it's having an issue with the --d:l1 parameter. It's not allowed for differential backups (the base image tells it what to back up)."
Gee, that's a big help. Last night I sent them a picture of my granddaughter and a model ship I'm working on and asked what they'd rather spend their time doing. Deciphering cryptic answers, or spending time with family and my hobby.

This morning I found a message telling me how to do it.

So, that's all working, now, like I say, I want to move the backups from last month to a sub-folder before making a new set.

Thanks to all the great (real) support from y'all, I'm well on my way to that as well.

THANKS, ALL. Y'all rock!
Clewless

Lenovo ThinkServer
Intel(R) Core(TM) i3-4130 CPU @ 3.40GHz
20 GB RAM
3 TB WD Black, hardware raid 5 - /home
Nvidia GeForce GT 220
32" TV
In the Man Cave :)


Mint Mate 17.1

Media Center, image and video editing

Post Reply

Return to “Scripts & Bash”