Selective deleting

About writing shell scripts and making the most of your shell
Forum rules
Topics in this forum are automatically closed 6 months after creation.
Locked
User avatar
mintymatrix
Level 4
Level 4
Posts: 400
Joined: Sat Dec 21, 2013 9:05 pm

Selective deleting

Post by mintymatrix »

I have a large amount of data (between 1 to 2 Tb) in which I wish to delete most of the media files (like mp4, mpg, mpeg, avi, mp3 etc) from all its folders and subfolders. As the data is very large for my computer to handle at one time, I might have to repeat the process multiple times. So I am thinking of writing a script.

As I am new to writing scripts for Linux and am afraid of deleting important files from that data by mistake, I need help to know how can I write a script that will delete files of particular extension/s and delete empty folders (if any). If it is not too difficult for my ability, I would also like to know how can I make that script create a txt file and store the list of files deleted (including its path).

Code: Select all

#!/bin/bash
echo -e "Deleting mp4 files"
find . -type f -name '*.mp4' -delete
echo -e "Deleting avi files"
find . -type f -name '*.avi' -delete
echo -e "Deleting mpg files"
find . -type f -name '*.mpg' -delete
For writing to a file I think it should be like this before the delete commands : find . -type f -name '*.mp4' > 2delete-mp4.txt
Is this how it should be done ?
Is it a safe way to do it ? Or anyone has any better solution ?
Last edited by LockBot on Wed Dec 28, 2022 7:16 am, edited 1 time in total.
Reason: Topic automatically closed 6 months after creation. New replies are no longer allowed.
User avatar
Pierre
Level 21
Level 21
Posts: 13192
Joined: Fri Sep 05, 2008 5:33 am
Location: Perth, AU.

Re: Selective deleting

Post by Pierre »

whilst that script will work,

it's that I'm more prone to use Manual Handling, than automation:
- click, use Shift & click - to select a whole block of files - - Delete.
then wait & wait, because some files take ages & others won't be deleted coz of some weird error. .. .

& repeat,
:)

you can even sort by File Type, first off, & then Block Select & Block Delete.
8)
Image
Please edit your original post title to include [SOLVED] - when your problem is solved!
and DO LOOK at those Unanswered Topics - - you may be able to answer some!.
lmuserx4849

Re: Selective deleting

Post by lmuserx4849 »

Good start. Your delete will work. Checkout find's fprint.

There is a command line trashcan utility (trash-cli), but I haven't used it.

What about a different tack - move to a NEWDIR, verify, and then delete NEWDIR when you know all the files are not wanted (rm -r NEWDIR).

When testing put echo's in front of commands or check the man page to see if there is something like a --dry-run option.
When done, you may also want to delete the current directories, if they are empty (find . -type d -empty)

Regarding the example below, $1 is input argument 1. The script is expecting the file extension (ie., jpg), although no checking is done. You will get an error if not supplied (:? - See bash man page Parameter Expansion). Just run the script for each file type. ./SCRIPT mp4 ... ./SCRIPT avi, etc. Check directory locations. I made assumptions.

Code: Select all

#!/bin/bash
#set -u
#set -x
scriptname="${0##*/}"
ft="${1:?Error: Missing file type}"  # Input argument - file type extension
dir='.'                                             # media directory 
ftdir="${dir}/delete-${ft}/"               # file type NEWDIR
ftlist="${dir}/list-${ft}"                     # file type list
ftlistmv="${dir}/list-${ft}-mv"          # file type list for mv
ftnum=1
ftdupnum=1

if ! mkdir "${ftdir}"; then
  printf -- '%s: Error: mkdir failed - %s\n' "${scriptname}" "${ftdir}"
  exit 1
fi

echo "Processing ${ft} files"

### Option 1: If you know for sure there are no duplicate file names, or there are no subdirectories, this would work.
#
find "${dir}" \( -path "${ftdir}" \) -prune -o -type f -name "*.${ft}" -fprint "${ftlist}" -exec mv --target-directory="${ftdir}" '{}' \+

exit 0

### Option 2: Comment out the find and exit above if you think there are duplicate file names.
# Prevent overwrite of duplicate file names in subdirectories. Just tack on an integer.
# Files are separated with a null. 
# ${file##*/}" - See bash parameter expansion. This removes everything on the left until the last slash, basically the basename.
#
while IFS='' read -r -d $'\0' file; do
  if [[ -e "${ftdir}${file##*/}" ]]; then
    mv -v "${file}" "${ftdir}${file##*/}-$((ftdupnum++))" >> "${ftlist}-mv"   # Duplicate file name
  else
    mv -v --target-directory="${ftdir}" "${file}" >> "${ftlistmv}"            # No duplicate
  fi
  ((ftnum++))
done < <(find "${dir}" \( -path "${ftdir}" \) -prune -o -type f -name "*.${ft}" -fprint "${ftlist}" -print0)

echo "Processed $((ftnum-1)) files"
echo "Duplicate file names $((ftdupnum-1))"
Some folks use find and pipe it to xargs, some use rsync. Bash has a shopt option called globstar that could also be used to create a pattern and transverse directories with a for loop.
User avatar
mintymatrix
Level 4
Level 4
Posts: 400
Joined: Sat Dec 21, 2013 9:05 pm

Re: Selective deleting

Post by mintymatrix »

That's a good idea.
But before I checked yours, I tried the script I wrote and found that it was working 'case sensitive'.
It deleted .mp4 files but not .MP4 ones. Is there any switch to toggle that ?
lmuserx4849

Re: Selective deleting

Post by lmuserx4849 »

mintymatrix wrote:That's a good idea.
But before I checked yours, I tried the script I wrote and found that it was working 'case sensitive'.
It deleted .mp4 files but not .MP4 ones. Is there any switch to toggle that ?
Use -iname instead of -name

man find
User avatar
mintymatrix
Level 4
Level 4
Posts: 400
Joined: Sat Dec 21, 2013 9:05 pm

Re: Selective deleting

Post by mintymatrix »

lmuserx4849 wrote:
mintymatrix wrote:That's a good idea.
But before I checked yours, I tried the script I wrote and found that it was working 'case sensitive'.
It deleted .mp4 files but not .MP4 ones. Is there any switch to toggle that ?
Use -iname instead of -name

man find
Thanks, I will check that option.
Locked

Return to “Scripts & Bash”