[Just Sharing] Script For Backing Up My Home Folder

About writing shell scripts and making the most of your shell
Forum rules
Topics in this forum are automatically closed 6 months after creation.
Locked
User avatar
roboticforest
Level 1
Level 1
Posts: 34
Joined: Fri Jun 19, 2020 4:11 pm

[Just Sharing] Script For Backing Up My Home Folder

Post by roboticforest »

Greetings all,

I know there is a backup utility that comes with Mint, but I didn't like it. I don't like making giant snapshot backups of my entire home folder, and prefer to backup different pieces of it at different rates. Some parts of the home folder just don't need to be backup up very often.

:!: Oh! Before you read the code, just keep in mind that this is probably the second or third non-trivial script I've ever written in Bash. I'd appreciate comments or suggestions, but don't rip it apart please. I'm not asking for help with anything and the script is working perfectly for my needs. I just wanted to share it.

Before I made the script I searched the web for backup script template and found nothing, so I made my own. Feel free to use this as a template and modify it for your own needs.

Code: Select all

#!/bin/bash 
set -euo

# IFS is used in variable expansion. Setting it to a semicolon instead of the default of space/tab/newline. This makes sooooo much stuff easier to do.
SAVEIFS=$IFS
IFS=$';'

# Colorize lines of text.
green() { printf '\e[1;32m%b\e[0m\n' "$@"; }
yellow() { printf '\e[1;33m%b\e[0m\n' "$@"; }
red() { printf '\e[1;31m%b\e[0m\n' "$@"; }

# Valid arguments are:
# code, documents, photos, references, music, videos, games, or everything.
# NOTE: Only one argument is processed. Choose one of the above options to create one or all archive types.

echo
green "Running backup script..."
yellow "$(date)"
echo

# General variables.
dest="Archives"
external_media="/media/username/Storage"
day=$(date +%Y-%m-%d)
hostname=$(hostname -s)

# Set aside a folder for holding the archives, if it doesn't already exist.
mkdir -p "$dest"

#############################################################################################################################
# File variables.
# Separate all paths with a semicolon ';' character.
#############################################################################################################################

# Code projects and config files, including IDE settings.
code_files="CLionProjects;\
Code;\
.gitconfig;\
.config/JetBrains"
code_archive="$day $hostname Code Projects.tar.bz2"

# General and misc documents, including downloads, shared files, and file templates.
doc_files="Desktop;\
Documents;\
Downloads;\
Public;\
Templates;\
.bashrc;\
.face;\
.cinnamon/backgrounds/user-folders.lst;\
.config/keepassxc/keepassxc.ini"
doc_archive="$day $hostname Documents and Downloads.tar.bz2"

# Reference documents.
reference_files="Reference"
reference_archive="$day $hostname Reference Documents.tar.bz2"

# Pictures and sorted photos.
pic_files=$(find "Pictures" -maxdepth 1 -not -path "." -not -path "Pictures" -not -path "Pictures/hydrus-image-database" -not -path "Pictures/hydrus-image-database/*" -printf "%p;")
photo_files="Pictures/hydrus-image-database"
pic_archive="$day $hostname Home Folder Pictures.tar.bz2"
photo_archive="$day $hostname Hydrus Photo Database.tar.bz2"

# Music
music_files="Music"
music_archive="$day $hostname Music.tar.bz2"

# Videos and video projects.
video_files="Videos"
video_archive="$day $hostname Videos.tar.bz2"

# Game saves and config files, including controller and/or emulator settings.
game_files=$(find "Gaming" -maxdepth 1 -not -path "." -not -path "Gaming" -not -path "Gaming/minecraft" -printf "%p;")
game_archive="$day $hostname Game Saves.tar.bz2"
minecraft_files="Gaming/minecraft"
minecraft_archive="$day $hostname Minecraft Worlds and Notes.tar.bz2"

#############################################################################################################################
# Create the appropriate archive selected by the user.
#############################################################################################################################

if [ "$1" = "" ] || [ "$1" = "help" ]
then
	echo
	echo "Makes a compressed archive of select items of your home folder."
	echo "Usage: ./backup.sh [option]"
	echo
	echo "Options include: code, documents, photos, references, music, videos, games,"
	echo "or everything."
	echo
	echo "Run this script from your home folder. A directory named \"Archives\" will"
	echo "be created for you containing the archive option you requested. After that"
	echo "you may move the archive to an external device."
	echo
	echo "If you have your primary Storage drive plugged in while this script is running"
	echo "then all backups in the Archive folder will automatically be moved to the"
	echo "external drive."
	echo
fi

# if [ "$1" = "test" ] || [ "$1" = "everything" ]
# then
# 	echo "Argument \$1 read successfully!"
# fi

# Code projects and config files.
if [ "$1" = "code" ] || [ "$1" = "everything" ]
then
	tar cvjf "$dest/$code_archive" $code_files
fi

if [ "$1" = "documents" ] || [ "$1" = "everything" ]
then
	tar cvjf "$dest/$doc_archive" $doc_files
fi

if [ "$1" = "photos" ] || [ "$1" = "everything" ]
then	
	tar cvjf "$dest/$pic_archive" $pic_files
	tar cvjf "$dest/$photo_archive" $photo_files
fi

if [ "$1" = "references" ] || [ "$1" = "everything" ]
then
	tar cvjf "$dest/$reference_archive" $reference_files
fi

if [ "$1" = "music" ] || [ "$1" = "everything" ]
then
	tar cvjf "$dest/$music_archive" $music_files
fi

if [ "$1" = "videos" ] || [ "$1" = "everything" ]
then
	tar cvjf "$dest/$video_archive" $video_files
fi

if [ "$1" = "games" ] || [ "$1" = "everything" ]
then
	tar cvjf "$dest/$game_archive" $game_files
	tar cvjf "$dest/$minecraft_archive" $minecraft_files
fi

#############################################################################################################################
# Optionally, copy the backup files if my storage drive is plugged in.
#############################################################################################################################

if [ -d "$external_media" ]
then
	echo
	yellow "Storage drive detected!"
	green "Moving all backup archives to the external drive."
	echo

	# Set aside a folder for holding the archives, if it doesn't already exist.
	mkdir -p "$external_media/$dest"
	# Move all the local archives to the external drive.
	mv -vf "$dest"/* "$external_media/$dest"
fi

IFS=$SAVEIFS

echo
green "Backup script has finished running."
echo

The script assumes it's being run from the home folder and that the following directories exist:
  • CLionProjects (this is temporary and I'll be removing it soon)
  • Code
  • Desktop
  • Documents
  • Downloads
  • Gaming
  • Music
  • Pictures
  • Public
  • Reference
  • Templates
  • Videos
Most of these directories are standard with the exception of the CLionProjects, Code, Gaming, and Reference directories.

I specifically made this script in order to specially handle certain things during a backup. The two big ones you can see in the code are how the Pictures and Gaming directories are dealt with.

In the Gaming folder I have a minecraft directory. This is actually the hidden .minecraft folder from the my home directory. I take full advantage of how most programs can't tell the difference between a symbolic link and an actual directory so I move all of my game save folders into the Gaming directory, leaving sym-links behind in the game's install folders named identically to the original file. Minecraft worlds are pretty big, change a lot, and I want them in a separate archive. Also, when I get around to reinstalling my family servers the script is already set for me to add those into that particular backup. I may even specifically add a "minecraft" argument to the script.

Photos are handled specially too. I recently started using a program called Hydrus Network to sort and tag family photos. The pictures directory then mostly contains photos that I haven't sorted yet, screen captures, temp image files, desktop backgrounds, and other misc pictures. I used the symbolic link trick again to rip the user database out of the Hydrus install directory and into my Pictures folder. My script then makes a backup of that database separately from all the other random misc pictures that I have.

The last thing this script does is check to see if I have my backup drive plugged in. The fact that Linux treats everything as a file makes that trivial to check.
Last edited by LockBot on Wed Dec 28, 2022 7:16 am, edited 1 time in total.
Reason: Topic automatically closed 6 months after creation. New replies are no longer allowed.
No trees were harmed in the sending of this message, but millions of electrons were severely inconvenienced.
User avatar
fatso
Level 4
Level 4
Posts: 255
Joined: Sat Jul 11, 2020 6:48 pm

Re: [Just Sharing] Script For Backing Up My Home Folder

Post by fatso »

Thanks for sharing that! I'm just a noob tinkerer, myself, so this looks like something I could learn a lot from and play around with. At the very least, I'm confident that I'll find countless ways to break it. :lol:
1000
Level 6
Level 6
Posts: 1039
Joined: Wed Jul 29, 2020 2:14 am

Re: [Just Sharing] Script For Backing Up My Home Folder

Post by 1000 »

1. Try find something like that:
bash example script rsync backup
linux example script rsync backup
2. Why rsync ? https://en.wikipedia.org/wiki/Rsync
By default, rsync determines which files differ between the sending and receiving systems by checking the modification time and size of each file
This allows him to make a copy of the changed files when making a second copy of the files.
Without making a complete copy again.
This greatly reduces the backup time with a very large number of files.

3. I am afraid of using archives
- A long time ago I made an archive. The file was corrupted and I could not recover my data.
Easiest to break on external drives, because some of them have less space than the system shows,
and manufacturer declared.
- Text files are easier to compress than binary files. So it's not always profitable.

Edited
About 3. You can try test archive: (test if " listing of the tarball " with tarball will work )
https://stackoverflow.com/questions/200 ... ompressing

4. The script should have a warning if there is not enough space on the disk partition for a backup.

5. It should create an extra file when script starts and delete when it exits.
If starts and this file exists. This will certify that the last copy of the data has failed.

6. It should also allow you to select copies from a list of copies.
But I don't know how to do it with rsync.

7. You can ( try ) use "logger" to save error to log
example from internet:
https://www.perfacilis.com/blog/systeem ... rsync.html
Edited
https://www.urbanautomaton.com/blog/201 ... to-syslog/

Edit
This is something other but also with tar. There is example with restore
https://wiki.archlinux.org/index.php/Fu ... p_with_tar

Edit
8. You can create function for create and second for restore backup
then you can create menu https://www.thegeekstuff.com/2010/07/ba ... statement/
This will allow you to use the same script for these two things.
./script --backup
./script --restore
./script --logs
Locked

Return to “Scripts & Bash”