[SOLVED] Trying to determine my 'Backup' method

[dalexunil]

Ok, full disclosure here:

I've got another thread that is currently open (unsolved?) just a couple posts down but want to start this one to actually find out other opinions regarding HOW other Linux users backup their data to prevent potential serious loss somewhere into the future.

I really need to get this issue settled so I can move on with life!

TYIA for your replies and input...

[WharfRat]

dalexunil,

I use rsync to backup to a locally attached external HDD and to a remote computer that holds backups.

Generally I don't bother with system files as Mint is easy to reinstall. However when I do backup the system I use dd to write the partition to a file.

I find using rsync to be the most dependable method of backing up data files etc.

[dalexunil]

dalexunil,

I use rsync to backup to a locally attached external HDD and to a remote computer that holds backups.

Generally I don't bother with system files as Mint is easy to reinstall. However when I do backup the system I use dd to write the partition to a file.

I find using rsync to be the most dependable method of backing up data files etc.

thanks for reply...

I'm not familiar with "dd".

How does one do backups using "rsync"?

[MintBean]

I use Unison to backup to my home server. It's similar to Rsync. I back up all files from my separate data partition, plus a few settings - primarily from Thunderbird and Firefox. The rest of the OS is pretty painless to reinstall if needed.

[WharfRat]

All of my important stuff resides on a partition mounted to /media/shared and the external mounts to /media/wd/backup so my single rsync line is

rsync -aviuh --progress --delete-after --exclude={.Trash-1000,lost*} /media/shared/ /media/wd/backup/shared/

which keeps both locations in sync.

rsync has many options, so I would suggest checking the man page for additional settings.

[dalexunil]

All of my important stuff resides on a partition mounted to /media/shared and the external mounts to /media/wd/backup so my single rsync line is

rsync -aviuh --progress --delete-after --exclude={.Trash-1000,lost*} /media/shared/ /media/wd/backup/shared/

which keeps both locations in sync.

rsync has many options, so I would suggest checking the man page for additional settings.

Thanks for the little deeper explanation WharfRat! More food for thought

[slipstick]

I've tried several different methods and still don't have it optimized. Believing in the belt + suspenders approach to backups, I am currently using a combination of methods, which duplicates the effort, but makes me feel a little better. My equipment currently comprises my main computer (steve-Z97X) and my old computer (steve-desktop), now used as a spare in case my main one fails and also used as a secondary repository for backups. These are both on my local wired network. In addition I have a 500 GB external USB drive as my primary backup repository.

For backup of system files, I use Timeshift to backup to another partition on my internal hard drive. Of course, this doesn't help if the drive fails, but I can always just re-install. This backup is really just in case an update borks the system (or I screw something up), so I can quickly restore the system. I also use Aptik to backup everything (data and system files) to a 64GB USB thumb drive - this is only updated every month or so and I keep the thumb drive in a fire resistant safe. This is my tertiary backup, after the USB external and the spare computer backups.

I have a separate data partition on my main computer, (/home/steve/Data), so I need to backup that as well as my home partition. Before I started using Back In Time for this, I used Grsync with two separate sessions, one for home, and one for Data, to back up to a partition on the external USB drive. With the Grsync options I have checked, the actual rsync commands executed are:

Code: Select all

rsync -r -n -t -p -o -g -x -v --progress --delete -u -l -H -D --numeric-ids -i -s -A -X /home/steve /media/steve/Z97_LM_Backup/home_backup

rsync -r -n -t -p -o -g -x -v --progress --delete -u -l -H -D --numeric-ids -i -s -A -X /home/steve/Data /media/steve/Z97_LM_Backup/Data_backup

(Of course, if you do the rsync from the terminal, you could use the -a option which bundles up the -rlptgoD options, but the above is the way Grsync lists them)

Actually, I'm still using this in addition to Back In Time (belt and suspenders). With Back In Time, I just set the "where to save" setting to /media/steve/Ext_Backup which is another partition on the external USB drive. I set the "include" setting to /home/steve, and I think I just used the default for the excludes. I also ticked the option boxes for "full rsync mode" and "preserve ACLs". This stores snapshots of both my home and Data partitions to the external drive. In addition to keeping the BIT snapshots on the external drive, I also periodically sync this to my spare computer with a (crude) script file I wrote:

Code: Select all

#!/bin/bash
# add -n option to rsync commands for dry run

# this line duplicates the /home (and ~/Data) snapshots to the backup PC and writes a log to ~/home_snapshot_list

rsync -aAuHXvis --progress --delete --numeric-ids -e ssh --exclude=lost+found /media/steve/Ext_Backup/backintime/steve-Z97X/steve/1/$(readlink last_snapshot) steve@steve-desktop.local:/home/steve/Backup/backintime/steve-Z97X/steve/1 | tee  ~/home_snapshot_list

echo "Finished transferring data backup"
sleep 10
exit 0

I also have Grsync sessions to copy my Data partition and firefox config and bookmarks and my Thunderbird config and data to the spare PC. I don't just backup the entire home because I have Cinnamon on my main computer and MATE on the spare PC. If the main PC fails, and I have to use the sprare, I want to have all my bookmarks and emails available. Grsync commands executed for these sessions are:

Code: Select all

rsync -r -n -t -p -o -g -x -v --progress --delete -u -l -H -D --numeric-ids -i -s -e ssh --exclude=lost+found -A -X /home/steve/Data/ steve@steve-desktop.local:/home/steve/Data

rsync -r -n -t -p -o -g -x -v --progress --delete -u -l -H --numeric-ids -i -s -e ssh -A -X /home/steve/.mozilla/firefox/mwad0hks.default/ steve@steve-desktop.local:/home/steve/.mozilla/firefox/mwad0hks.default

rsync -r -n -t -p -o -g -x -v --progress --delete -u -l -H --numeric-ids -i -s -e ssh -A -X /home/steve/.thunderbird/zsw2x4o8.default/Mail/ steve@steve-desktop.local:/home/steve/.thunderbird/zsw2x4o8.default/Mail

Eventually, I'll probably just stop using Grsync to backup to the external USB drive and stay with Back In Time to the external drive and spare PC and the above Grsync backups to the spare PC. I''m still trying to determine what I really want to do. I don't really recommend duplicating what I'm doing, just hope maybe it can give you some ideas.

[chrisuk]

There's many different ways to achieve the same result, everyone has their favourite, none of which may be right for you. You can use a one-line command - you can use a one-line command that gives you incremental backups (using rsync's --link-dest argument), so only the first backup takes any time - you can use software that you "set and forget" (I use Timeshift... my complete system, including Home, is restored in a few minutes; restored to another PC regularly too, as Timeshift takes care of grub and fstab for you).

Just wait until you get a lot of replies, test out a few methods - especially, test out the restore process; then pick one that works best for you.

[Cosmo.]

I recommend Back in Time (it uses rsync as backend). Very good customable, able to do fully automated backups.

[dalexunil]

Just wait until you get a lot of replies, test out a few methods - especially, test out the restore process; then pick one that works best for you.

I like that! Thanks

I've tried several different methods and still don't have it optimized.

That seems to be some of the beauty of Linux - there are soooo many options!

I recommend Back in Time (it uses rsync as backend). Very good customable, able to do fully automated backups.

Cosmo, I'm leaning more toward BiT simply because I've had some experience with it. However, I'm not opposed to learning another way!

Thanks for your input guys!

[austin.texas]

For my Data partition, I use Grsync to back up to a 500GB external drive. I like it because it is so fast and easy, once you have it configured. TUTORIAL.
Then I can connect my 500GB external drive to my other computers, and use Grsync to sync the data from the external to the internal drives, so that all computers remain identical.

For the root partition, I use fsarchiver to create a compressed image. TUTORIAL.

[DwightD]

I take a little different approach.

1. I connect a second hard drive to my motherboard (easy access via removable panel).

2. Load live dvd of linux mint.

3. use dd command to clone sda to sdb. This copies the entire operating system and user data.

For a 1TB drive it takes about 2 hours to complete. I usually keep two backups. One backup would be about a month old and the other about two months old. These drives are stored away from the computer.

[rbsudo]

I have a couple of older pc's that are currently running Lubuntu OS and have been using Backups. On this pc, on which I just installed Mint 18.2 w Cinnamon 3.4.4 recently, I also used Backups previously when it was running another OS. So I have been used to Backups for a little while now. I find it very easy to use. I went to the Software Manager to see if it was available there, but it was not. So I installed it via the terminal....

sudo apt-get install deja-dup

After install, it will appear as Backups on the Menu.

I've used it to do a few restores in the past, and it has worked very well.

[JoeFootball]

Adding to these excellent options, I use Clonezilla Live. It backs up the entire hard drive, so if the drive ever fails, or I need to restore the entire OS back to a previous point for whatever reason, I can easily do so.

Also, Clonezilla only backs up the used blocks of the device, making for a faster backup process, and the resulting archive being smaller than the source itself. I just save these archives to an external hard drive, and as I need space there, I just delete the oldest archive.

Admittedly, if you're someone that that has huge volumes of movies, music, pictures, or other large files, this may not be the most efficient option, but it is nice to have bare metal restores at your disposal.

Joe

[phd21]

Hi "dalexunil",

I just read your post and the good replies to it. Here are my thoughts on this as well.

It might help you, if you provided a link to your unresolved post.

It would help to know more about your system setup. If you run "inxi -Fxzd" from the console terminal prompt, highlight the results, copy and paste them back here, that should provide enough information.

Here is another post that is similar to this with good information
viewtopic.php?f=46&t=247720&hilit=luckybackup

Hope this helps ...

[dalexunil]

It might help you, if you provided a link to your unresolved post.

Here is the link:
viewtopic.php?f=90&t=249734

It would help to know more about your system setup. If you run "inxi -Fxzd" from the console terminal prompt, highlight the results, copy and paste them back here, that should provide enough information.

My System:

Code: Select all

dale@Bimmer ~ $ inxi -Fxzd
System:    Host: Bimmer Kernel: 4.4.0-21-generic x86_64 (64 bit gcc: 5.3.1)
           Desktop: Cinnamon 3.4.4 (Gtk 3.18.9-1ubuntu3.3)
           Distro: Linux Mint 18.2 Sonya
Machine:   System: Hewlett-Packard (portable) product: HP Pavilion dm4 Notebook PC v: 0692100000204710000620100
           Mobo: Hewlett-Packard model: 1793 v: 41.1C
           Bios: Hewlett-Packard v: F.08 date: 01/17/2012
CPU:       Dual core Intel Core i5-2450M (-HT-MCP-) cache: 3072 KB
           flags: (lm nx sse sse2 sse3 sse4_1 sse4_2 ssse3 vmx) bmips: 9977
           clock speeds: max: 3100 MHz 1: 905 MHz 2: 916 MHz 3: 999 MHz
           4: 1000 MHz
Graphics:  Card: Intel 2nd Generation Core Processor Family Integrated Graphics Controller
           bus-ID: 00:02.0
           Display Server: X.Org 1.18.3 drivers: intel (unloaded: fbdev,vesa)
           Resolution: 1366x768@60.03hz
           GLX Renderer: Mesa DRI Intel Sandybridge Mobile
           GLX Version: 3.0 Mesa 11.2.0 Direct Rendering: Yes
Audio:     Card Intel 6 Series/C200 Series Family High Definition Audio Controller
           driver: snd_hda_intel bus-ID: 00:1b.0
           Sound: Advanced Linux Sound Architecture v: k4.4.0-21-generic
Network:   Card-1: Intel Centrino Wireless-N 1030 [Rainbow Peak]
           driver: iwlwifi bus-ID: 02:00.0
           IF: wlo1 state: up mac: <filter>
           Card-2: Realtek RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller
           driver: r8169 v: 2.3LK-NAPI port: 2000 bus-ID: 04:00.0
           IF: eno1 state: down mac: <filter>
Drives:    HDD Total Size: 750.2GB (11.2% used)
           ID-1: /dev/sda model: WDC_WD7500BPKT size: 750.2GB
           Optical: /dev/sr0 model: hp DVDRAM GU60N
           rev: MU02 dev-links: cdrom,cdrw,dvd,dvdrw
           Features: speed: 8x multisession: yes
           audio: yes dvd: yes rw: cd-r,cd-rw,dvd-r,dvd-ram state: running
Partition: ID-1: / size: 682G used: 73G (12%) fs: ext4 dev: /dev/dm-0
           ID-2: /boot size: 472M used: 63M (15%) fs: ext2 dev: /dev/sda1
           ID-3: swap-1 size: 6.38GB used: 0.00GB (0%) fs: swap dev: /dev/dm-1
RAID:      No RAID devices: /proc/mdstat, md_mod kernel module present
Sensors:   System Temperatures: cpu: 52.0C mobo: 37.0C
           Fan Speeds (in rpm): cpu: N/A
Info:      Processes: 225 Uptime: 9:43 Memory: 1213.8/5914.9MB
           Init: systemd runlevel: 5 Gcc sys: 5.4.0
           Client: Shell (bash 4.3.481) inxi: 2.2.35 
dale@Bimmer ~ $ 

Thank you for your interest!

[phd21]

Hi "dalexunil",

You are welcome...

You seem to be having issues with BackInTime backup application, why not look at the link I gave you and try, Aptik, or LuckyBackup, or FreeFileSync, etc... For drive and partition image backups use something like "Clonezilla Live" ...

Personally, I found that the Backup Snapshot applications like BackInTime and TimeShift are slow and take up a lot of space for each backup snapshot, except if you are using the "Timeshift BTRFS" with the Linux BTRFS file system, then that is super fast and does not take up a lot of space. I can do an image backup of my entire drive by booting up to Clonezilla Live in less time than it takes either of these Snapshot applications to finish (except when using "btrfs") at least that is what happens on my ancient computer hardware. But, the "BTRFS" file system does not have all the nice repair tools that Linux has for the "ext4" file systems (the default Linux Mint file system) if you need them.

The file synchronization tools (LuckyBackup, FreeFileSync, etc...) are super quick after the first run, because they only update the changes to your system, new files, changed and updated files, etc...

I don't see any problems with the results of your "inxi -Fxzd" command, except that your "Dual core Intel Core i5" CPU processor is not running at its full potential, in fact it is running about a 1/3 of that.

CPU: Dual core Intel Core i5-2450M (-HT-MCP-) cache: 3072 KB
flags: (lm nx sse sse2 sse3 sse4_1 sse4_2 ssse3 vmx) bmips: 9977
clock speeds: max: 3100 MHz 1: 905 MHz 2: 916 MHz 3: 999 MHz 4: 1000 MHz

See this post to improve CPU performance ...
viewtopic.php?f=56&t=249555


Hope this helps ...

[dalexunil]

Wow, I really appreciate your trying to help me with this decision! I have limited time to work on this so it's going to take a few more days before I reach any reasonable conclusions but I trust your knowledge and experience based solely on your in-depth replies as well as your signature / Linux Mint Forum identity! Please stay tuned...!

[dalexunil]

I took a look at the link to the thread for improving my CPU performance. Did I correctly enter the commands in Terminal to accomplish this?

Code: Select all

dale@Bimmer ~ $ sudo apt-get install cpufrequtils
[sudo] password for dale: 
Reading package lists... Done
Building dependency tree       
Reading state information... Done
cpufrequtils is already the newest version (008-1).
The following packages were automatically installed and are no longer required:
  mint-backgrounds-serena python3-xlib
Use 'sudo apt autoremove' to remove them.
0 upgraded, 0 newly installed, 0 to remove and 31 not upgraded.
dale@Bimmer ~ $ sudo nano /etc/default/cpufrequtils
dale@Bimmer ~ $ sudo update-rc.d ondemand disable
insserv: warning: current start runlevel(s) (empty) of script `ondemand' overrides LSB defaults (2 3 4 5).
insserv: warning: current stop runlevel(s) (2 3 4 5) of script `ondemand' overrides LSB defaults (empty).
dale@Bimmer ~ $ 

What does not show up here in the code is when I entered the command in nano - GOVERNOR="performance" - exactly as I have presented it here, with the quotation marks. Is that correct? Thanks for your time

[phd21]

Hi "dalexunil",

You are welcome again...

You will have to reboot, then re-run "inxi -Fxzd" to see the results and if your CPU is running at optimum performance.

FYI: I just changed my Bios to disable the Intel SpeedStep (too not "let OS handle CPU speed"), and my CPU's work at full speed. Obviously, if you have a Bios option to enable multiple CPU cores and CPU threads, that should be enabled.


Hope this helps ...