How to backup your stuff UNIX-style

[deadguy]

My suggestion would be to use labels as a compromise between UUIDs and legacy. Labels have a distinct maintenance advantage over UUIDs and still allow for a unique ID for name collision purposes.

Fred

I agree with Fred here!

and, if that is indeed Scorp, glad to see him back

cheers,

Deadguy

[sofasurfer]

Scorp123.
I'm making good progress and learning a lot. But I don't understand one thing. How do you run a script such as yours at the beginning of this thread?

[kpswalin]

This a great walk through, and I am customising it for my specific partitioning scheme and system. I have however hit a snag and was hoping for some guidance from someone more experienced.

When I perform the following command,

Code: Select all

 tar --one-file-system --exclude=/tmp/* --exclude=/dev/* --exclude=/proc/* --exclude=/mnt/* -pzcvf /home/'My Home Directory'/Desktop/RootFS_backup.tar.gz /

I get this when the action completes.

Code: Select all

/srv/
/dev/
tar: /dev/: file is on a different filesystem; not dumped
/sys/
tar: /sys/: file is on a different filesystem; not dumped

I assume it has something to do with '/proc/mounts' based on some Google searches, but I am not experienced enough to solve it or change my command.

Any guidance is appreciated.

Linux Mint Rocks!!!

Ok, so I see the '--one-file-system' keeps tar from backing up anything that is not within the local file system, i.e. remote mounts. But why is /dev (even though it is excluded) and /sys on a remote file system. It appears to me they are local?

Code: Select all

~ $ ls -lh /
total 704K
drwxr-xr-x   2 root root 4.0K 2008-12-17 19:49 bin
drwxr-xr-x   6 root root 4.0K 2009-01-02 09:07 boot
lrwxrwxrwx   1 root root   11 2008-11-28 19:32 cdrom -> media/cdrom
drwxr-xr-x  15 root root  14K 2009-01-04 21:25 dev
drwxr-xr-x 147 root root  12K 2009-01-04 21:25 etc
drwxrwxr-x   8 root root 4.0K 2008-11-28 22:21 Gallery_Remote
drwxr-xr-x   4 root root 4.0K 2008-12-01 20:41 home
lrwxrwxrwx   1 root root   32 2009-01-02 09:05 initrd.img -> boot/initrd.img-2.6.27-9-generic
lrwxrwxrwx   1 root root   32 2008-11-28 19:42 initrd.img.old -> boot/initrd.img-2.6.27-7-generic
drwxr-xr-x  16 root root 4.0K 2008-12-20 16:32 lib
drwx------   2 root root  16K 2008-11-28 19:31 lost+found
drwxr-xr-x   3 root root 4.0K 2009-01-03 14:24 media
drwxr-xr-x   4 root root 4.0K 2009-01-04 14:30 mnt
drwxr-xr-x   2 root root 4.0K 2008-10-29 16:53 opt
dr-xr-xr-x 153 root root    0 2009-01-03 09:14 proc
drwxr-xr-x  31 root root 4.0K 2008-12-26 23:43 root
drwxr-xr-x   2 root root 4.0K 2009-01-02 09:04 sbin
drwxr-xr-x   2 root root 4.0K 2008-10-29 16:53 srv
drwxr-xr-x  12 root root    0 2009-01-03 09:14 sys
-rw-r--r--   1 root root 615K 2009-01-02 09:16 tftpboot
drwxrwxrwt  17 root root 4.0K 2009-01-04 21:44 tmp
drwxr-xr-x  13 root root 4.0K 2008-12-26 22:57 usr
drwxr-xr-x  14 root root 4.0K 2008-11-28 22:17 var
lrwxrwxrwx   1 root root   29 2009-01-02 09:05 vmlinuz -> boot/vmlinuz-2.6.27-9-generic
lrwxrwxrwx   1 root root   29 2008-11-28 19:42 vmlinuz.old -> boot/vmlinuz-2.6.27-7-generic
~ $


Here is my fstab too

Code: Select all

# /etc/fstab: static file system information.
#
# <file system> <mount point>   <type>          <options>                       <dump>  <pass>
proc            /proc           proc            defaults                        0       0
/dev/sda2       /               ext3            relatime,errors=remount-ro      0       1
/dev/sda1       /boot           ext3            relatime                        0       2
/dev/sdb1       /home           ext3            relatime                        0       2
/dev/sdb2       none            swap            sw                              0       0
/dev/scd0       /media/cdrom0   udf,iso9660     user,noauto,exec,utf8           0       0
nas:/share      /mnt/nas        nfs             defaults                        0       0


I thought I was getting better at this, but I guess I have not graduated from noob yet.

[kpswalin]

Can I just safely exclude /dev and /sys directories. I have seen on the web where others are suggesting that. What are the implications? If there are none I assume my new command would be something like this

Code: Select all

tar --one-file-system --exclude=/tmp/* --exclude=/dev --exclude=/sys --exclude=/proc --exclude=/mnt/* -pzcvf /home/'My Home Directory'/Desktop/RootFS_backup.tar.gz /


But that may require that I recreate the /dev and /sys directories when restoring a system. I am wondering if scorp123 just added the "--exclude=/dev/*" so that the /dev directory would be recreated during a restore?

Why didn't scorp123 exclude the /sys directory if it is in fact not needed?

[calibre97]

AARGH! I am sooo close. Here's my setup:
Dell D630 with an internal 250GB drive, and a 120GB external USB drive.
/dev/sda1 XP (I back it up after booting to Acronis TrueImage Home 11 and connecting a USB drive)
/dev/sda5 Data1 (ext3, 100GB)
/dev/sda6 Data2 (ext3, 50GB)
/dev/sda7 M7Home (ext3, 5GB, for Mint 7 "/home")
/dev/sda8 M7Root (ext3, 10GB, for Mint 7 "/")
/dev/sda9 swap
/dev/sda10 M6Home (ext3, 5GB, for Mint 6 "/home")
/dev/sda11 M6Root (ext3, 10GB, for Mint 6 "/")
/dev/sda12 (ext4 so no label, 5GB for Kubuntu 9.04 "/home")
/dev/sda13 (ext4 so no label, 10GB for Kubuntu 9.04 "/")

I'm trying out the techniques described in this thread, but backing up to Data1 for now (I'll rsync that partition to the external USB drive). I boot up with RescueCD v1.1.7 and create directories under "/A/" for all points. For example, /A/K9Home, and so on, and then mount them.
I can use the tar command for /A/K9Home to /A/Data1/K904/K9home_backup.tar.gz and /A/K9Root/boot to /A/Data1/K904/K9boot_backup.tar.gz. I'm stuck trying to tar up "/" though. I keep getting kicked back with "zsh: no matches found: --exclude=/A/K9Root/tmp/*. Here's the command I'm using, straight from your examples:
root@sysresccd /A/Data1/K904 % tar --one-file-system --exclude=/A/K9Root/tmp/* --exclude=/A/K9Root/dev/* --exclude=/A/K9Root/proc/* -pzcvf K904root_backup.tar.gz /A/K9Root

I've also tried that with a trailing slash at the end for "/A/K9Root/".
I run ls on /A/K9Root/tmp and it's there so I don't understand why the --exclude is choking on it. I have a lot of partitions so I can try out various versions of Mint and Kubuntu. I'm about to blow away the Mint6 stuff, but I'd like a backup just in case. I'd also of course like a backup of everything else. Previously, I was booting the rescueCD and rsyncing each partition to a like partition (with Data1 and Data2 not being the same size of course) on the external 120GB USB drive. I thought I'd try this technique, creating TAR balls on Data1 so I could just rsync Data1 to the external drive and have the stuff handy in case a partition went tits up.

Can you explain why I'm having so much trouble backing up "/"? Is it because it's ext4 perhaps? I'll try doing it with the Mint6 root partition, which is ext3, and see how that goes.

UPDATE: Nope. Doesn't work for Mint 6 either. If I remove the tmp option, it chokes on /dev. I try --exclude=/tmp/* and --exclude=/A/M6Root/tmp/* (thinking since I'm booted to the RescueCD, I need to give the explicit path to the /tmp directory in the Mint 6 partition. But no go.

I'm officially stumped. I guess I'll go back to rsyncing all of the individual partitions to the external drive for now.

[cldx3000]

This is a crazy awesome tutorial, actually i was looking for the one which explains how to set up partitions in a smart way since i just started with Mint but this definetely is one of the most insight tuts ive ever read, ill copy it to my USB Drive alongside my Backups for future reference for sure. I was wondering if the same procedure would be working on an external Harddrive thats connected to my Router, or would that be on the stupid side of things to do?

[eiver]

I have read the above thread and also the very interesting: http://www.halfgaar.net/backing-up-unix
I decided to use dar instead of tar after reading the link above.
The only question is - should I include /dev in my backup. I saw some people arguing for, but also some tutorials suggesting to exclude it.
When I include it I get some errors while restoring /dev to a new HDD. Also the system reports some errors during boot. The system boots however and is usable.

What should I do to get the file system as close to the original as possible (including all metadata like permissions, hard and soft links, etc.)
Include or ignore /dev?

Consider a scenario: HDD failed and was replaced, but all other hardware is still there
Consider a second scenario: The whole machine is fried and the system must be restored to a different machine with different hardware

[webb1976]

Thanks for this.

I just took a beginner's Linux class and what you posted MADE A TON OF SENSE. It is amazing what you can do when you know the basics of how things work. If this were Windows people would be making restore points and all other craziness.

[kwevej]

This tutorial is way too complicated.
That's what i use:
(system and /home partitions separated)

system backup (you can run this while working with your system):

Code: Select all

#!/bin/bash
DEV=/dev/sda2
mount -o remount,acl,user_xattr $DEV
BPATH='./sys/back_sda2_'$(date '+%F')'.fsa'
echo 'saving fs'
nice ./fsarchiver -A -o -z7 savefs $BPATH $DEV -e /lost\+found -e /media -e /mnt -e /cdrom -e /proc -e /dev -e /tmp -e /sys
echo 'remounting fs'
mount -o remount $DEV
echo 'done'


system restore (run from liveCD):

Code: Select all

#!/bin/bash
DEV=/dev/sda2
MNT=/media/sysrmnt
nice ./fsarchiver restfs $1 id=0,dest=$DEV
mkdir -p $MNT
mount $DEV $MNT
mkdir $MNT/tmp
chmod 777 $MNT/tmp
mkdir $MNT/sys
mkdir $MNT/proc
mkdir $MNT/dev
mkdir $MNT/media
mkdir $MNT/mnt


(gpg encryption left out of the scripts)

for home backup i use incremental backup https://help.ubuntu.com/community/DuplicityBackupHowto

[steev]

I think the correct permissions for /tmp are 1777 (this prevents users from deleting files that don't belong to them)
Otherwise the script looks fine.

[agrandir]

A very useful and educational set of instructions which I personally have found very useful, especially in regard to saving MBR and lpartition tables, as well as list of packages installed, which are always things one tends to forget and on re-installation it becomes guesswork for those with a poor memory as to what to put where. However, as this Howto was written 3 years ago and Grub2 has now replaced the legacy versions used at this time, then might I suggest a modification so that newbies, like myself, don't screw up their system in trying to modify UUIDs. menu.lst no longer exists in the /boot/grub directory.

I don't feel confident enough for suggesting how to go about changing UUIDs so if someone more competent happens to read this ...

[oldfart]

You don't need to worry about the UUID. There is no need to change the fstab. If you are going to put your backed up system on a different device all you need do is change the UUID, of the device you are going to write to, to the UUID that you backed up from. To do this one uses the appropriate filesystem utility. Something like tune2fs or tunefs.reiserfs. It's a simple command. Read the man pages or do a google search to learn more.

Also if your MBR gets thrashed you failed to follow the advice to THINK before acting. So one shouldn't need a backup of their MBR. Also it can be rewritten a number of different ways depending on what is the OS that wrote it. In windows it's fdisk /mbr and with a grub system just reinstall grub with grub-install or input lilo for a lilo boot loaded system. You do need to be chrooted to do that but it's not hard.

Now a days hard drives are cheap. I bought a 2 Terabyte drive on sale for $80. ( The first hard drive I had was ten megs and cost hundreds of dollars. ) So I use a very large drive in an enclosure and just copy and paste it in some sort of file manager. Don't forget your dot files.

This was a slightly interesting idea but it's over kill when I can do the same thing with a copy paste. Yes your ego might get a stroke or two from using the command line, I got over it you can too, but that is no longer needed to do a simple backup.

Time marches on. March with it or be left behind.