Recommended image/cloning app

[chameau]

Hi,

I'm after suggestion/recommendation for what application to use to make a full image/backup of a disk or partition in Linux Mint 20.2 (dual-boot with Windows XP on one machine and Windows 7 on an other machine)
Back in Windows xp days, I used to use Norton Ghost 15 - and it saved my bacon a few times.
I’m not that comfortable using the terminal, so I’m looking for a GUI based app. I looked at 3 apps ‘Rescuezilla’ ‘Redo Rescue’ and ‘Foxclone’ I read all I could find about them all (only Foxclone has a detailed manual) They all seem to be based on the same core app ‘Partclone’ so they are likely to be similar in their functionality!? But none of them seem to has the ability to backup from the running system itself (Ghost can backup from within a running Windows system, but for restore needs booting of a CD) and the other major difference from Ghost is that, (as of current version), none can look at and extract an individual files from a backup images (Foxclone has a beta function - I think).
I don’t care too much about the backup from a running system, but I think the file extraction/restore from a backup function could be very useful. Out of the 3 I mentioned, which would you recommend? or maybe an other one I don't know about yet?

From reading the manual of Foxclone, I have a follow up question:
Under Windows I used to backup every partition separately. C drive and 2 data drives (2 partitions on a second physical drive in the same computer). I still have the 2 data partitions on the same hard drive (formatted as NTFS). The other drive is now dual-boot Windows/Mint20.2 That drive is in a caddy so I can swap it with an other drive. I have one with just Mint 19.3 on it (separate partitions for /, home, swap) and another dualboot with Windows XP. I also have an external usb hard drive for backups. But according to Foxclone manual, the recommendation is to back whole drives instead of individual partitions because if I change anything in the partitions (resize, move, etc…) between the last backup and a restore, the whole system would be stuffed! Is that a general problem with all image/backup tools?
I don’t think that was the case with ‘Ghost’ (Windows only system) or I was very lucky, as I certainly did change/resize partitions and never had a problem.
Thanks

[Pierre]

there has been an few programs that could do something like that.

Macrium Reflect
www.macrium.com/reflectfree
viewtopic.php?t=264431

FoxClone
www.foxclone.com
viewtopic.php?t=315557

you could use either of those two programs
- Macrium Reflect
to clone the entire drive .. to another drive

- FoxClone
to copy the one partition, or an number of partitions,
or to copy the entire drive .. to another drive

[Peter Linu]

Go with Foxclone. The maker is AndyMH of this very forum. He will tell you that there are (at least) 2 levels of security:
1. File level using Timeshift and BackInTime (saved my arse).
2. Foxclone. Even Foxclone itself has several alternatives strategies.

[AndyMH]

As you noted, underneath they all use partclone (which comes from the same stable as clonezilla). This works at the block level, it knows nothing about files, all it does is copy used blocks in partitions. For a viable backup you have to ensure that nothing in the partition changes during the backup. The only safe way to do this is make sure the partition is unmounted when you back it up. You cannot unmount your / partition booting normally. Now I suspect you could run a backup within your running system, but you would need to ensure that no files change during the backup (maintaining a mapping of every file to the blocks they use to check this). You have just increased the complexity of the software significantly. Given all the utilities you mentioned are FOSS, so cost you nothing, and generally developed by individuals 'for the love of it', this is a big ask. Not something I am prepared to do. To give you an indication of the level of effort, the next version of foxclone (not available yet) is currently +6,000 lines of code and has taken me most of this year to develop. A significant proportion of the code is making sure the user doesn't do stupid things.

The only linux backup utility I know of that might let you image a running system is Veeam, but this is enterprise level stuff, not free, and last time I looked the linux interface was terminal only, so not for newbies.

Extracting files from a backup image - again this is possible but... The default is to compress images to save space, and all the utilities are pretty efficient at this, typically you can expect anywhere between a 30%-50% reduction in image size compared to the used blocks on the source (I put some metrics in the user guide on this), so definitely something you want to do. Want to read an individual file - you need to decompress the image so need free space at least equal to the used blocks on the original partition. Then you can mount the image to see the files. More complexity, more work.

Why image the complete drive - there is nothing to stop you just making backups/restores of individual partitions, but as mentioned, change the partition table then what? You either restore the partition table - now you have a mismatch between the table and the other partitions on the drive - start sector, size potentially the file system. Or restore the partition and leave the table alone. You are now restoring a partition that may no longer fit on the drive, most likely partclone will throw an error, worst case it overwrites the following partition (used blocks are not nicely arranged at the start of a partition, they are spread out through the whole partition). Either way, the end result is an unbootable system and data loss. I know I've done this. This issue is common to foxclone, rescuezilla, et-al.

As long as you make no changes to size/position of partitions on a drive you are safe restoring individual partitions. On a restore, foxclone checks the partition table and if it has changed since the backup insists on restoring it.

If you want image level backup, then use foxclone or one of the alternatives, if you want file level backup then use the many excellent utilities for this purpose. I use timeshift (for the system) and backintime (for my data). Ideally you should use both, as I do, you cannot have too many backups.

What I do - all my PCs have an internal/removable 2TB HDD used only for backup. It has one large ext4 partition (where the backups live) and a small (12GB) ext4 partition with another copy of LM installed in it. I have foxclone installed in that version of LM. I have timeshift and backintime running daily automatically (after the first snapshot, they only copy what has changed = quick). Infrequently, I'll boot LM on the backup drive and take an image backup of my system drive with foxclone. If I screw up or lose data, I can restore individual files with timeshift/backintime. If I have a major problem or drive failure, I can boot LM on the backup drive, run foxclone and restore from image. I can then do a timeshift/backintime restore to get back to yesterday. On my dual boot desktop I have taken this one step further, I have my win C: mounted in mint at /home/andy/winC. This means that backintime takes file level backups of win.

[chameau]

Hi AndyMH, Thank you for creating Foxclone much appreciated, I will certainly give it a trial. I could not find a 'Donation' link on your site? be sure add one I be happy to contribute.
As I said I'm not to fust if I can not image my system while its running (I think Windows uses something called 'Shadow-Copy' and Linux doe not have an equivalent). It's not a big deal to plug in a usb stick and re-start the system.

Extracting files from a backup image - again this is possible but... The default is to compress images to save space, and all the utilities are pretty efficient at this, typically you can expect anywhere between a 30%-50% reduction in image size compared to the used blocks on the source (I put some metrics in the user guide on this), so definitely something you want to do. Want to read an individual file - you need to decompress the image so need free space at least equal to the used blocks on the original partition. Then you can mount the image to see the files. More complexity, more work.

When you say 'Need free space' do you mean you have to allow free space within the image file? or on the backup drive where the image is stored? if the later than that's not a problem, I always leave lots of free space on my drive/partitions. Is this something you're working on for the next version? I can see a good use for that. I assume any new version will be backward compatible with images made with Foxclone. How about images made with others like Clonezilla?

As long as you make no changes to size/position of partitions on a drive you are safe restoring individual partitions. On a restore, foxclone checks the partition table and if it has changed since the backup insists on restoring it.

So changes to content (changes to the amount of used vs unused space within the partition) is ok? I always leave lots of free space in my partitions and even in between them (see attached screenshots), so I'm unlikely to move/resize partitions.
As my systems does not change that much, I probably just do an initial image and then when I upgrade to a new kernel or before I mess-around in the terminal (just in case I break something).
Timeshift is limited by the fact that it only supports Linux file systems like ext4 - my backup drive is NTFS formatted (as I used it for Windows before) and so is the Data drive as it get data from Windows and Linux (Linux can read/write to NTFS, but Windows does not even see ext4!).

Could you explain the difference between a backup image and a clone image. Thanks

Main drive

Data drive

[AndyMH]

I could not find a 'Donation' link on your site? be sure add one I be happy to contribute.

No there isn't, I'm not in it for the money*

When you say 'Need free space' do you mean you have to allow free space within the image file? or on the backup drive where the image is stored?

On the drive, so if you had an image file that achieved 50% compression and was 50GB in size, you would need free space of 100GB to store the uncompressed image.

Is this something you're working on for the next version?

No not at the moment, some aspects are trivial, uncompressing the image file would be a command along the lines of:

Code: Select all

cat imagefilename | unpigz > uncompressed_imagefilename

where it gets more complicated is trying to make it idiot proof - e.g. making sure the user has enough free space wherever they are extracting it to (which means parsing the *.backup file to find out how big it will be), then mounting it for them. If I do do this, it would be a separate utility.

New version - backwards compatibility - yes it has to be. The new version is a rewrite, largely to improve maintainability, basic functionality is unchanged. There are 'improvements' - most windows are resizable, text size can be changed (for the 'hard of seeing') and there is more error checking.

Clonezilla - at the image file level, probably yes, but foxclone is a 'system', it holds information about a backup in a backup file (just a text file) that it reads to determine the content of a backup. Image, partition table and bootloader files follow a defined naming format. For compatibility with clonezilla I would need to understand how clonezilla 'packages' a backup and then build that into foxclone. I know the rescuezilla dev has spent a lot of time on this. My focus for foxclone has been to make it as newbie friendly as possible - simple point and click UI (doesn't mean it is simple to implement), decent user guide (the Achilles heel of so much linux software) and stopping users doing stupid things.

Could you explain the difference between a backup image and a clone image.

Same thing. But to try and make foxclone idiot proof, you can only 'restore' to the drive you took the image from (it checks the model name and serial number). Want to clone a backup to another drive - you need to select 'clone from file' on the clone tab.

Why clonezilla, foxclone, rescuezilla, redo rescue? In the 'old' days you only had two choices - clonezilla: the most capable but user unfriendly, or redo. So I used redo, it worked. Problem was it hadn't been updated since 2012, wouldn't boot on a lot of modern hardware (and there were bugs). Unknown to each other, the rescuezilla dev and I came to the same conclusion - a replacement was needed. We both launched within a couple of months of each other. Rescuezilla took over the redo user base, answered their questions, had a very similar UI to redo. This put the redo dev's nose out of joint and redo was rewritten and relaunched as redo rescue. If redo had been maintained, there wouldn't be a foxclone or rescuezilla.

* occasionally the rescuezilla dev and I talk, you can contribute to rescuezilla. I did ask him (probably about two years ago) how much he had got - about $50, it's not worth it.

[chameau]

Hi AndyMH,
Thanks for the thorough reply and explanation, I will certainly give Foxclone a go.
As you and many others said 'A backup or image is only as good as the restore'
I want to test a restore, but I do not want to risk my main drive by restoring to the same drive/partition I just backed up.
Can Foxclone restore to a VirtualBox VM? I assume I would have to make a 'clone' not a backup/image of the entire drive and not just a partition as I would be restoring it to different hardware? If Foxclone can not do that, I need to find a spare hard drive
To my knowledge, VirtualBox can restore/install from an iso, or vdi file and other VM files but not .image files?
So how could I restore a .image file made by Foxclone?
Thanks

[AndyMH]

No, foxclone cannot restore to a VM, it's just a file. I believe rescuezilla can. Yes you would need a spare drive and clone your backup to the other drive using 'clone from file' on the clone tab. The drive needs to be the same size or larger than the drive you backed up. You won't be able to boot that drive alongside your main drive (same UUIDs on partitions, confuses BIOS), something I might have a look at fixing.

[chameau]

Hi Andy,
I downloaded and used Foxclone to image my entire drive. I left all the settings as default. The backup took app 11 minutes and is app 21Gb - the total used space on the drive was only app 51Gb, so a very good compression ratio. Overall very good.

I do have one question: As I said, I left the default settings as is, but should I have untick the 'Extended' partition, and the 'Swap' partition? the data in the Swap is transient and it was empty anyway? The Extended partition holds Logical partitions but does not contain any data itself?
If I want to image just one partition and that partition is within an Extended partition, do I need to select both the partition and the Extended partition??

May I make a couple of suggestions for the next version (if not already done).
1) There is a time and date display in the GUI, but it is GMT time (8 hours behind for my time zone). There doesn't seem to be an app to change the time zone? and it seems to also change my hardware time (the time in the BIOS)? Could you add a time zone app, or if that is not possible just remove the time and date function it's only used as the default name of the backup that can be overwritten.
2) It's good that you have a CPU temp display, I have my CPU temp alarm set to 70 degree centigrade in the BIOS and it peeped a fair bit (the temp display reaching 78 at times) I would have loved to have a 'Pause button' or a way to throttle the CPU to keep my CPU from overheating, I know it can go to about 100 and still work OK, but it would shorten it's live - Any chance to get a pause button in the next version?

I will be testing the other 2 imaging apps 'Rescuezilla' and 'Redo-Rescue' and report my findings here.

[AndyMH]

The extended partition is part of the partition table. With a legacy partition table you are limited to four primary partitions (there isn't enough space for any more) so the extended partition is a 'fix', it is a pointer in the main partition table to another partition table on the drive that contains your 'logical' partitions. In a foxclone backup the partition table is always saved and if it detects it has changed on a restore it will insist on restoring it. All foxclone saves for a swap partition is the UUID, the start sector and size are in the partition table. If you restore a swap partition foxclone uses mkswap to create it with the original UUID. The time taken to 'save' the swap partition is negligible.

Be careful about taking a backup of an individual partition, you are okay if nothing else has changed. But if you have resized/moved/added other partitions after you did the backup you will be in trouble. Restore that partition, foxclone insists on restoring the partition table and you end up with a drive where there is a mismatch between the partition table and the partitions = no boot, lost data. I know I've done it. So, to be safe, always have one full disk backup. You can then subsequently backup individual partitions. Make any changes to the partition table - take another full backup. There is no easy way round this*.

Time - when I originally looked at this I couldn't find a simple way of setting the time so I locked the datetime to the hardware clock. I'll have another look at it for the next version.

There is no way I can implement a pause button. foxclone uses partclone to image the partitions, once it has started there is no way of pausing it other than killing it.

You should find very similar performance with rescuezilla and redo-rescue, they also use partclone to do the partition images with the same compression level. The major differences are in the user interface, and until recently rescuezilla didn't do clone (not sure about redo-rescue). Originally the only half-decent linux GUI backup utility was redo, problem was it was abandoned in 2012, no UEFI boot, bugs, wouldn't boot on a lot of modern hardware. Unknown to each other and in parallel, I did foxclone and Shasheen did rescuezilla. rescuezilla is a redo 'look-a-like', I did a completely new UI targeted specifically at newbies. rescuezilla took over the redo user base, this upset the original redo dev who got his finger out and updated redo as redo rescue. The iso for rescuezilla is more up to date than foxclone, last time I looked it booted a 5.11 kernel, foxclone is 5.4. redo-rescue was a 4.19 kernel (built off debian, foxclone & rescuezilla are built off ubuntu).


* the next version of foxclone quite deliberately makes it more difficult to backup individual partitions. By default all partitions all selected for backup and you have to change another setting before you can select/deselect partitions - trying to stop newbies from making mistakes.

[MAlfare]

.......... so the extended partition is a 'fix', it is a pointer in the main partition table to another partition table on the drive that contains your 'logical' partitions.

Sorry, but that is not true!
There is no other partition table.
Logical partitions are daisy chained. Each one points to the beginning of the next.

[AndyMH]

I was keeping it simple and yes there is more than one partition table.

[MAlfare]

I was keeping it simple and yes there is more than one partition table.

Its not a table (containing list of logical partitions) it's more a link.
EBPR, beside of the description of the logical partition it belongs to, holds a link to to the next logical partition (its EBPR), daisy chained, as I said.
Its good to keep things easy, but not so easy that they get false
I teached those stuff in the nineties of the last century.

[Aztaroth]

I was keeping it simple and yes there is more than one partition table.

Hi Andy,
Just FYI.
You're dealing with the guy who knows better than TG how Timeshift works (viewtopic.php?p=2098069#p2098069), who knows better than Pjotr (viewtopic.php?p=2102014#p2102014) what to recommend users...
So, you've a hard time coming in front of you if you enter this debate
Won't mix again. Just trying to save some of your time.

[AndyMH]

I wasn't going to Any differences are down to semantics (or maybe me being a pedant).

Now on the subject of timeshift... Mine runs automatically and it insists on mounting my backup drive/partition at /run/timeshift/backup even though I have it mounting at /media/ultrabay via fstab. It has a cron job running hourly, so even if I manually unmount it, an hour later it will have remounted it. Any ideas on stopping this other than another cron job that checks if rsync is running and if not, umounts it? I have filed a bug report on it. It annoys me, timeshift should leave the filesystems in the same state it found them once it has taken its snapshot.