Foxclone - linux image backup, restore & clone

[Dumfy]

I've not tested with with btrfs, but partclone (which foxclone uses) does support btrfs, so it may work. Try and post back.

Hi Andy - just following up from previous to say that FoxClone works with btrfs paritions.

I backed up and restored a btrfs partition with FoxClone successfully - after a minor hiccup which was probably down to me.

After restoring, the system failed to boot with a GRUB error, so I used a Mint ISO for Boot Repair and all good thankfully. During restore, I de-selected the Partition Table option as nothing had changed and FoxClone gave me no error. Should I have left the option selected and this is the reason for the GRUB problem? As I exited out of the bootable back up drive, it hung up at the Mint logo and didn't shutdown and I had to reset. May be that caused it?

The main thing is the btrfs partition backed up and restored. I installed Mint to a USB drive along with your .deb file.

I'm a Linux newbie trying to move away from Windows and have used Mint for the past couple of months while dual booting with Windows. A friend suggested I try openSUSE Leap 15.1 which I did - and hence the btrfs question. openSUSE is very polished and looks nice, but not near as user friendly like Mint for someone new to Linux. I'll switch back to Mint when 20 comes out. I did try Leap on Mint via VirtualBox. but it seems to run better as a proper install

The btrfs partition is 393GB and I thought FoxClone would create a bit-for-bit image of it and be the same size, but the resulting file size was approx 9GB on compression level 2. It was quick too with back up taking 5mins and restore around the same.

Anyway, hope the feedback is useful and thank you for all your work with FoxClone . . . . now a regular for me.

Dumfy

[AndyMH]

Thanks for reporting back.

When you backup a partition with foxclone it will automatically backup the partition table. When you restore, it checks the current partition table against the backup and will insist on restoring it if the partition table has changed since the backup (it will tell you if you try to deselect the partition table on restore). So, assuming nothing else has changed, you shouldn't have had a problem with boot.

Yes, it is bit-for-bit, but foxclone only backs up used blocks on a partition, not the whole partition. There is also a default setting that compresses the backup. Typically this will reduce the size of the backup to 50% - 70% of the used space on the partition. That will be the reason you thought it was quick and the file size small. In this respect it is no different to the other linux image backup utilities such as clonezilla and rescuezilla.

[rene]

When you backup a partition with foxclone it will automatically backup the partition table. When you restore, it checks the current partition table against the backup and will insist on restoring it if the partition table has changed since the backup

FWIW (I've never used Foxclone and haven't read the thread/issue in detail) unless in very specific circumstances it's normally fine to restore a parition of original size X to any other partition of precise size X, at whichever, potentially other, absolute on-disk location. As to partition ordering Linux-sides the "modern" methodology of specifying file-systems via FS-UUID rather than partition identifier (or PT-UUID) takes care of that; for Windows partitions you just need to keep relative ordering to not have e.g. D: and E: change role.

As such, allowing restoration to identically sized partitions even if the table as such is unequal might supposedly be an "advanced option" somewhere...

[Drugwash]

As proven in my previous post here, a raw image of a partition (even boot one) can be restored to another partition of any size, provided the target size is at least equal to the source size. Haven't tried with a smaller target, not sure if the Disks utility would be able to trim the free space of the source image to fit the target space.
Then, GPartEd's Check option would stretch the free space of the original image to fill the unallocated space in the target partition.
I succesfully restored the raw image of a 53GB bootable partition into a larger 84GB partition, which I'm using right now. So it can be done using the same tools used by Foxclone - it just needs some adjustments. Hopefully such feature will be implemented in Foxclone soon.

[rene]

a raw image of a partition (even boot one) can be restored to another partition of any size, provided the target size is at least equal to the source size.

You are then wasting disk-space until you have in a second step either shrunk down the partition or conversely expanded the filesystem (as you suggest). At that point it to me seems fairly clear that you do not in fact want to use Foxclone in the first place. I.e., if you want/need that kind of control then why are you not doing this manually to begin with? A tool like Foxclone that's intended to be used by those that do not want to or know how to do that should not leave things partly done since it shouldn't be expecting its users to then go do the second part independently.

As such, restoring to an equal size partition may be a good addition I feel; to any partition of greater or equal size not so much.

[Drugwash]

What I did was deliberate. I wanted a larger target partition than the initial one was. For an exact clone one of the tools used in the process should provide an automatic way to create, or shrink or enlarge an existing partition to the exact size of the source. Shouldn't be hard to do. It's not only about the exact size down to a byte, but also the sector size which can differ from source to destination.

All operations could be done from within Foxclone if such option would be available. The only delicate part would be editing fstab when dealing with restoring a bootable partition, or when restoring a normal partition with a different file system and/or name. But whoever takes on such task should already have some knowledge of what they're up to, and/or be provided with a detailed manual covering all possible options and situation - and I heard very good things about Foxclone's manual so far.

These are all advanced features that some people may need. I needed this badly - see the screenshots with the failing HDD, and it would've been great if Foxclone already provided such features; in absence I had to perform everything manually, which took more time. If I didn't already have the raw image taken a day before I might've lost that partition forever.

Generally speaking, I started to observe long ago a sad fact, that is most software (including operating systems!) dealing only with the most obvious (and simple) situations, completely leaving out complicated situations and corner cases - that is, precisely what people would need the most help with.

[rene]

It's not only about the exact size down to a byte, but also the sector size which can differ from source to destination.

No. Foxclone makes a straight byte-for-byte image of the source. Neither what sized "sectors" the kernel eventually read from the source device nor what sized "sectors" the kernel will eventually write to the destination device is of interest to it. All it conceptually knows is that it has N bytes of data -- usually comprising a file system with its own block size which while ideally related to, an integer multiple of, the sector size of the underlying device is still conceptually irrelevant to foxclone: it just clones whatever it's told to clone.

Anyways. I'll leave further or non comment on what Foxclone is and/or should be to Andy.

[Drugwash]

Sorry, it was not sector size but block size - my bad. The block size issue popped up in Boot Repair's log - dunno what this is related to:
Warning: The driver descriptor says the physical block size is 512 bytes, but Linux says it is 2048 bytes.
The log is 54.2kB in size.

I believe our personal knowledge and experience should help Andy in building a better tool.

[AndyMH]

My original objective with foxclone was to provide a simple to use image backup/restore/clone tool for newbies. It was my answer to the demise of redo (which unknown to me, at the time was being updated and is now available as rescuezilla - we are in touch and talk ). A limited range of functionality as idiot proof as I can make it. Apart from a few 'own goals', most of the changes since January have been protecting the user from themselves, such as:

• Trying to use a partition on a drive without a partition table. Disks will let you create this, I know, I did it about three years ago. Just one reason I don't like disks.

• Running as root so that it will not throw up errors if the user is backing up to a partition created by gparted and they don't know they have to change the permissions if they want to use it.

• Automatically splitting backup files when backing up to a fat32 partition - how many newbies are aware of the 4GB file limit?

I will continue to make changes as users find new ways of breaking its basic functionality, but I've no plans to extend the functionality further. As I've said before, if you want something more sophisticated - use clonezilla.

While I was interested in what drugwash was doing - copy just one partition to a new drive. The code for that would be non-trivial:

• Check the partition table on the new drive - is there enough space in one contiguous lump?

• If not enough space - can enough space be made available? which partitions should be shrunk and/or moved to create it? Now have to look at the used blocks on each partition. How much free space should I leave on each partition? Or do I leave this to the user - sounds like a gparted style GUI, in fact it sounds exactly like gparted.

• Shuffle the partitions around to make the space. Now this is okay with linux, it uses the UUIDs, not sure about win = more research.

• Make sure the partition table is up to date, add in the 'new' partition.

• Restore/clone the new partition - the easy bit!

• Now what about boot?

I started foxclone about March last year, it took until December for the beta release and I've been doing 'bug fixes' about every 2-3 weeks since. It's approaching 4,000 lines of code with nearly 3,000 lines of comment. To extend the functionality as above would be, best guess, three months minimum, more likely six months. Hence my lukewarm interest.

For info, foxclone knows nothing about files, all it understands are blocks of bytes in partitions.

[AndyMH]

Problems with the website - for some reason the download is sticking a bunch of line feeds to the end of the download, don't know why, we are working on it. The net result is the wrong md5sum and an iso that won't boot.

If you download the iso and the md5sum does not match that shown on the website - a quick fix is to trim the iso:

Navigate to the folder where the iso was downloaded and in a terminal:

Code: Select all

dd if=foxclone39-02.iso of=test.iso bs=591396864 count=1 status=progress

Replace test.iso with a name of your choosing.

Check the md5sum on the new iso:

Code: Select all

md5sum test.iso

Check the output against the md5sum shown on the website:

Code: Select all

8ee69e5d37d492f4eecfdd9acfda1ffa  test.iso

Sorry

[Drugwash]

Andy, your work is appreciated. Clonezilla is a mess, I would never use it.
The information I offered was all in good faith, hoping it would help in some way.
And I myself am a newbie in regard to Linux - for 20 years I've been running MS-DOS and Windows, so all the Linux stuff is new to me.
Nobody knows how many other people may find themselves in a situation similar to mine.
I know what programming takes, I've done my share in the Windows world. All open-source, of course, because I do believe in freedom.
It's all up to you if you want to take this application to a higher level or not. If yes I'm sure there will be feedback to help you.

Thank you for all your work so far!

[thx-1138]

Problems with the website - for some reason the download is sticking a bunch of line feeds to the end of the download,
don't know why, we are working on it. The net result is the wrong md5sum and an iso that won't boot.

...Andy, have you (and Larry i assume) thought of maybe hosting the .iso files over at Github instead?
(Someone can also host the whole website there as well: https://pages.github.com/)
Just a thought that crossed my mind, so don't really take it much seriously,
as i'm pretty unfamiliar in regards to how webhosting works in those days, the current bandwidth costs etc...

[AndyMH]

Larry says he has fixed it, haven't checked myself yet.

Two reasons we haven't used github, wanted control, second I've no idea how to use github - the last year with foxclone has involved a lot of learning and research, didn't want to learn yet one more thing and no idea if you can host a 600MB on github. Okay three reasons, can't count

Drugwash - just in case you read my response that way, no criticism/offence implied. Just that I'd like to do something else for a while. Think for the functionality you suggested, the best way would be a standalone program, not part of foxclone - it wouldn't be for a newbie user - I was impressed with the way you found a solution. Subject to bugs being found, the next major thing I must do with foxclone is figure out a way to verify backup images.

[Drugwash]

Drugwash - just in case you read my response that way, no criticism/offence implied. Just that I'd like to do something else for a while.

No worries Andy, we're cool. The information is out there should you or anyone else need it. People helped me in time and I like to help too whenever/however I can in return.
All the best to you all!

[bob466]

I've been doing some more testing.

I created 5 separate Images of my spare 500GB HDD running Mint Cinnamon 19.1 and restored each separately and all worked just fine...great.

As we all know...HDD or SSD failure is the worst thing that could happen and could occur at any time...the reason I have a Image of my Drive stored on an External HDD...unfortunately this is where FoxClone Fails completely...which is a shame.

Using the same 500GB HDD as above...I wiped it and tried to restore any of the five Images and every time got...Warning...Drive ata st 500 was on sdb when the backup was taken. It is now on sda...continue with restore to sda ? I clicked...Yes... Restore...but wont restore the Image and get...FoxClone is not responding.

We are warned in the Instructions..."Restore will only restore to the drive the backup was taken from".It also should say...If you wipe or replace your HDD or SSD...FoxClone wont work.

[AndyMH]

It is now on sda...continue with restore to sda ? I clicked...Yes... Restore...but wont restore the Image

I'll do some more testing, this was a 'fix' I did a couple of months back when I realised depending on what a user does, drive XYZ maybe on sda one day (when the backup was taken), but sdd the next.

In the meantime, I think you can do what you want with 'clone from file' - you tell foxclone which drive you want to clone your backup to.

[sanmig]

...the reason I have a Image of my Drive stored on an External HDD...unfortunately this is where FoxClone Fails completely...

Um, don’t know what you mean? Nearly all my FoxClone backups are written to ext. USB drives. I’ve also backed up USB thumbdrives to internal and other USB drives?

...If you wipe or replace your HDD or SSD...FoxClone wont work.

I think your issue is the term “restore”? FoxClone!
Not responding after error message, do you mean all buttons dimmed?
Yes, that does happen.

[sanmig]

(I’m a n00b, please correct me if I’m wrong)!
- Generally the terminology may be a bit confusing, let me try (also see wikipedia):

(A disk) Image is thought a 1:1 copy of a drive into a (usually) single file. Used and unused space is taken sector by sector, regardless of garbage, including deleted stuff, ready to recover files, photos, emails, likely your life from that image.
- That’s the best way to “catch it all” (and to waste space and time) although we should probably talk about free space.
Some SW will have an option to spare unused space, but that’s not very common (?) and not helpful in case you are searching for lost / deleted data.

Example (@Drugwash): Linux Disks will write a 1TB drive into one single 1TB image file, there is no option to exclude unused space, no option to segment into 4GB files.
The image file probably can be mounted (Disk Image Mounter, but doesn’t mount ntfs partitions inside an image - ?) and checked.
- Here FoxClone “fails” because of standing on it’s own foot: Permission denied.
Good luck (e.g. with restore) if the (single) 1TB img file got corrupted later, or immediately when taking the image because of a failing drive.
With suspicious drives: Take at least two!

Verify means to compare the saved file(s) with the source. That’s better than nothing, but only half of the cake.
The only way to confirm success is writing back and to check/boot. Be brave!

Backup in terms of Foxclone means to copy sectors of used space of a drive / partitions into separated files. To know what is “used” it (Ubuntu 18.4) has to understand the filesystem (not the files themselves). If it can’t, FoxClone will write a 1:1 full sector copy of that partition into the saved partition’s file (e.g. with Winwoes recovery/OEM/… partitions).

- With FoxClone the first MB is always saved 1:1 as a “Date.grub” file, containing the boot sector and partition table, probably very handy in case of MSDOS drives (don’t know much regarding GPT).
FoxClone writes one .img file per partition, that may be an advantage in case of disaster.
As a default (option) any partition file is Gzipped to save even more space,
but that may be dangerous:
“wrong pwd” == “This is not a valid Gzip-file”, difficult if not impossible to “repair”.

At each backup FoxClone writes a couple of files, simple and useful, kudos, see the docs.
But:
Always read the log files after a backup,
success at FoxClone means successfully written into the log files!

Restore means to write saved files back to a drive.
As a safety measure, FoxClone will only “Recover” to the very same drive the backup was taken from:
A n00b’s drive mismatch will not destroy innocent drives (Disks will! )
- Contrary to Disks one can’t use “FoxClone - Recover” to move stuff to a new drive.

Clone is the (not only FoxClone) function to ‘restore’ to different drives (of same size or bigger), either directly (drive to drive) or from backup files to drive.

E.g. @Drugwash, having a MSDOS (MBR) source drive, could have easily used FoxClone to backup only the first partition (the first MB, containing boot sector and PT, is automatically saved as .grub) and use “Clone from file” instead of “Recover” to write it to a new drive.
- The other source partitions would have been created but without FS and empty.

Of course the target drive must have space for all of his 3 source partitions.
- Otherwise the .sfdisk file would have to be # edited …
The beauty of FoxClone is it’s simple design!

My take:
- fstab, grub repair and partition size are issues separated from backup / clone.
- Extended partitions, UEFI and GPT add more burden to a “one fits all” solution.
- Before going into expert mode, FoxClone should proceed on it’s way to a simple, foolproof (n00b) tool.

Subject to bugs being found, the next major thing I must do with foxclone is figure out a way to verify backup images.

Hmmm …
I’d prefer to compare a second backup (on other drive) to the first one: Safety first.
Only then try to restore!

[AndyMH]

As we all know...HDD or SSD failure is the worst thing that could happen and could occur at any time...the reason I have a Image of my Drive stored on an External HDD...unfortunately this is where FoxClone Fails completely...which is a shame.

Using the same 500GB HDD as above...I wiped it and tried to restore any of the five Images and every time got...Warning...Drive ata st 500 was on sdb when the backup was taken. It is now on sda...continue with restore to sda ? I clicked...Yes... Restore...but wont restore the Image and get...FoxClone is not responding.

Did some more testing:

• Took a 320GB HDD (Hitachi), connected to laptop via sata/usb cable.

• Installed LM19.3 to it in legacy mode. Drive is showing as sdb and installed grub to sdb. Rebooted to it - okay.

• Did a foxclone (V39) backup.

• Checked backup by wiping the drive - put a GPT partition table on it with a single NTFS partition, did a foxclone restore. Rebooted to it - okay.

Now to move the drive to a different sata port:

• Wiped drive again. Attached another 500GB drive via a second sata/usb cable and rebooted. Hitachi is now showing as sdd - it has moved from sdb.

• Did a foxclone restore - warning message comes up - backup was from sdb, drive is now on sdd - continue - YES.

• Click restore button, okay on confirm window, restore proceeds.

• Check result with gparted - sdd (Hitachi) is now msdos partition table with an ext4 partition.

• Reboot to sdd - LM19.3 boots as it should.

[bob466]

Hi Andy,

I tried "Clone File to Disk" and it worked perfectly. I used another spare HDD running Linux Lite...Booted into FoxClone and selected..."Clone File to Disk" and from the drop down Menu selected."Drive to Drive"...then went from there.

Once finished Cloning the File... I've found you must remove both Flash Drive and Portable HDD containing the Image because if you leave the Portable HDD plugged in when Re-Booting...you just get a black screen.

The spare HDD that had Linux Lite...now has an Image of Cinnamon 19.1 and works fine...so I tried to Re-Image this Drive with an Image of Cinnamon 19.1 from another spare HDD but it wouldn't work even though it's the same Image...something to remember.