Foxclone - linux image backup, restore & clone

[sanmig]

I'm happy to continue here as long as the mods are happy.

OK.
I’ve now tried FoxClone 35-02 from USB (I know I’m *really* slow).

First: The User Guide is very useful, clear and detailed, kudos, that’s very uncommon!
I was successful with backup and restore of an internal drive partition (sda1 MSDOS bootable old Linux), but the handling is not easy for a n00b because of some inconsistencies you may know of.

Anyway, here my questions:

1) To me the most interesting one:
How did you manage to have Disks and GParted working together?
I thought Mint is based on Ubuntu 18.04 bionic?
After booting Mint, Disks shows my sda correctly with primary and extended partition. But after checking with GParted (partitions OK) Disks will not show sda correct again until the next boot.

But with your FoxClone booted from USB Disks doesn’t have that problem after using GParted (thanks for adding both, each has it’s own advantages).
Mint and FoxClone have same versions of: Disks 3.28.3 (UDisks 2.7.6) and GParted 0.30.0
The problem seems to be caused by an old issue with udisks and extended partitions, way above my head, see:
viewtopic.php?p=1585332#p1585332
Very strange!
So the bug is only visible in Mint and not visible in Ubuntu 18.04.4?
Mint 20 will then still have the same issue?

2) What about the “.grub” file, it is visible (and correct) in the uncompressed backup files, as written in the doc.
But there is no such file in the (default) compressed backup files. Is it included in the img.gz file? See cloning section below.

3) The backup log files (2.x MB ! ) have hundred lines of similar scary Error message, just different in the compressed and uncompressed backup version.
Last line says success? What are these messages good for?

Compressed has dozens of:

Code: Select all

image write ERROR:Broken pipe
read(1048576) and write(-1) different
image write ERROR:Broken pipe
read(217088) and write(-1) different
image write ERROR:Broken pipe

BUT:

Code: Select all

Syncing... OK!
Partclone successfully cloned the device (/dev/sda1) to the image (-)

Uncompressed has dozens of:

Code: Select all

image write ERROR:File too large
read(217088) and write(-1) different
image write ERROR:File too large

BUT:

Code: Select all

Syncing... OK!
Partclone successfully cloned the device (/dev/sda1) to the image ...

4) I had issues when cloning the sda1 backup from file to an external disk.
I was semi-aware of the “full backup” mentioned in the pdf, but as it was possible in FoxClone I tried:

Cloning the compressed (default, 2) partition sda1 to the external drive was reported “successful”.
It was very fast (5’), but the drive isn’t shown in Mint’s Nemo.
(Nemo’s “Computer” has it, but is “Unable to mount location”).

- Disks has the drive visible with all sda partitions (1 primary, free space, 4 extended) but could mount only sda5 the swap (didn’t try, don’t have Mint configured with swap, don’t know what could happen). All other partitions do not have the triangle to mount.
- GParted reports all partitions, but filesystem unknown, also for sda1 (except linux-swap).
Btw. checking again with Disks the cloned external drive is correctly showing partitions, while my sda is “mangled” again.

- Clone did not delete parts of the previous EFI system on the drive (it was an old OS X Time Machine disk).
- The GRUB part is missing (all 00h), but the partition table is there from 462 to 511 ending in 55 AA.
That made me nervous, because my backup + restore was writing over the still functional sda, I did not try to destroy sda1 or boot sector first, would restore of partition sda1 “repair” such a problem, in contrast to clone?

I didn’t try to clone from the uncompressed backup, because I think it would be a similar result only including the boot sector?
Shouldn’t “Clone” prevent using a single partition?
Or clone everything that is included in the backup, partition or full?
(n00b here, I have no idea if possible – but halfway isn’t good anyway)
Would my bootable sda7 include bootsector as well as bootable sda1?
Why include other than selected partitions, why not the file system of the selected partition?

- Sorry for so many questions, curios is my second given name ….

[AndyMH]

Lots of questions, not a lot of useful info for me to work with. Glad you liked the user manual - did you read the section on troubleshooting?

How did you manage to have Disks and GParted working together?
I thought Mint is based on Ubuntu 18.04 bionic?

I just bundled disks and gparted in the iso, nothing more. Personally I don't like disks, don't think it is very good and nearly all it does can be done in gparted. Mint19 is based on 18.04 and the foxclone iso is a minimal 18.04.

2) What about the “.grub” file, it is visible (and correct) in the uncompressed backup files, as written in the doc.
But there is no such file in the (default) compressed backup files. Is it included in the img.gz file? See cloning section below.

The grub file should be present in a backup irrespective of whether the images are compressed or not. It is simply a binary copy of the first MB of the drive. The user has no choices here, it is always created. The grub file and partition table file (sfdisk) are the first things created when doing a backup.

3) The backup log files (2.x MB ! ) have hundred lines of similar scary Error message, just different in the compressed and uncompressed backup version.
Last line says success? What are these messages good for?

Log files more than a few bytes and with error messages mean there was a problem with partclone - you have an error. What's causing it - reading between the lines I think you are saying this was with an uncompressed backup? I suspect I've got the command line to run partclone wrong, apart from some early testing, all my testing has been with compression switched on. Read the troubleshooting section, repeat what you did but run foxclone from a terminal with foxclone --dump, copy/paste the output into a text file (there may be quite a lot) and send it as an attachment to help@foxclone.com. A gparted screenshot of the drive in question would also be useful. I can then try and figure out what is going wrong.

4) I had issues when cloning the sda1 backup from file to an external disk.
I was semi-aware of the “full backup” mentioned in the pdf, but as it was possible in FoxClone I tried:
Cloning the compressed (default, 2) partition sda1 to the external drive was reported “successful”.
It was very fast (5’), but the drive isn’t shown in Mint’s Nemo.
(Nemo’s “Computer” has it, but is “Unable to mount location”).

When I said clone from a full backup, I meant it! If you have just cloned a single partition you will have a drive with a partition table pointing at non-existent partitions (except for the one you cloned) and don't expect it to boot or do anything useful.

Shouldn’t “Clone” prevent using a single partition?

This has crossed my mind, but there is a trade-off between flexibility and trying to make it idiot-proof, the user manual is quite explicit - first words after the heading "Clone File to Drive":

49. Starting point – you must have a full backup of the drive you are cloning from.

[sanmig]

(Success! )
Thanks for your reply, regarding useful info I didn’t know if you’d like to follow up.
Yes, I have read the troubleshooting part, however, I did not expect trouble when the GUI stated “successful”.

Re Disks vs. Gparted I agree, the advantages of Disks are the GUI design (not like Win XP), the immediate visibility of all available drives (good luck to spot them in Gparted for a n00b) and – the most important – Disks is included in Mint. FoxClone is interesting because of the partition image, Disks has the same option for single partitions but no compression. However, I prefer Gparted before Disks, too.

Gparted and Disks in Mint 19.x:
So it is a bug in Mint that GParted and Disks don’t work together, it’s OK in Ubuntu 18.04.
Very sad, because the Mint team doesn’t know about and thus will have the same issue in Mint 20 as well.

Re grub file:
The funny thing: It was only created / written in the uncompressed backup session (but wait, read on!).
Now I’ve checked: The grub file (first MB) is always created, also with non bootable partitions (hmmm, don't know if I like it, esp. when writing back to disk).
- "Disks" partition image doesn’t include the first MB even when imaging a bootable partition (bad!).
To backup / restore the first MB is great, but to be useful both, backup and restore, should have selection boxes for that.

Re log files:
A log file containing error messages is the default in Linux , so I didn’t really care.
Important to me was: GUI stated success, and both backup log files ended in:
“Syncing… OK! Partclone successfully cloned the device (/dev/sda1) ...”.
(- Hmm: The logfile is from backup, but it says “cloned” and “device” but it’s backup & partition)

“- you have an error” - Yep, it’s me, not your command line
My first test was with an external drive, backup + restore = success.
To dig deeper with the clone option I’ve used an other drive as the backup destination: Size was OK, but the drive was formatted as FAT32, what I did not realize at that moment was the file size limit!
Thus the image files were written up to 4.2GB and then silently (!) closed

Why the grub wasn’t written first (to disk?) when compressed + FAT32 I don’t know, but the GUI listed writing the grub in the 3. box before partition 1 as “complete”.

run foxclone from a terminal with foxclone --dump,

Yesterday I’ve tried the (compressed) terminal dump (to FAT32), and interestingly this time the grub file was created at first, together with the .sfdisk file.
To double check I’ve tried again from GUI and the grub was there, too! I dunno what has happened.
The dump is very interesting, I guess you did not expect this kind of blunder
Anyway, I’ll send you the dump to check by email and add more of the log files and the .backup file here:

The content of the log file is identical for ext4 and FAT32 except the error message block “Broken pipe”:
- For compressed backup (what is the meaning of the (-) ?):

Code: Select all

Partclone v0.3.11 http://partclone.org
Starting to clone device (/dev/sda1) to image (-)
Reading Super Block
memory needed: 5626084 bytes
bitmap 3528928 bytes, blocks 2*1048576 bytes, checksum 4 bytes
Calculating bitmap... Please wait... 
done!
File system:  EXTFS
Device size:  115.6 GB = 28231424 Blocks
Space in use:  45.1 GB = 11004000 Blocks
Free Space:    70.6 GB = 17227424 Blocks
Block size:   4096 Byte
Total block 28231424
image write ERROR:Broken pipe
read(1048576) and write(-1) different
### … lot of identical messages, first always 1048576, later other numbers.
image write ERROR:Broken pipe
Syncing... OK!
Partclone successfully cloned the device (/dev/sda1) to the image (-)

- For the uncompressed backup (see the path/filename in contrast to compressed (-) + message “File too large”):

Code: Select all

Partclone v0.3.11 http://partclone.org
Starting to clone device (/dev/sda1) to image (/mnt/foxclone/sdb4/FOXCLONE/Uncompressed/20200418.sda1.img)
Reading Super Block
memory needed: 5626084 bytes
bitmap 3528928 bytes, blocks 2*1048576 bytes, checksum 4 bytes
Calculating bitmap... Please wait... 
done!
File system:  EXTFS
Device size:  115.6 GB = 28231424 Blocks
Space in use:  45.1 GB = 11004000 Blocks
Free Space:    70.6 GB = 17227424 Blocks
Block size:   4096 Byte
Total block 28231424
image write ERROR:File too large
read(1048576) and write(-1) different
image write ERROR:File too large
read(1048576) and write(-1) different
### … lot of identical messages, first always 1048576, later other numbers:
image write ERROR:File too large
read(8192) and write(-1) different
image write ERROR:File too large
read(1048576) and write(-1) different
image write ERROR:File too large
read(217088) and write(-1) different
image write ERROR:File too large
Syncing... OK!
Partclone successfully cloned the device (/dev/sda1) to the image (/mnt/foxclone/sdb4/FOXCLONE/Uncompressed/20200418.sda1.img)

The .backup file:

Code: Select all

Apr 16 2020, 20:02
Compression:YES
Split files:NO
Model:ATA Samsung SSD 850
Mount point:sda
Partition:sda1 : ext2 : primary : 114GB : 38% used [PCLOS] : Flags-boot *PCLinuxOS*
Blocks:sda1,42243248
Partition:sda2 : extended
Blocks:sda2,0
Partition:sda5 : swap : e6a556d2-fad2-4105-bfd3-cfb8f6459c35
Blocks:sda5,0
Partition:sda6 : ext4
Blocks:sda6,0
Partition:sda7 : ext4
Blocks:sda7,0
Partition:sda8 : ext4
Blocks:sda8,0
End

Conclusion:
- The issue is not detecting the user’s fault, the file size limit should give a warning and choice to split the image.
I’ve seen the option in the docs, but yes, didn’t notice the format before using the partition.
- Also the success messages should be avoided in case of error(s), both GUI and log file.
- For the cloning of a single partition from file – contrary to the advise of the “legal” fine print in the documentation - I’d suggest a warning, as you see here, idiots are plenty!
- Backup and restore of the first MB is very useful, though. Which simple GUI SW would do that?

[AndyMH]

Thanks for the response, I hadn't checked backups to fat32 (all my testing was done with ntfs or ext4 as the destination). I'll have a look at that in slower time. I'll also think about a warning if a clone from file is attempted without a full backup (the info is there in the backup file to check if it was full).

Not going to change the backup of 'grub' on non-system disks, it is after all only one MB. It would mean a lot of extra code to determine if the drive was bootable. It's not necessary on a UEFI boot, but I left it in because it was too much trouble to change.

Log file contents (and output on completion) - this is straight from partclone, I can't change that. Did think about not bothering with log files and sending the output to dev/null, but left it in, think you've proved it can have a useful purpose.

foxclone --dump is for me to help bug fixing, every procedure/function called (well, most of them) is dumped to the terminal together with the parameters passed to it. If there is a problem, it helps to find out where it went wrong. If you look the dump at some point you will find the command line passed to run partclone, e.g.

Code: Select all

CmdLine =pkexec partclone.ntfs -c -F -s /dev/sda4 -L "/media/andy/winback/20200426.sda4-log.txt"  | pigz -c -2 > "/media/andy/winback/20200426.sda4.img.gz"

In this case, the output from partclone is piped to pigz (to compress it). That's why you get the 'image (-)' in the output - partclone doesn't know what the destination is, and file too large (and I think broken pipe) is it hitting the fat32 limits (which foxclone doesn't currently check for).

[sanmig]

Not going to change the backup of 'grub' on non-system disks, it is after all only one MB. It would mean a lot of extra code to determine if the drive was bootable.

Um, I can see the asterisk for “bootable” in the GUI? Anyway, no problem:
The grub backup isn’t in question, although I’d like to see that in the GUI (2nd box, e.g. dimmed) or in the docs e.g. page 9 just before point 16 (e.g. “The first MB will be always backed up in an extra file named .grub”).
On the contrary, it’s an advantage to have it in an extra file, easy to compare.
- Oh, renaming the .grub before restore / clone will silently jump over, I love it!

Re log files, I guess you could check if there is an error message in the log and give a warning to check it?
However, to cheer “success” should be avoided in this case

One more point for you to check:
At Clone from file, after drive, select partition (2nd box), then to select the backup folder/file at first the File Manager opens with a selection I don’t understand (ssh-, systemd-???), - but after “Cancel” another window pops up with the folders of the selected drive + partition.
(Btw the FoxClone File Manager is useless to me, I don’t understand it, as well as the “Display” link)

[AndyMH]

At Clone from file, after drive, select partition (2nd box), then to select the backup folder/file at first the File Manager opens with a selection I don’t understand (ssh-, systemd-???), - but after “Cancel” another window pops up with the folders of the selected drive + partition.

I'd need a screenshot to understand what your problem is here. When you click on the partition containing the backup, foxclone is mounting that partition and showing you the folder/files on that partition so you can navigate to the folder containing the backup.

Btw the FoxClone File Manager is useless to me, I don’t understand it, as well as the “Display” link

I needed a lightweight file manager, that was easy to install in the iso (in the ubuntu main repositories). It is pcmanfm, I tend to agree with you, others rave about it.

The display application (arandr) is there so if you need to, you can change the screen resolution. When I plug my ultrawide (2560x1080) into my laptop and boot foxclone it comes up 1024x768, looks awful. With arandr I can change it to 1920x1080 - still not brilliant, but better, and perhaps more important, make it my primary monitor so that I can see the panel.

[pbear]

Btw the FoxClone File Manager is useless to me ...

What are you trying to do? If you want to be able to move or copy things to your hard drive or a flash drive (e.g., a screenshot), you need to mount the target. Might be some other way, but easily done in Terminal. How I do it is to create a sub-directory in /home/ubuntu, then mount to that.

[AndyMH]

V37 is now out. Automatically splits files on a backup to a fat32 partition to avoid the 4GB file size limit.

[sanmig]

I'd need a screenshot to understand what your problem is here.

OK, I’ll try (still on 35-02):

[ First fighting with Screenshot: When saving a screenshot, at first the folder “Ubuntu” is given, the listed options don’t show my desired target partition. Now using “Other…” the File Manager’s “Other locations” present most of my local partitions, but not the dev/sdb4 (FAT32) “MiscData” with my “FOXCLONE” folder where I e.g. tried to save the FoxClone files to.
(#Rem: Can’t take a screenshot of that when I’m at saving screenshot.)
When I click at “Computer” (the File Manager’s first entry) in an attempt to see all attached drives (like with Nemo) to mount one, it ends at home/ubuntu, no way to find/select sdb or esp. sdb4.
OK, I’ll use an other partition.
(To change the filename the text is already selected, but typing does nothing: One has to click it and select it again.)
But: When I take a screenshot of the Desktop before starting FoxClone the save dialog lists my MiscData partition to select from ??? ]

Anyway, here are the screenshots (*):
This is before selecting my source partition (sdb4):
https://pasteboard.co/J5Wub8d.png
At selecting a strange dialog window is shown:
https://pasteboard.co/J5WuRmo.png
After “Cancel” this second window comes up and it is ready to select my sdb4/FOXCLONE folder:
https://pasteboard.co/J5WveNQ.png
Canceling again will then send me back to FoxClone’s partition selection.

(*) Important: There is a difference in clicking at the text part or clicking at the checkbox to select the partition!
a) Only the very first selection of any partition will open the “strange” window.
b) Clicking the text (sdb …: whatever […]) will open only one window - the strange one or the correct one,
but clicking the checkbox will always result in 2 windows to “Cancel” to be back at the partition selection, strange+correct or correct+correct.
Likely this sounds confusing, so tell me if you need more hints.
- But is it only me, this should happen on every machine booted with FoxClone 0.35?
Didn’t anybody else report that?

Re “Display”,
The “View” options 1:4, 1:8 or 1:16 change the box and font size, the button with the green hook doesn’t do anything.
Open/Save Layout have a nice view of my drives, OK, but …?
With “Outputs - HDMI1” I’ve tried 900x600 (to increase the GUI size), nothing happens - gosh, the green hook throws: “XRandR failed: cannot find mode 960x600”.
- I told you, I’ll always try the wrong door first, now I see several other resolutions will work, flickering ones included.
System/Keybindings (whatever that is) throws an error “gconf not available.” ?

Btw., at least in 0.35:
Why is the "Networking" enabled by default, sounds creepy for a simple backup boot solution without browser?

Thank you for the quick update, I’ll check the V37 soon!

[sanmig]

What are you trying to do?

What I’m trying to do is to use the File Manager to see my partitions and folders, probably create another subfolder to store my files to, copy / move, yes, something like that.
The FoxClone File Manager, when opened, shows /home/ubuntu, with the folder “Desktop” and a file "foxcloneV6.pdf". The folder is OK, but the .pdf has no application to open (reader not installed).
Clicking “Home Folder” it’s empty, "Application" has some icons:
Accessories: 4 icons OK, User Guide throws an error.
Other: Empty
Preferences: Desktop Preferences throws an error, Display and Network OK.
Programming: Empty
System Tools: Disk Usage Analyser is very good, thanks! Disks OK, FoxClone throws an error, GParted OK, HW-Info OK, Terminal doesn’t open, TestDisk too.
So what to do here?
Menu “Go” - Devices: Operation not supported. What did I expect?

... easily done in Terminal.

All good, I love the Win95 style of PCManFM , I can handle that - only here at FoxClone boot it seems to be crippled beyond minimum. I can’t freely look around as I can do e.g. in the “Display”s Open/Save dialog (except that there the file type is locked).
The Terminal … well, that reminds me back to the era before Win 3.1, even further down to my first steps with CP/M. That might be the last resort when a Desktop system fails badly, but rest assured, when you send me to the Linux Terminal, within 5 minutes I’ll have my HD destroyed and smoke coming out of monitor and power supply.

Nowadays many freak out when they see a computer mouse: Three buttons, a wheel, why? A 105 key keyboard? B&W screen? No smileys? SERIOUSLY?
[/rant]

[pbear]

Any problem is insoluble if you try hard enough. This one, though, may resist your best efforts and be solvable anyway.

In Foxclone, open both Terminal and File Manager. In Terminal, run sudo mkdir /home/ubuntu/Mint (or Home or Data, wherever your data files live), then sudo mount /dev/sdxn /home/ubuntu/Mint (or whatever), replacing sdxn with the actual partition ID, e.g., sda1. ScreenshotS (Alt-PrScr) will go to /home/ubuntu. Rename; copy or move to desired location In /Mint (or whatever). When ready to shutdown Foxclone, sudo umount /home/home/Mint (or whatever). Boot Mint. The files you copied/moved there while booted in Foxclone should be easy to find, as you put them there.

[AndyMH]

As I've said earlier, I wanted a lightweight file manager. I'm not a fan of pcmanfm. What I specifically do not like is that in nemo (cinnamon fm) it shows you a list of devices/partitions in the left pane, click on one and it will mount and open it. Can't find any way of doing that in pcmanfm. So the GUI way to do that is open disks (another program I don't like) and use that to mount a partition. Then you will see it in pcmanfm. Clunky.

Yes, if you try and open the pdf in pcmanfm it will fail, as will a number of other items inside pcmanfm. I'm not going to fix it. They all work from either the desktop or menu. Why would you want to launch them inside the file manager? Everything in the iso is to support foxclone, not the other way round.

I might have a look at a different file manager, but not high on my list of priorities and a change is likely to increase the size of the iso.

I remember CP/M, it's what I did my first programming in (assembler, dbase II and turbo pascal).

[sanmig]

In Terminal, run sudo ...

- Nice try, but I’ll resist this time, thanks anyway!

[sanmig]

Why would you want to launch them inside the file manager?

No, I don’t. I’m a fan of clean solutions, just what is necessary, no distraction.
- GUI of course
Booting from USB to backup/restore requires FoxClone and the docs.
A (bad, hmm...) file manager isn’t necessary (*), FoxClone has everything to get the job done (thumbsup!).
Right click to (hopefully) change screen resolution is standard.
The menu at left bottom has everything else, good tools, neatly organised.

- My problem: When there is a button I’ll click it (my curiosity, sorry).

But I’d like to repeat the point regarding networking, is it necessary to run FoxClone?
Wake on LAN included?

———
(*) Re PCManFM, I think there is an install issue. See it’s menu Edit /Preferences /Volume Management, mountable volumes should be mounted automatically. At “Layout” there is an (not set) option to show “Devices” in the side pane, but there it runs into “Operation not supported”.
(Start the Disk Usage Analyzer, all drives/partitions are there, click one to mount. Then double-click it on the desktop, the PCManFM works nearly as expected.)
Anyway, not worth the effort, it’s not necessary for FoxClone's function.

V37 is OK with FAT, thanks!

[pbear]

In Terminal, run sudo ...

- Nice try, but I’ll resist this time, thanks anyway!

Refusing to use terminal is one thing. It's quite another to complain you can't do things when the reason is that you refuse to use terminal.

Anyhoo, I'll try to remember not to pester you with solutions in future.

[sanmig]

Anyhoo, I'll try to remember not to pester you with solutions in future.

Well, when your solution is limited to the terminal world, that’s an option,
but you are welcome anyway.

[harry_fine]

Been trying to get to the Foxcline site for 2 days, just get this in my browser:

Fatal error: Uncaught PDOException: SQLSTATE[21S01]: Insert value list does not match column list: 1136 Column count doesn't match value count at row 1 in /home/foxclone/public_html/index.php:27 Stack trace: #0 /home/foxclone/public_html/index.php(27): PDO->prepare('INSERT INTO dow...') #1 {main} thrown in /home/foxclone/public_html/index.php on line 27

[sanmig]

Fatal error: Uncaught PDOException:

Thank you, they've fixed it already!

[AndyMH]

Thank you, they've fixed it already!

They in this case is Larry78723, and yes he fixed it. Without him, there would be no website

[racer-x]

Just want to say thanks for this awesome s/w. It's definitely easier to use than clonezilla. I've been using it for the past two days creating and restoring between two Linux OS's. It's worked out great with only a couple of minor issues. Very well done.

Thanks!