Cloning a hard disk - In search of best practice

Questions about hardware,drivers and peripherals
Forum rules
Before you post please read how to get help
Post Reply
111MilesToGo
Level 2
Level 2
Posts: 90
Joined: Fri Oct 14, 2016 5:43 am

Cloning a hard disk - In search of best practice

Post by 111MilesToGo » Wed Jun 13, 2018 10:26 am

I would like to gather your recommendations as to the best practice for cloning a hard disk to a larger one, enlarging one out of several partitions in the process. Sorry if my thoughts below are complicated and convoluted, and if I am asking for education, but such cloning is a major, important and error-prone task...

I have a 2 TB SSD as my main drive in the laptop, and I want to clone it to a 4 TB SSD. The reason is an enormous growth of my music collection due to more and larger hi-rez files, such that one of my partitions needs to grow considerably.

My current SSD is GPT partitioned, EFI booted. It has a number of ext4 partitions for two Linux operating systems on it (four partitions, holding separate root and home each), plus a separate swap partition, plus an EFI boot partition, plus an ext4 partition to hold my VirtualBox virtual machines, plus an NTFS partition serving as a software store for both Windows and Linux, plus an ext4 partition for the music. This is the gdisk output for my current SSD:

Code: Select all

sudo gdisk /dev/sda
[sudo] Passwort für myself: 
GPT fdisk (gdisk) version 1.0.3

Partition table scan:
  MBR: protective
  BSD: not present
  APM: not present
  GPT: present

Found valid GPT with protective MBR; using GPT.

Command (? for help): p
Disk /dev/sda: 4000797360 sectors, 1.9 TiB
Model: Samsung SSD 850 
Sector size (logical/physical): 512/512 bytes
Disk identifier (GUID): xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
Partition table holds up to 128 entries
Main partition table begins at sector 2 and ends at sector 33
First usable sector is 34, last usable sector is 4000797326
Partitions will be aligned on 2048-sector boundaries
Total free space is 2669 sectors (1.3 MiB)

Number  Start (sector)    End (sector)  Size       Code  Name
   1            2048         2099199   1024.0 MiB  EF00  EFI System Partition
   2         2099200       203425791   96.0 GiB    0700  Mint Root
   3       203425792       304089087   48.0 GiB    0700  Mint Home
   4       304089088       505415679   96.0 GiB    0700  Tumbleweed Root
   5       505415680       606078975   48.0 GiB    0700  Tumbleweed Home
   6       606078976       840959999   112.0 GiB   0700  Virtual Machines
   7       840960000       908068863   32.0 GiB    0700  Exchange
   8       908068864      3961450495   1.4 TiB     0700  Common Data
   9      3961450496      4000796671   18.8 GiB    8200  Linux Swap
And here is the blkid output:

Code: Select all

sudo blkid
/dev/sda1: UUID="4912-D9DE" TYPE="vfat" PARTLABEL="EFI System Partition" PARTUUID="xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
/dev/sda2: LABEL="Mint Root" UUID="xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" TYPE="ext4" PARTLABEL="Mint Root" PARTUUID="xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
/dev/sda3: LABEL="Mint Home" UUID="xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" TYPE="ext4" PARTLABEL="Mint Home" PARTUUID="xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
/dev/sda4: LABEL="Tumbleweed Root" UUID="xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" TYPE="ext4" PARTLABEL="Tumbleweed Root" PARTUUID="7xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
/dev/sda5: LABEL="Tumbleweed Home" UUID="xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" TYPE="ext4" PARTLABEL="Tumbleweed Home" PARTUUID="xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
/dev/sda6: LABEL="Virtual Machines" UUID="xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" TYPE="ext4" PARTLABEL="Virtual Machines" PARTUUID="xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
/dev/sda7: LABEL="Exchange" UUID="xxxxxxxxxxxxxxxx" TYPE="ntfs" PARTLABEL="Exchange" PARTUUID="xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
/dev/sda8: LABEL="Common Data" UUID="xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" TYPE="ext4" PARTLABEL="Common Data" PARTUUID="xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
/dev/sda9: UUID="xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" TYPE="swap" PARTLABEL="Linux Swap" PARTUUID="xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
(All UUIDs replaced by xxx's)

My goal is: Clone this 2 TB SSD to a new 4 TB SSD, leaving all partitions the same size except for the one "Common Data" (/dev/sda8) partition, which should grow to encompass the additional 2 TB on the new SSD (i.e. push the swap partition further out accordingly). All identifiers (UUIDs) must remain identical, so that (a) both Linux systems, (b) a Windows 8.1 virtual machine, and (c) a Windows 8.1 physical machine residing on an external hard disk continue to function flawlessly, in particular being able to mount all partitions on the new SSD as they do now on the old SSD.

What is the best practice to achieve this?

(A) I am not experienced with Clonezilla (yet). In particular I wouldn't know whether Clonezilla is able to achieve the desired enlargement of the one music partition (/dev/sda8) on the fly. Use it for my purpose?

(B) Although I have Acronis True Image Home 2018 running from my physical Windows 8.1 machine and use it regularly to backup things, I do not at all trust this black box enough yet for my current purpose.

(C) I would like to use transparent tools from the terminal to do the job in several steps. I can imagine to do something like the following from a live system (since one doesn't want the original and the target SSDs to be mounted):
(i) Use gdisk to set up the GPT partitioning of the new SSD to my needs. Ordering and sizes of these new partitions are thus achieved as I want them.
(ii) How do I assure that all identifiers needed by my Linux AND Windows operating system get cloned properly? What identifiers do exist, anyway? From the above gdisk output, I see an overall disk identifier (GUID), and per partition I see a partition label (PARTLABEL) and partition UUID (PARTUUID), plus a filesystem label (LABEL) and filesystem UUID (UUID). For the NTFS partition, the filesystem UUID has a different structure as compared to the ext4's. What other identifiers do I have, e.g. I read about s.th. like a Windows serial number. Again, how to set ALL identifiers?
(iii) I suppose I can then clone the EFI and the Linux partitions via Clonezilla, one by one. Or maybe use rsync to "copy" the home partitions and the ext4 partition holding the virtual machines ("Virtual Machines", /dev/sda6).
(iv) What about the NTFS partition ("Exchange", /dev/sda7)? I guess it might be best to copy its contents under Windows, switching a file read/write check on. But maybe I can also clone it via Clonezilla.
(v) Finally, I am concerned about my music partition, the "Common Data" (/dev/sda8). My point of concern is: Is there something like a reserved size of the directory? By this I mean: When formatting a partition to a certain type (ext4, or NTFS), e.g. via gparted, then gparted thereafter reports that x MiB's of this partition are in use. What is that x MiB's good for? Is it a directory size, and would it have to differ for the different sizes of my music partitions on the old SSD (1.4 TiB) as compared to the targeted 3.4 TiB on my new SSD?
(vi) Thus, how to clone my music partition? Format it to ext4 (e.g. via gparted) as the first step, give it the proper identifiers, then simply use cp or rsync or a graphical filemanager to copy the files as the second step? I would like to check data integrity in the end, using checksums (e.g. Krusader under Tumbleweed).

(D) I can also think of using a simple "dd" to copy partitions which don't change size, or use "copy" and "paste" within gparted. I think the latter turns to dd or similar to do the job.

Please, if you kept reading up to this point, could you help me by recommending a best practice step by step, and pointing out what to take care of in particular? Thanks a lot in advance.

User avatar
WharfRat
Level 20
Level 20
Posts: 11371
Joined: Thu Apr 07, 2011 8:15 pm

Re: Cloning a hard disk - In search of best practice

Post by WharfRat » Wed Jun 13, 2018 3:54 pm

I'm not sure if dd is the best practice for cloning disks, but I've cloned several disks using it with no problems.

Just clone the entire disk then you can manipulate/resize the partitions of the new one,
ImageImage

Ken Parkes
Level 2
Level 2
Posts: 60
Joined: Thu Jan 14, 2010 4:19 pm

Re: Cloning a hard disk - In search of best practice

Post by Ken Parkes » Thu Jun 14, 2018 11:44 am

Have you considered setting up a RAID, a natural for three discs like you have. Look at
https://raid.wiki.kernel.org/index.php/RAID_setup for a good guide.

Ken

User avatar
AndyMH
Level 4
Level 4
Posts: 430
Joined: Fri Mar 04, 2016 5:23 pm
Location: Wiltshire

Re: Cloning a hard disk - In search of best practice

Post by AndyMH » Thu Jun 14, 2018 1:44 pm

I would avoid dd (disk destroyer), it is so easy to make a simple typo and trash your system. Within the last month we've had a post where this is exactly what happened.
Thinkpad T430 i7-3632 Cinnamon 18.3, Thinkpad T420 Cinnamon 18.3, Thinkpad T410 Cinnamon 17.3, Thinkpad T60 19.0 Mate

User avatar
all41
Level 12
Level 12
Posts: 4375
Joined: Tue Dec 31, 2013 9:12 am
Location: Computer, Car, Cage

Re: Cloning a hard disk - In search of best practice

Post by all41 » Thu Jun 14, 2018 10:13 pm

it is so easy to make a simple typo and trash your system
+1
Within the last month we've had a post where this is exactly what happened
Another posted today!
viewtopic.php?f=47&t=271142#p1481041
Yes--not only a typo, but a lapse of concentration as this member is reporting--I feel his anguish.

That being pointed out though dd is the method I have successfully used many times.
Firstly I make sure I have backups of vital, unique, and irreplaceable files externally.
I use workstations that have easy access to the drive bays and I normally disconnect all but the
source and target drives, as well as all external devices. Then I make doubly, nay--trebly sure the source
and target are correct. Then usually before I initiate the command I check everything once again,
and even after all that there is still anxiety and stress.`
Proud to be a supporter and monthly contributor to Mint.

111MilesToGo
Level 2
Level 2
Posts: 90
Joined: Fri Oct 14, 2016 5:43 am

Re: Cloning a hard disk - In search of best practice

Post by 111MilesToGo » Fri Jun 15, 2018 9:37 am

WharfRat wrote:
Wed Jun 13, 2018 3:54 pm
I'm not sure if dd is the best practice for cloning disks, but I've cloned several disks using it with no problems.
AndyMH wrote:
Thu Jun 14, 2018 1:44 pm
I would avoid dd (disk destroyer), it is so easy to make a simple typo and trash your system.
all41 wrote:
Thu Jun 14, 2018 10:13 pm
it is so easy to make a simple typo and trash your system
+1
Yes, everybody is very anxious to use dd, me too, of course.

What I didn't know yet is: In Germany, we have the big "Linux Handbook" by Kofler. Its 2016 edition states in bold: Use parted only to work on partitions, NOT on the filesystems contained therein. Kofler is referring to the parted manual, saying that filesystem operations are going to be removed from parted in future versions. He recommends to use gdisk or parted to work with partitions, and e.g. mkfs.ext4 and resize2fs to work with filesystems.

Thus I think the need for extra care has been pointed out well enough.

I am currently contemplating to use Clonezilla in partition-to-partition mode to clone single partitions from one SSD to the other, after setting up the GPT on the new SSD via gdisk. I hope to find ways to set all relevant UUIDs and maybe other identifiers on the new disk, partitions and filesystems, but I am not sure yet whether I am aware of all low-level commands necessary to do that. And I am still not knowledgeable enough to know ALL relevant identifiers... ?

When considering to just clone the entire old SSD to the new one via Clonezilla in disk-to-disk mode, I am afraid of two potential issues:
(a) The backup GPT table on the old SSD is located just at its (2 TB) end - would it end up near the 2 TB mark on the new 4 TB SSD?
(b) And how would I have to adjust the ext4 filesystem of my music partition, which on the old SSD is 1.4 TB in size, but will have to be 3.4 TB on the new SSD? After cloning via Clonezilla in disk-to-disk mode, would it be enough to enlarge the music partition, then do a resize2fs?

Elsewhere, I read the advice to format a partition to a certain filesystem type with the "original" operating system, in particular, format to NTFS using Windows. Not sure about the validity as of today, but I might adhere to this anyway.

Well, I will grab older disks of mine and practice the cloning procedure with these first.

Any further recommendations are very welcome. And I hope for answers to the issues raised in this post.


Finally, in a broader context:
Ken Parkes wrote:
Thu Jun 14, 2018 11:44 am
Have you considered setting up a RAID, a natural for three discs like you have. Look at
https://raid.wiki.kernel.org/index.php/RAID_setup for a good guide.
Yes, I had been thinking about a RAID or a NAS. However, these are not the first considerations in my setup. I want my laptop to be transportable, and I want to have everything needed on its one internal disk:
(I) Two Linux systems (one might break in an update...)
(II) My music collection in its entirety, thus the need to swap the internal 2 TB SSD against a 4 TB one
(III) A Windows 8.1 virtual machine, since I cannot get away without Windows yet (CD ripping via dBpoweramp/AccurateRip, audio file tagging via mp3tag, a.o.)
Well, Linux provides marvellous audio playback via MPD/Cantata, doing bit-perfect output to external USB-DACs via ALSA. On the other hand, I do still rely on foobar2000 under Windows for some features that MPD doesn't offer (yet). A virtual Windows machine under VirtualBox still has horrible audible problems besides being me unsure about bit-perfectness, so
(IV) I need an additional real physical Windows machine. That one is installed on an external SSD which I take along and connect only when needed.
Thanks for your tip, though!

User avatar
Flemur
Level 15
Level 15
Posts: 5805
Joined: Mon Aug 20, 2012 9:41 pm
Location: Potemkin Village

Re: Cloning a hard disk - In search of best practice

Post by Flemur » Fri Jun 15, 2018 3:57 pm

111MilesToGo wrote:
Wed Jun 13, 2018 10:26 am
, but such cloning is a major, important and error-prone task...

I have a 2 TB SSD as my main drive in the laptop, and I want to clone it to a 4 TB SSD. The reason is an enormous growth of my music collection due to more and larger hi-rez files, such that one of my partitions needs to grow considerably.
If you're copying music (or other "data", as opposed to an OS with an MBR and such), it makes more sense to simply create the bigger partitions (w/gparted) and just copy the files to it, rather than copy them as part of a partition clone. Simpler, safer.

You can set the new partition UUID to be the same as the old one with

Code: Select all

blkid
to get the old value, and

Code: Select all

tune2fs /dev/{device} -U {uuid}
to set the value in the new unmounted partition. (Watch out for duplicate UUIDs, though, they're one reason why I don't clone partitions any more - ended up with two partitions mounted to the same mount-point ... got outa there ASAP).

My 2 cents...
Please edit your original post title to include [SOLVED] if/when it is solved!
Your data and OS are backed up....right?
Mint 18.3 Xfce/fluxbox/pulse-less
Xubuntu 17.10/fluxbox/pulse-less

User avatar
administrollaattori
Level 13
Level 13
Posts: 4994
Joined: Tue Sep 03, 2013 4:51 am
Location: Finland
Contact:

Re: Cloning a hard disk - In search of best practice

Post by administrollaattori » Fri Jun 15, 2018 4:49 pm

You can clone using Etcher also, if Unsafe mode is enabled.
unsafe-mode.jpg

User avatar
AndyMH
Level 4
Level 4
Posts: 430
Joined: Fri Mar 04, 2016 5:23 pm
Location: Wiltshire

Re: Cloning a hard disk - In search of best practice

Post by AndyMH » Sat Jun 16, 2018 12:20 pm

I used redo to clone my SSD recently (changing from a smaller to bigger SSD in the same laptop). Did a normal backup with redo to an external USB HDD and restored. Straightforward, but I did get caught out by a couple of things:
  • redo didn't copy grub and I was left with an unbootable SSD - had to download and install boot repair - that fixed that problem.
  • the UUID for swap changed, which was giving me long boot times, quick query here and someone pointed me at swap - updated fstab problem solved.
Then used gparted to resize my partitions (root and home) and job done. I left a couple of GB unallocated which seems to be the advice for SSDs.

I think (but don't know) that clonezilla will copy grub and you won't have the problem I did.
Thinkpad T430 i7-3632 Cinnamon 18.3, Thinkpad T420 Cinnamon 18.3, Thinkpad T410 Cinnamon 17.3, Thinkpad T60 19.0 Mate

111MilesToGo
Level 2
Level 2
Posts: 90
Joined: Fri Oct 14, 2016 5:43 am

Re: Cloning a hard disk - In search of best practice

Post by 111MilesToGo » Wed Jun 20, 2018 10:29 am

Thanks to you all for recommendations and calls to caution.

For my GPT-partitioned, EFI/GRUB2-booted SSD, my current procedure is as follows (short summary):

Use gdisk to set up the GPT partitions on the new target disk. As mentioned above, all except my big music partition are to remain identical in size and UUIDs; the music partition is supposed to grow by a lot.

I use Clonezilla in partition-to-partition mode to clone all partitions. This includes the EFI boot partition, excludes the swap partition (of course), and - for my purpose - excludes the music partition. Clonezilla clones the filesystem UUIDs as required.

The reason for my excluding the music partition from Clonezilla is that I want to set its ext4 directory from scratch, i.e. with correct directory size for the larger partition - probably this is overly cautious, Clonezilla calls resize2fs after cloning. I will do a simple file copy or rsync to get all files on the music partition copied over.

Finally, one has to apply the finishing touches to the new target disk: Adjust the /etc/fstab files of the two operating systems to reflect the new UUID of the swap partition; plus the same edit for the /etc/default/grub files in their “resume“ statements, with an update-grub thereafter.

Up to now, no problems encountered. Clonezilla has worked flawlessly for me up to now.

PS: Sure, I am aware of the fact that this cloning process looks simple at hindsight. :wink: But it gave me a bit of anxiety in the beginning, and I think this thread might be useful to some others as well.

gnappi
Level 1
Level 1
Posts: 46
Joined: Fri Aug 31, 2012 1:39 pm

Re: Cloning a hard disk - In search of best practice

Post by gnappi » Wed Jun 20, 2018 11:41 am

For a 100% reliable, off line, inexpensive SATA drive cloner that is very Linux friendly AND an external dual drive E-sata drive appliance the new Sabrent USB 3.0 to SATA External Hard Drive Docking Station is as good as it gets. No scripts, no typing, push a button and voila a perfectly functional clone every time regardless of O/S or data on the drive. .

YMMV, but if a reliable hardware option exists over a moving target piece of software, I'll take hardware every time.

PS, they're ~$22 direct from Sabrent on Ebay. I like free, but not at the expense of trial and error.

User avatar
AZgl1500
Level 7
Level 7
Posts: 1661
Joined: Thu Dec 31, 2015 3:20 am
Location: /Home/Desktop
Contact:

Re: Cloning a hard disk - In search of best practice

Post by AZgl1500 » Wed Jun 20, 2018 9:30 pm

gnappi wrote:
Wed Jun 20, 2018 11:41 am
For a 100% reliable, off line, inexpensive SATA drive cloner that is very Linux friendly AND an external dual drive E-sata drive appliance the new Sabrent USB 3.0 to SATA External Hard Drive Docking Station is as good as it gets. No scripts, no typing, push a button and voila a perfectly functional clone every time regardless of O/S or data on the drive. .

YMMV, but if a reliable hardware option exists over a moving target piece of software, I'll take hardware every time.

PS, they're ~$22 direct from Sabrent on Ebay. I like free, but not at the expense of trial and error.
ummm, I am seeing prices $100 more than that.

did you have a typo?

gnappi
Level 1
Level 1
Posts: 46
Joined: Fri Aug 31, 2012 1:39 pm

Re: Cloning a hard disk - In search of best practice

Post by gnappi » Wed Jun 20, 2018 11:30 pm

No typo. I got mine direct from them at $25.99 delivered. I have no idea where you're looking.
Last edited by gnappi on Wed Jun 20, 2018 11:37 pm, edited 1 time in total.

gnappi
Level 1
Level 1
Posts: 46
Joined: Fri Aug 31, 2012 1:39 pm

Re: Cloning a hard disk - In search of best practice

Post by gnappi » Wed Jun 20, 2018 11:34 pm

The $22 specimen is a single drive model, sorry. The dual without fan is item #282993104637 and $25.99
is

Post Reply

Return to “Hardware Support”