[Solved] Copy one disk to another

[bigal]

I have the task of making a copy of one old HDD to another. The original is not S.M.A.R.T. enabled and is 80GB in size. As it is formatted as an MBR disk it has a block size of 512 MB and is a bootable disk. The recipient disk is the larger of the two and was totally blank but I have made a 80 GB partition at the start to receive the copy. When completed, this too, must be bootable.

After looking around it seems to me that ‘dd’ is the tool to use but would appreciate some clarification on some of the switches. The command I have found and am looking at is this

Code: Select all

$ sudo dd if=/dev/zero of /dev/sdd bs=2048 count=32 && sync

but think it needs some modification for my purposes.

Clarification please on the following points.

Code: Select all

if=/dev/zero

I am assuming that is the first partition on the donor drive. If is not, can

Code: Select all

if/dev/sdb (or c or whatever)

be substituted?

Code: Select all

of /dev/sdb

is, I understand, the second partition the recipient drive. How does dd know that this is a partition on a separate drive?

Code: Select all

bs=2048

As both drives are MBR drives should the block count be 512 as opposed to 2048?

I have no idea and cannot seem to find the purpose of

Code: Select all

count=32

I have also seen

Code: Select all

count=1

being suggested for the same purpose. What is the ‘count’ for and which is correct please?

Finally

Code: Select all

&& sync

It seems that while running the dd command some of the data is written to a cache and the the addition of sync is to ensure that that cache is emptied prior to the command completion. Is this a correct reading of the phrase?

[rene]

1. /dev/zero is not a partition, it is a "fake" device that simply returns all-zero bytes when read; an unending source of zero's. It is used when you for example want to overwrite a partition with zero's; not when copying anything, such as in your case.

2. You have wrt. spelling several variants of if/of listed. The only correct syntax for either is, e.g., if=/dev/sdx1, of=/dev/sdy2.

3. /dev/sdx1 for whatever values of "x" and "1" is the device identifier for a partition. You are going to need to identify your source and destination partitions; the output of sudo fdisk -l could be helpful. Be careful: dd simply writes bytes from source (the "if" device) to destination (the "of" device): if you get the destination wrong you overwrite the wrong partition, or "disk" even if you do not specify a partition number such as in your examples.

4. "bs=N" tells dd to read/write N bytes at a time and defaults to a mere 512. Reading only 512 bytes at a time is slow and you'd a such specify e.g. "bs=1M" to read 1 megabyte at a time -- but the parameter makes no functional difference and other than for said speed issue you can leave it out. It has no relation to hardware block sizes.

5. "count=M" tells dd to read/write M bs-sized chunks total and you'd in your case leave out fully: you simply want to read all of the source device, not just a part.

6. The "&& sync" is not part of the dd command; the "&&" has the shell execute the latter part iff the the former exited indicating success. That latter part, "sync", indeed flushes (OS-, software-) caches.

In this situation you'd ideally also make sure that the destination partition is the exact same size as the source: you're transferring a filesystem that expects to be on a partition of that size. If the destination is bigger you'd grow the filesystem to once again fill the partittion afterwards; if it's smaller, dd will not work (cleanly) at all.

As to having the destination disk be bootable: on an MBR system the GRUB bootloader is located in the space preceding the first partition of the boot drive. Depending on what the goal of the destination drive is -- is it going to fully replace the source? -- you could simply "dd" that over as well but you'd in that case need to provide some more information. The above mentioned output of sudo fdisk -l could be enough.

[bigal]

Thank you rene.

Forgive me for attempting to clarify and understand your comments. I have never used dd before and want to fully understand the commands so I can be sure of it doing exactly what I want it to do.

1. /dev/zero is not a partition, it is a "fake" device that simply returns all-zero bytes when read; an unending source of zero's. It is used when you for example want to overwrite a partition with zero's; not when copying anything, such as in your case.

As I emphatically do not want to overwrite any of the data on the source disk how do I instruct dd how to copy as opposed to overwrite?

2. You have wrt. spelling several variants of if/of listed. The only correct syntax for either is, e.g., if=/dev/sdx1, of=/dev/sdy2.

Thank you for pointing that out. The incorrect entries were typos. Seeing your post I realised I had not copied the code correctly when typing it out.

3. /dev/sdx1 for whatever values of "x" and "1" is the device identifier for a partition. You are going to need to identify your source and destination partitions; the output of sudo fdisk -l could be helpful. Be careful: dd simply writes bytes from source (the "if" device) to destination (the "of" device): if you get the destination wrong you overwrite the wrong partition, or "disk" even if you do not specify a partition number such as in your examples.

Neither the source or destination disks are in any machine at present. I will have to physically install them in a spare machine to get the output of $ sudo fdisk -l and report back. I certainly do not intend running dd until I am 100% sure of getting it all correct.

4. "bs=N" tells dd to read/write N bytes at a time and defaults to a mere 512. Reading only 512 bytes at a time is slow and you'd a such specify e.g. "bs=1M" to read 1 megabyte at a time -- but the parameter makes no functional difference and other than for said speed issue you can leave it out. It has no relation to hardware block sizes.

It goes to shew just how wrong it is to assume. I was fully convinced that bs stood for block size. Thank you for correcting me.

5. "count=M" tells dd to read/write M bs-sized chunks total and you'd in your case leave out fully: you simply want to read all of the source device, not just a part.

Again, thank you for the illumination.

The "&& sync" is not part of the dd command; the "&&" has the shell execute the latter part if the the former exited indicating success. That latter part, "sync", indeed flushes (OS-, software-) caches.

It is good to know that I was (nearly) correct about something.

In this situation you'd ideally also make sure that the destination partition is the exact same size as the source: you're transferring a filesystem that expects to be on a partition of that size. If the destination is bigger you'd grow the filesystem to once again fill the partition afterwards; if it's smaller, dd will not work (cleanly) at all.

As to having the destination disk be bootable: on an MBR system the GRUB bootloader is located in the space preceding the first partition of the boot drive. Depending on what the goal of the destination drive is -- is it going to fully replace the source? -- you could simply "dd" that over as well but you'd in that case need to provide some more information. The above mentioned output of sudo fdisk -l could be enough.

I had added the disk to a spare machine, ran Gparted from the os that lives there to create the partition. Once done I removed the disk again. Once both the source and destination disks are added to a machine I can re-run Gparted to ensure that the destination partition is the same size as the original disk. However, if there is an error and the destination partition ends up being slightly larger than the source am I correct in saying that this is of less importance than the reverse? The rest of the destination drive will never be used so in that sense the size of the destination is of no concern.

Yes, the intended destination disk will be a replacement. I will go away and connect up the two drives and report the output of $sudo fdisk -l.

bigal

[rene]

As I emphatically do not want to overwrite any of the data on the source disk how do I instruct dd how to copy as opposed to overwrite?

An answer to this part while waiting for the fdisk output...

The most important thing to realise about dd is that it is an extremely simple tool: all it does is read bytes from its "if" device and write them to its "of" device. With a few optional parameters to be able to specify how many bytes to read/write, and for example to skip reading the first bit of the if device and/or start writing at some given offset into the of device, but still just "read from if, write to of". As such the answer to your question is "specify the correct if and of devices".

On current Linux the /dev/sda, /dev/sdb and so on devices represent actual disks, actual hardware devices. Partitions on a disk are more software than hardware but represented similarly by say /dev/sda1, /dev/sda2, /dev/sdb5 and so on. You will just need to identify the source and destination partitions and specify them with if= and of= respectively.

And yes, the destination being larger than the source is not a problem if you don't actually care for the remainder of the destination drive. You may in fact in that case be able to copy things at disk level rather than partition level which would copy GRUB right alongside (but fdisk will probably whine about geometry differences when doing that). Let's wait to see the partition tables (i.e., fdisk -l output).

[bigal]

I'm sorry about the delay in responding. Just to clarify, my test machine does not normally have HDDs installed. On this occasion only the two drive we are discussing were connected. It was booted using a 32 bit Mint 18.2 DVD. In the results below /dev/sdb is the donor drive (I have deleted the partition) and /dev/sda will be the recipient.

Here are the results you requested:

Code: Select all

$ sudo fdisk -l
Disk /dev/loop0: 1.5 GiB, 1620574208 bytes, 3165184 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/sda: 149 GiB, 160000000000 bytes, 312500000 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0xc9089d2d


Disk /dev/sdb: 74.5 GiB, 80026361856 bytes, 156301488 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x00011c34

Device     Boot  Start       End   Sectors  Size Id Type
/dev/sdb1  *        63    208844    208782  102M 83 Linux
/dev/sdb2       208845 156296384 156087540 74.4G 8e Linux LVM


Disk /dev/mapper/VolGroup00-LogVol00: 73.4 GiB, 78819360768 bytes, 153944064 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/mapper/VolGroup00-LogVol01: 1 GiB, 1073741824 bytes, 2097152 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

bigal

[rene]

LVM makes growing the current 80G partition to the 160G of the destination drive after copying less straightforward -- even if only because I personally wouldn't know how to best/cleanly do that, having never bothered with LVM -- but let us in fact decide to not care and simply copy over all of sdb, partition table, bootloader and LVM structures included.

Having booted from the 18.2 DVD the partitions on /dev/sdb will not have been automatically mounted but let us make sure they are not anyway: sudo eject sdb. This will error on sdb not in fact being ejectable ("inappropriate ioctl") but that's fine: mounted partitions on the disk will still be unmounted. Then just copy over the whole shebang:

Code: Select all

dd if=/dev/sdb of=/dev/sda bs=1M status=progress

The "status=progress" provides for something to look at while it's chugging away; the "bs=1M" should be enough to allow for maximum throughput. If sdb booted that machine when booted from disk when it was sda there seems to be no reason the machine won't boot from sda after the copy.

[EDIT] No reason indeed, but I do expect you should disconnect sdb before rebooting: LVM otherwise probably stumbles over duplicate volumes. Also wondering upon rereading if it wouildn't have been simpler to simply install a new Mint system on the 160G drive and copy over anything you want preserved from sdb after doing so and being booted into that new system -- but oh well.

[bigal]

Ahh well. All that work and I rather spoilt the happy ending. Despite checking and double checking I got the two disks mucked up. I never was much good at proof reading.

Thank you so much for your help rene. At least I have learnt something of the power of dd. I'm just off to hide in a corner somewhere.

bigal

[rene]

[bigal]

I reversed the drive letters.

[rene]

Yes, gathered as much. That particular smilie is described as "confused" here on the forum -- but I always find it to be more "sadly resigned".

If you near-immediately hit Ctrl-C when copying, if /dev/sdb2 is in fact still intact, you'll be able to salvage it. I sort of doubt that though...

[bigal]

Unfortunately, dd ran its full course. It stopped with the massage saying that there was insufficient space; I saw that and instantly realised my error.

Between posts I had noticed that the PATA DVD drive cable was in the primary socket in the motherboard while the HDD cable was in the secondary, so, as the machine was powered off at the time, I corrected them. The HDD that was to receive the copy was on a SATA connection. Swapping the PATA cables reversed the order that the two HDDs were seen by the machine hence my error. My checking and double checking prior to saying 'Go' was to ensure I had transcribed your code correctly. I had. What I failed to check was a fresh fdisk output; I should have done that as I had moved connections around. So easy to be wise after the event. It is totally down to me that I killed my disk.

Ah well.

bigal

[Pierre]

here is another thought:
you were using Gparted to resize those two disk - - so you can also use Gparted to copy those partitions, too.
- R/click on source partition - copy
- R/click on vacant area on destination disk - paste.
then click on apply.

the normal error checking of Gparted will still apply, so that is another safety check.
- you can't get the drives mixes up, so easily, either.

[bigal]

I didn't know that you could do that. Thank you.

[rene]

Not fully sure you'd in fact be able to do that in the case of LVM: there's potential for needing the outer LVM container at the exact same sector values so as to not upset values for the actual volume(s). Admittedly, inside things are probably defined relative to start so likely in fact okay, but not knowing any technical details of LVM together with the fact that GRUB was asked to be installed again as well anyway just moving over everything was a conscious choice.

The safe choice. Given no replugging of cables halfway through instruction and execution, of course....

[bigal]

Mine will now be a totally fresh installation so the gparted method is of interest only. As I said, I didn’t know that you could do that. I have only ever used it for creating, resizing or removing partitions. I do not believe that it can be used to create a LVM setup. I would only use it set up the partitions if need be. I would normally set the entire LVM, partitions included, from within the installer.

With my limited understanding of LVM, and specifically LVM2 I believe that grub only gets installed on one of the two volumes and has to be added to the second volume manually following installation. I am not sure how this would work if ever a disk has to be swapped out, hot or cold. Presumably it would have to added afresh once again.

We are getting a long way off topic here, albeit an interesting conversation.