[Solved, Partially] Moving data from Windows to Linux

All Gurus once were Newbies
Forum rules
There are no such things as "stupid" questions. However if you think your question is a bit stupid, then this is the right place for you to post it. Please stick to easy to-the-point questions that you feel people can answer fast. For long and complicated questions prefer the other forums within the support section.
Before you post please read how to get help
Netherprovinc3
Level 1
Level 1
Posts: 49
Joined: Mon Feb 04, 2019 9:29 pm

[Solved, Partially] Moving data from Windows to Linux

Post by Netherprovinc3 » Sat Jul 13, 2019 7:08 pm

I installed Linux Mint 19.1 Cinnamon recently. But, I still haven't completed the project of moving all of my files over from Windows.

I would also like to validate that the files, once moved, are 99.999% likely the same (that nothing got mis-copied). In other words, I would like to use something like MD5 or sha256sum to check.

At first, I thought that I would copy the files using a USB drive, but there seems to be some difficulty/ confusion on my part in checking that the files got copied properly.
I was following the chosen answer here:
https://stackoverflow.com/questions/855 ... nux-shares
But, when I ran this command in Powershell (on my Windows 8.1 computer):

Code: Select all

 find /source -type f -exec md5sum {} \; > MD5SUM 
instead of the word "source" I put the name of the folder which contains the files that I wanted to get the MD5 sums for.
the Windows computer does not execute the command and instead says that
"the term '>' is not recognized as the name of a cmdlet, function, script file, or operable program..."

So now I am thinking that I should just hook my Windows hard drive up as an external hard drive to my Linux computer and then I can run this command

Code: Select all

 diff -q -r /path/to/old/drive/ /path/to/new/drive/ 
Source:
https://unix.stackexchange.com/question ... to-another

But, I am reading that when you hook up the Windows OS installed hard drive as an external drive, you have to click "ok" to the user access control prompt and so maybe the whole plan only works when you are hooking up a drive to a Windows machine. I'd imagine that the data on a windows OS operating system partition has some blocking of directly accessing all of the data on it.

Edit: maybe I should use rsync, which is mentioned in the 2nd link in my post, the second answer posted.
Edit2: actually rsync seems to not verify the data after it was written to the disc, it's more to check that the data came through the network transfer ok.
Last edited by Netherprovinc3 on Tue Jul 16, 2019 6:07 pm, edited 6 times in total.

User avatar
Larry78723
Level 5
Level 5
Posts: 727
Joined: Wed Jan 09, 2019 7:01 pm
Location: Jasper County, SC, USA

Re: Moving data from Windows to Linux

Post by Larry78723 » Sat Jul 13, 2019 7:13 pm

Is the machine you want to move the files to a laptop or desktop? If it's a desktop and you have an available drive bay, you can use the original hard drive in it. Linux can read Windows drives.
Image

Netherprovinc3
Level 1
Level 1
Posts: 49
Joined: Mon Feb 04, 2019 9:29 pm

Re: Moving data from Windows to Linux

Post by Netherprovinc3 » Sat Jul 13, 2019 7:15 pm

Larry78723 wrote:
Sat Jul 13, 2019 7:13 pm
Is the machine you want to move the files to a laptop or desktop? If it's a desktop and you have an available drive bay, you can use the original hard drive in it. Linux can read Windows drives.
It's a desktop computer.

User avatar
Larry78723
Level 5
Level 5
Posts: 727
Joined: Wed Jan 09, 2019 7:01 pm
Location: Jasper County, SC, USA

Re: Moving data from Windows to Linux

Post by Larry78723 » Sat Jul 13, 2019 7:23 pm

OK, so let's make sure I understand what you're trying to do. From what I'm understanding, you have a desktop machine that has a HD with Mint installed. You have a HD from a windows machine that you want to use the files from windows in Mint. Is my understanding correct?
Image

Netherprovinc3
Level 1
Level 1
Posts: 49
Joined: Mon Feb 04, 2019 9:29 pm

Re: Moving data from Windows to Linux

Post by Netherprovinc3 » Sat Jul 13, 2019 7:30 pm

Larry78723 wrote:
Sat Jul 13, 2019 7:23 pm
OK, so let's make sure I understand what you're trying to do. From what I'm understanding, you have a desktop machine that has a HD with Mint installed. You have a HD from a windows machine that you want to use the files from windows in Mint. Is my understanding correct?
Yes, that's correct. And I want to have the files on my Linux Mint computer only.

To add some detail, I want to move about 90% of the files that are on my Windows computer to my Linux Mint computer.

User avatar
Larry78723
Level 5
Level 5
Posts: 727
Joined: Wed Jan 09, 2019 7:01 pm
Location: Jasper County, SC, USA

Re: Moving data from Windows to Linux

Post by Larry78723 » Sat Jul 13, 2019 7:41 pm

Got it! Can you install the windows HD in the desktop? If yes, we can set it up so that Mint can read the windows HD directly. That will eliminate the need to transfer the files entirely.
Image

Netherprovinc3
Level 1
Level 1
Posts: 49
Joined: Mon Feb 04, 2019 9:29 pm

Re: Moving data from Windows to Linux

Post by Netherprovinc3 » Sat Jul 13, 2019 8:04 pm

Larry78723 wrote:
Sat Jul 13, 2019 7:41 pm
Got it! Can you install the windows HD in the desktop? If yes, we can set it up so that Mint can read the windows HD directly. That will eliminate the need to transfer the files entirely.
No, because I still want to use the Windows computer to run some programs that are not available on Linux.
We can temporarily move it there as an external USB drive, and then move it back after the copy.

User avatar
Larry78723
Level 5
Level 5
Posts: 727
Joined: Wed Jan 09, 2019 7:01 pm
Location: Jasper County, SC, USA

Re: Moving data from Windows to Linux

Post by Larry78723 » Sat Jul 13, 2019 8:31 pm

I can't find any info on transferring files and doing validation too. Sorry I couldn't be more help.
Image

athi
Level 5
Level 5
Posts: 576
Joined: Sun Mar 30, 2014 10:15 am
Location: USA

Re: Moving data from Windows to Linux

Post by athi » Sat Jul 13, 2019 8:50 pm

Found this posting on how to use diff command to recursively compare files in 2 different directories, might work for what you want to do.
https://superuser.com/questions/473510/ ... n-on-linux
Mint Mate 19.1. Main rig is HP 800G2 I5 6500 16GB ram, 120GB boot drive, 2x3TB, 1x4TB data drives. Oldest rig is Mate 18.3 on Dell D620 with 32bits core2duo.

User avatar
lsemmens
Level 8
Level 8
Posts: 2161
Joined: Wed Sep 10, 2014 9:07 pm
Location: Rural South Australia

Re: Moving data from Windows to Linux

Post by lsemmens » Sat Jul 13, 2019 11:06 pm

Are both computers connected to the same internet connection (i.e. modem)? You can set up file sharing so that you can then share files across the network. That both machines will always have access to the "latest and greatest" version of whichever file you are working on.
Kernel: 4.15.0-46-generic x86_64 bits
Desktop: Cinnamon 3.8.9
Distro: Linux Mint 19 Tara

Laptop HP-ProBook-470-G2 8Gb RAM SSD
Server AMD Phenom 9650 - GEForce 9400GT 6Gb RAM
+ three other Mint machines
Out of my mind - please leave a message

Netherprovinc3
Level 1
Level 1
Posts: 49
Joined: Mon Feb 04, 2019 9:29 pm

Re: Moving data from Windows to Linux

Post by Netherprovinc3 » Sun Jul 14, 2019 8:46 am

lsemmens wrote:
Sat Jul 13, 2019 11:06 pm
Are both computers connected to the same internet connection (i.e. modem)? You can set up file sharing so that you can then share files across the network. That both machines will always have access to the "latest and greatest" version of whichever file you are working on.
I am going to take one of the computers offline in the near future so I actually don't need to share the files back and forth. Just need to do a 1 time move.

User avatar
AndyMH
Level 8
Level 8
Posts: 2163
Joined: Fri Mar 04, 2016 5:23 pm
Location: Wiltshire

Re: Moving data from Windows to Linux

Post by AndyMH » Sun Jul 14, 2019 11:06 am

I'd simply connect the windows drive via a usb adaptor and copy the files over, either in the file manager or manually via a terminal. I wouldn't bother with validation, I've done this before and not had any problems.

You could use rsync to do the copy, I'll let you find out how - man rsync, but there is a --checksum option (expect this to slow down the copy significantly) and a dry-run option so you can see what it would do before doing it for real.
Homebrew i5-8400+GTX1080 Cinnamon 19.0, 3 x Thinkpad T430 Cinnamon 19.0, i7-3632 , i5-3320, i5-3210, Thinkpad T60 19.0 Mate

Netherprovinc3
Level 1
Level 1
Posts: 49
Joined: Mon Feb 04, 2019 9:29 pm

Re: Moving data from Windows to Linux

Post by Netherprovinc3 » Sun Jul 14, 2019 12:26 pm

AndyMH wrote:
Sun Jul 14, 2019 11:06 am
I'd simply connect the windows drive via a usb adaptor and copy the files over, either in the file manager or manually via a terminal. I wouldn't bother with validation, I've done this before and not had any problems.

You could use rsync to do the copy, I'll let you find out how - man rsync, but there is a --checksum option (expect this to slow down the copy significantly) and a dry-run option so you can see what it would do before doing it for real.
rsync seems to rely on some other mechanism to give an error if the write process produces an error.
https://unix.stackexchange.com/question ... cal-drives
But maybe rsync with some flags does the trick.
diff seems like the best to verify that the copied data is the same as the source, based on what I have read (some of what I was reading was old, though, circa 2011).

Netherprovinc3
Level 1
Level 1
Posts: 49
Joined: Mon Feb 04, 2019 9:29 pm

Re: Moving data from Windows to Linux

Post by Netherprovinc3 » Sun Jul 14, 2019 12:37 pm

I got the drive hooked up as an external drive and I copied the files over.

However, when I run the command

Code: Select all

 
diff -q -r /media/joe2/29824168924121AA/Users/joe1/My Documents/File history/ /home/joe2/File History/
terminal says

Code: Select all

diff: extra operand 'history/'
diff: Try 'diff --help' for more information.

athi
Level 5
Level 5
Posts: 576
Joined: Sun Mar 30, 2014 10:15 am
Location: USA

Re: Moving data from Windows to Linux

Post by athi » Sun Jul 14, 2019 1:06 pm

Maybe the space in the name of location is the problem, I noticed that some commands do not like spaces or special characters.
Mint Mate 19.1. Main rig is HP 800G2 I5 6500 16GB ram, 120GB boot drive, 2x3TB, 1x4TB data drives. Oldest rig is Mate 18.3 on Dell D620 with 32bits core2duo.

User avatar
phd21
Level 18
Level 18
Posts: 8813
Joined: Thu Jan 09, 2014 9:42 pm
Location: Florida

Re: Moving data from Windows to Linux

Post by phd21 » Sun Jul 14, 2019 1:33 pm

Hi Netherprovinc3,

I just read your post and the good replies to it. Here are my thoughts on this as well.

I would recommend that you first check your MS Windows files with an antivirus anti-malware software before copying them to your new Linux Mint system if you have not already done that.

Once you have the MS Windows drive attached to the Linux computer and mounted which Linux should be able to read and write from (You can open your file manager and just click the drive), then you can also use "FreeFileSync" and or "LuckyBackup" to copy your files.

I think FreeFileSync is available as a Flatpak from the software manager, but I usually just download their Linux archive file, extract it, copy the folder to my home "applications" folder and run it from there.

Download the Latest Version - FreeFileSync
https://freefilesync.org/download.php

Verification of the copied files using a checksum? - FreeFileSync Forum
https://freefilesync.org/forum/viewtopic.php?t=1495


Also, You can create a checksum file of the files you want to copy from the MS Windows drive and then after copying on Linux drive to make sure they match.

MS Windows

How to Download and Install Microsoft's FCIV Tool (if it is not already installed)
https://www.lifewire.com/how-to-downloa ... iv-2625185

File Checksum Integrity Verifier (FCIV) Examples - Wikibooks, open books for an open world
https://en.wikibooks.org/wiki/File_Chec ... )_Examples

How to use the Microsoft FCIV command-line checksum tool
https://www.computerweekly.com/tip/How- ... cksum-tool

In Linux
Example: using the command below I created a "pdf-checksum-256.list" file containing all my PDF documents checksum values (sha256) in my PDF folder. (tilda "~" = /home/YourUserName/)

Code: Select all

find ~/PDF -type f -exec sha256sum {} \; > ~/PDF/pdf-checksum-256.list
or even better use the appopriate checksum extension which some apps will auto-recognize.

Code: Select all

find ~/PDF -type f -exec sha256sum {} \; > ~/PDF/pdf-checksum-256.sha256

To verify the files I ran this command.

Code: Select all

sha256sum -c ~/PDF/pdf-checksum-256.list
or

Code: Select all

sha256sum -c ~/PDF/pdf-checksum-256.sha256
command line - Create checksum sha256 of all files and directories? - Ask Ubuntu
https://askubuntu.com/questions/1091335 ... irectories


Tip: in the sample commands below, you can replace "md5" with "sha256sum" for creating or reading sha256 checksum values.

How to Generate md5 checksum for all Files in a Directory - TecAdmin
https://tecadmin.net/generate-md5-check ... directory/

bash - md5 all files in a directory tree - Stack Overflow
https://stackoverflow.com/questions/369 ... ctory-tree

Learn Linux Tricks to Generate Checksums and Verify Data Integrity
https://linoxide.com/linux-how-to/linux ... checksums/

Hope this helps ...
Last edited by phd21 on Sun Jul 14, 2019 4:55 pm, edited 3 times in total.
Phd21: Mint KDE 18.3 & 19, 64-bit Awesome OS, Ancient Dell OptiPlex 780 Core2Duo E8400 3GHz,4gb Ram,256gb SDD, Video: Intel 4 Graphics, DVD Lightscribe. Why I use KDE?:https://opensource.com/life/15/4/9-reasons-to-use-kde

Netherprovinc3
Level 1
Level 1
Posts: 49
Joined: Mon Feb 04, 2019 9:29 pm

Re: Moving data from Windows to Linux

Post by Netherprovinc3 » Sun Jul 14, 2019 3:07 pm

phd21 wrote:
Sun Jul 14, 2019 1:33 pm
I think FreeFileSync is available as a Flatpak from the software manager, but I usually just download their Linux archive file, extract it, copy the folder to my home "applications" folder and run it from there.

Download the Latest Version - FreeFileSync
https://freefilesync.org/download.php
Seems like a program that would be useful to me for routine backup tasks. I do see on the list of features that it does some error detection. Also, you can then change the (last) expert setting so that the program does binary compare on the files. I am not totally clear what binary compare is but I don't know if binaries are that useful for say, text files.
https://freefilesync.org/manual.php?top ... t-settings
The information on their website links to a Microsoft page that goes into some detail about CopyFile but I'm not sure that's relevant when running the software on Linux.

phd21 wrote:
Sun Jul 14, 2019 1:33 pm
Also, I think you can create a checksum file of the files you want to copy from the MS Windows drive and then after copying on Linux drive to make sure they match.
I have tried but I don't have much of a clue what the syntax means when reading online. When I tried to run something very similar, mentioned in the OP, I get an error message.

Code: Select all

 find /source -type f -exec md5sum {} \; > MD5SUM 
Possible problems:
maybe I should not have run this on my Windows machine (booted in Windows).

User avatar
phd21
Level 18
Level 18
Posts: 8813
Joined: Thu Jan 09, 2014 9:42 pm
Location: Florida

Re: Moving data from Windows to Linux

Post by phd21 » Sun Jul 14, 2019 3:31 pm

Hi "Netherprovinc3",

You are welcome...

I just updated my last reply, so please re-read it.
Netherprovinc3 wrote:Seems like a program that would be useful to me for routine backup tasks. I do see on the list of features that it does some error detection.
FreeFileSync (FFS) is a great application that I have been using for many years in MS Windows and now in Linux as a backup tool, and FFS can also be used to copy files as well. The "failsafe" option makes sure the copy operations were accurate and successful.

Although I make drive and partition image backups using Clonezilla Live (monthly or more), and Aptik backups for Linux customizations (after original Linux setup the way I want and after adding a lot of new software), I use FFS more often because after the first run it only backs up new and or changed files, so it is very quick for frequent backups to my external USB drive and they are uncompressed copies of the folders and files making them easily accessible from the USB drives. I also think it is a good idea to make long-term more permanent backups of important files, folders, data, multimedia stuff to DVD or Blu-Ray discs that cannot be erased.
Netherprovinc3 wrote:Also, you can then change the (last) expert setting so that the program does binary compare on the files. I am not totally clear what binary compare is but I don't know if binaries are that useful for say, text files.
It will not hurt you to experiment with various FFS settings, just make sure you have a backup of the original files first.

As for the checksum verification commands from the articles I linked, those are Linux commands. I did, however, update my last reply with MS Windows articles on creating and verifying checksum values. Note: Upper and lower case characters matter in Linux.

Hope this helps ...
Last edited by phd21 on Mon Jul 15, 2019 3:13 pm, edited 2 times in total.
Phd21: Mint KDE 18.3 & 19, 64-bit Awesome OS, Ancient Dell OptiPlex 780 Core2Duo E8400 3GHz,4gb Ram,256gb SDD, Video: Intel 4 Graphics, DVD Lightscribe. Why I use KDE?:https://opensource.com/life/15/4/9-reasons-to-use-kde

User avatar
Flemur
Level 16
Level 16
Posts: 6935
Joined: Mon Aug 20, 2012 9:41 pm
Location: Potemkin Village

Re: Moving data from Windows to Linux

Post by Flemur » Sun Jul 14, 2019 7:50 pm

Netherprovinc3 wrote:
Sun Jul 14, 2019 12:37 pm

Code: Select all

 
diff -q -r /media/joe2/29824168924121AA/Users/joe1/My Documents/File history/ /home/joe2/File History/
diff: extra operand 'history/'
You need quotes " or backslash on filenames with spaces. It's best to not use spaces if you can avoid it.

Code: Select all

diff -q -r "/media/joe2/29824168924121AA/Users/joe1/My Documents/File history/" "/home/joe2/File History/"
Please edit your original post title to include [SOLVED] if/when it is solved!
Your data and OS are backed up....right?
Mint 19.1 Xfce/fluxbox
Manjaro openbox/fluxbox

Netherprovinc3
Level 1
Level 1
Posts: 49
Joined: Mon Feb 04, 2019 9:29 pm

Re: Moving data from Windows to Linux

Post by Netherprovinc3 » Mon Jul 15, 2019 11:46 pm

phd21 wrote:
Sun Jul 14, 2019 3:31 pm
It will not hurt you to experiment with various FFS settings, just make sure you have a backup of the original files first.
Even with experimenting, it's tough to know what FreeFileSync is doing behind the scenes. 99.99%+ of the time it's going to say that the check passed but we are not going to be able to really see what it did or didn't do. We might need to look at the source code to get the answer.

You make good points on backing up files, I need to work some of that into my routine. Especially the part about DVDs and Blu-Ray. People forget about the advantages of that.

Post Reply

Return to “Newbie Questions”