Write errors with large data transfer to USB SSD drive

Forum rules
Before you post read how to get help. Topics in this forum are automatically closed 6 months after creation.
Locked
nagromnewo
Level 1
Level 1
Posts: 4
Joined: Mon May 31, 2021 11:02 pm

Write errors with large data transfer to USB SSD drive

Post by nagromnewo »

Hi

I have an old USB NTFS 2 Tb drive with about 1.7 Tb of data on it. I have checked the drive and it comes up with no errors. I am trying to transfer about 1 Tb of this data to a newer 2 Tb USB exFAT SSD drive. Most of the files are large, as in several hundred Mb each, mostly video in .AVI format.
I have tried dragging and dropping as well as rsync and grsync. It copies for a while, and then stops with an error message regarding the destination. When this happens, I am unable to read the destination drive and the only way to get it back is to disconnect and reconnect it.
Since I suspected that my old laptop (with only USB2) might be struggling to read and write all this data to two USB drives at the same time, I also tried to grsync about 150Gb to internal storage and then grsync from there to the new USB SSD drive. Grsyncing to the internal drive went fine, Grsyncing from there to the new USB SSD failed with the same error message as before.
It seems this proves that the issue is with the USB SSD, so the question is: Is it faulty or is there something I can do to control the process more tightly and prevent this error from occurring? Scanning the (more or less empty) disk did not show any errors.
I'm not at this computer at the moment, but can get the actual error messages tomorrow and check Mint version etc. if that helps.

Owen
Last edited by LockBot on Wed Dec 28, 2022 7:16 am, edited 1 time in total.
Reason: Topic automatically closed 6 months after creation. New replies are no longer allowed.
User avatar
Lady Fitzgerald
Level 15
Level 15
Posts: 5744
Joined: Tue Jan 07, 2020 3:12 pm
Location: AZ, SSA (Squabbling States of America)

Re: Write errors with large data transfer to USB SSD drive

Post by Lady Fitzgerald »

I use FreeFileSync (FFS) for onetime, massive file transfers (although I usually use it for making and updating backups). For some odd reason, using copy and paste (I almost never use cut and paste) for large file transfers, either just really large files or just a whole bunch of them, often gets snarled up. I don't understand why but FFS handles the transfers much better and if something goes wonky with a file, it will report it at the end of the transfer.

It requires a hack to enable it but FFS can be set to verify file transfers. Verifying doubles copy time but pretty much ensures all transfers go well. For the rare file that doesn't transfer well, FFS lists it at the end of the process so you know which one was the problem (usually, it was file that was corrupted). I rarely have a problem, though.
Jeannie

To ensure the safety of your data, you have to be proactive, not reactive, so, back it up!
nagromnewo
Level 1
Level 1
Posts: 4
Joined: Mon May 31, 2021 11:02 pm

Re: Write errors with large data transfer to USB SSD drive

Post by nagromnewo »

Thanks, I will try FreeFileSync and let you know how it goes. I actually don't care all that much how long it takes (within reason) as long as the process completes without errors or interaction.
I have forgotten a whole lot about Linux, since this computer has been working reliably without much need for change for about 5 years... :D

Owen
Moonstone Man
Level 16
Level 16
Posts: 6078
Joined: Mon Aug 27, 2012 10:17 pm

Re: Write errors with large data transfer to USB SSD drive

Post by Moonstone Man »

nagromnewo wrote: Tue Jun 01, 2021 1:06 am Thanks, I will try FreeFileSync and let you know how it goes.
You say the files are large, but provide no examples. How big are the largest files? And 'an error message regarding the destination' is meaningless. What is the exact error message?
nagromnewo
Level 1
Level 1
Posts: 4
Joined: Mon May 31, 2021 11:02 pm

Re: Write errors with large data transfer to USB SSD drive

Post by nagromnewo »

As I said, most of the files are videos in .AVI format and several hundred Mb in size, around 700 Mb is typical, but they all vary.
Copying stops with an error writing to the USB SSD, and I have to unplug and reconnect it to be able to access it.
I also said I can provide more details tomorrow. I'm at work right now, the computer is at home.

I was hoping someone might a have encountered a similar problem and could suggest a solution. I will try FreeFileSync and if that doesn't work, I'll provide more details tomorrow.

Owen
Moonstone Man
Level 16
Level 16
Posts: 6078
Joined: Mon Aug 27, 2012 10:17 pm

Re: Write errors with large data transfer to USB SSD drive

Post by Moonstone Man »

nagromnewo wrote: Tue Jun 01, 2021 3:09 am I also said I can provide more details tomorrow.
That has absolutely no bearing on anyone asking you pertinent questions.
rene
Level 20
Level 20
Posts: 12240
Joined: Sun Mar 27, 2016 6:58 pm

Re: Write errors with large data transfer to USB SSD drive

Post by rene »

nagromnewo wrote: Tue Jun 01, 2021 3:09 am I was hoping someone might a have encountered a similar problem and could suggest a solution.
Biggest chance will be formatting the drive with something other than exFAT. NTFS if access needed from Windows, ext4 if Linux only.
nagromnewo
Level 1
Level 1
Posts: 4
Joined: Mon May 31, 2021 11:02 pm

Re: Write errors with large data transfer to USB SSD drive

Post by nagromnewo »

rene wrote: Tue Jun 01, 2021 5:15 am Biggest chance will be formatting the drive with something other than exFAT. NTFS if access needed from Windows, ext4 if Linux only.
I tried to format the drive NTFS. The format completed in less than a minute, and now it is not recognized by the system at all. When I plug it in, nothing happens.

Owen
rene
Level 20
Level 20
Posts: 12240
Joined: Sun Mar 27, 2016 6:58 pm

Re: Write errors with large data transfer to USB SSD drive

Post by rene »

That it completed in less than a minute is expected but it not being recognized after of course not. Thing I can sort of imagine is that you pulled it after formatting without waiting for it to sync or that you did something odd as to formatting it itself. I.e, how did you format?

Don't panic in any case because that'll not be hard to fix. I tend to use command line tools, but the program "Disks" might be a better option here. When you start it (from the menus) with your drive plugged in, then select your drive in the left-hand sidebar and the relevant partition (its likely only partition) in the right-hand side display of that drive, you can under that click the "gears" icon and choose to "Format partition..." and then select NTFS.

It'll probably automount after that, but if not, and although I'd rather doubt that "Disks" wouldn't sync automatically, for good measure... under that right-hand side partition display in "Disks" it is also mentioned which "Device" the partition is; something like /dev/sdb1. Close "Disks" and then from a terminal say sudo eject sdb (i.e., not sdb1) disconnect the drive and reconnect. All well?

If not, can you post a screenshot of the "Disks" display for your drive, if any?
rene
Level 20
Level 20
Posts: 12240
Joined: Sun Mar 27, 2016 6:58 pm

Re: Write errors with large data transfer to USB SSD drive

Post by rene »

Oh, by the way, assuming that you picked NTFS because you want to share with Windows, i.e., that you have Windows, formating it from Windows as NTFS is of course also possible.
Petermint
Level 9
Level 9
Posts: 2981
Joined: Tue Feb 16, 2016 3:12 am

Re: Write errors with large data transfer to USB SSD drive

Post by Petermint »

Are you using USB 3? Those symptoms occur with some USB 3 chips in the external disks and they occur no matter which file system you choose. You can find the USB-SATA chip id from command lsusb.

You can also have problems when the SSDs heat up. 70 or 80 degrees is their limit. The Disks application can show their temperature if the devices have proper status reporting. You can also place an egg on the SSD and stop copying if the egg is cooking. :D

The same device might work in a USB 2 port with the copy running overnight.
Locked

Return to “Storage”