Copy files to USB problem?

All Gurus once were Newbies
Forum rules
There are no such things as "stupid" questions. However if you think your question is a bit stupid, then this is the right place for you to post it. Please stick to easy to-the-point questions that you feel people can answer fast. For long and complicated questions prefer the other forums within the support section.
Before you post please read how to get help
Locked
ugljesap
Level 1
Level 1
Posts: 13
Joined: Mon May 27, 2013 8:27 am

Copy files to USB problem?

Post by ugljesap » Sun Jul 07, 2013 11:09 am

I have a strange problem when i copy files to usb. Today, i copied several .rxx (winrar part files) to my kingston USB drive. All the files were succesfully copied, but when i arrived to a friends place, it showed only first file, and the rest of the drive was empty. So i chose a different USB stick, unrared all the files (it is single ISO file), it was also copied succesfully, but when i plugged it back, it was only 1.5gb instead of a 6.5gb. It also happens on my USB hard drives. It is really frustrating when you try to copy large amount of data, progress bar goes to 100%, and it lasts as it's supposed to, about 20-30 minutes for larger files, just to realize nothing has been copied.... And i noticed that while copying, mint makes a smaller "copy" of that file in my home folder, but lot,lot smaller in size. Any ideas?

chiefjim
Level 5
Level 5
Posts: 851
Joined: Sun Jun 07, 2009 7:26 am
Location: South Texas, USA

Re: Copy files to USB problem?

Post by chiefjim » Sun Jul 07, 2013 11:18 am

First thing I would do is check the permissions. Right click the drive and check properties.

Secondly was the drive formatted to Ext2,3, or 4?. If so and your friend is using Windows it may be only that MS will not recognize the device properly. Any time I need to move files to or from an MS device I always pick NTFS or FAT32 to assure compatibility.
Desktop:
Mint-18.3 Mate 64 bit
MSI B85M-E45
Intel G3258
Crucial Ballistic Sport 8GB DDR3 1600
2TB HDD WD Black 7200rpm

Laptop:
Acer Aspire AS5552-5898
Mint-17.3 Mate 64 bit
AMD Phenom II Triple-Core N830
4GB Memory DDR3
500GB HDD 5400rpm

ugljesap
Level 1
Level 1
Posts: 13
Joined: Mon May 27, 2013 8:27 am

Re: Copy files to USB problem?

Post by ugljesap » Sun Jul 07, 2013 11:26 am

I formatted it to NTFS using gparted. The problem is that when i replug that USB device into my computer, Mint LMDE, a moment after it finished copying it still happens, so it's not Windows issue. Same happens when i plug it to a laptop running Xubuntu, so it's not a drive issue nad file system issue, i suppose.

And fir the permissions, my user name is the owner...

User avatar
WharfRat
Level 21
Level 21
Posts: 13142
Joined: Thu Apr 07, 2011 8:15 pm

Re: Copy files to USB problem?

Post by WharfRat » Sun Jul 07, 2013 11:52 am

ugljesap wrote:I formatted it to NTFS using gparted. The problem is that when i replug that USB device into my computer, Mint LMDE, a moment after it finished copying it still happens, so it's not Windows issue. Same happens when i plug it to a laptop running Xubuntu, so it's not a drive issue nad file system issue, i suppose.

And fir the permissions, my user name is the owner...
To help pin-down what the problem is, try formatting and copying from the cl. Open a terminal and enter the following bold lines one at a time. Change ?? and yourname to the device's partition letter number e.g /sdc1. sdd1 ect. and yourname to your user account name.

sudo -i

Check if that device is mounted with mount and if it is unmount it with umount /dev/sd??
mount

Use this to locate the device if it wasn't discovered with mount
fdisk -l

Then make a filesystem (be carefull here :wink:
mkdosfs -F 32 /dev/sd??

mkdir /media/usbdrive

chown yourname:yourname /media/usbdrive

mount /dev/sd?? /media/usbdrive

cd /media/usbdrive

cp -a /path/to/file/file.iso . or to copy multiple files cp /path/to/files/* .

cd ~

sync

umount /media/usbdrive


Then unplug it and plug it back in to check the file(s).
ImageImage

ugljesap
Level 1
Level 1
Posts: 13
Joined: Mon May 27, 2013 8:27 am

Re: Copy files to USB problem?

Post by ugljesap » Sun Jul 07, 2013 12:56 pm

Ok, i did that and all went ok. But, i did not mount my USB to /media/usbdrive/ because it allready mounts itself to /media/*drive uuid*. And, i tried to bind it's uuid to a certain folder using fstab and it says "only root has permissions blah blah".
My point is, i will never be sure that my files are copied correctly, and when you are short of time, it's really annoying! Why would such a simple and essential operation be so complicated and bothersome in mint, i've never experienced this problem in any other distro. And, i have a fstab auto mounted partition named "games" in /media/games, and whenever i run gparted, it makes another entry in my file manager (caja) that can't be opened, mounted or anything, and when i hover it says "Mount and open Games". Other mounted partitions show only their path "/media/***/" when i hover over them. I really like mint, but these simple things not working properly are making me crazy...

User avatar
Spearmint2
Level 16
Level 16
Posts: 6770
Joined: Sat May 04, 2013 1:41 pm
Location: Maryland, USA

Re: Copy files to USB problem?

Post by Spearmint2 » Sun Jul 07, 2013 2:25 pm

Format the problematic USB thumbdrive to FAT32 using Windows instead of Linux and see if the same problem persists.
All things go better with Mint. Mint julep, mint jelly, mint gum, candy mints, pillow mints, peppermint, chocolate mints, spearmint,....

ugljesap
Level 1
Level 1
Posts: 13
Joined: Mon May 27, 2013 8:27 am

Re: Copy files to USB problem?

Post by ugljesap » Sun Jul 07, 2013 2:36 pm

Well, it happens on ALL of my USB removable devices. I have 4 usb thumbdrives (kingston, patriot, apacer and silicon power) and 2 external usb hdd drives. 2 thumbdrives are FAT32, and all the others are NTFS. Since this problem started, it happened on all the drives, no matter what file system it is. It really bothers me that i have no errors on those drives, in xubuntu they work like a charm, and working flawlessly in win7. I am just confused because everything seems to be ok, it mounts properly, it displays copying progress bar, it even takes the same amount of time to copy the data like other OS-es do. An after that, it decides not to copy some files, or to copy them partially. I really hate to reboot my system to win7 just to copy some files on flash drive.

User avatar
Spearmint2
Level 16
Level 16
Posts: 6770
Joined: Sat May 04, 2013 1:41 pm
Location: Maryland, USA

Re: Copy files to USB problem?

Post by Spearmint2 » Sun Jul 07, 2013 2:58 pm

ugljesap wrote:Well, it happens on ALL of my USB removable devices. I have 4 usb thumbdrives (kingston, patriot, apacer and silicon power) and 2 external usb hdd drives. 2 thumbdrives are FAT32, and all the others are NTFS. Since this problem started, it happened on all the drives, no matter what file system it is. It really bothers me that i have no errors on those drives, in xubuntu they work like a charm, and working flawlessly in win7. I am just confused because everything seems to be ok, it mounts properly, it displays copying progress bar, it even takes the same amount of time to copy the data like other OS-es do. An after that, it decides not to copy some files, or to copy them partially. I really hate to reboot my system to win7 just to copy some files on flash drive.
Interesting. I had that happen on ftp file transfers, but not on USB thumbdrives. On FTP using caja many times I'd hilight several files, but only the first would copy over, the others were there as empty files on the server. Happens with either cut/paste or using drag/drop in double pane view. When doing FTP I just do one file at a time. Probably should have mentioned it before. I know no fix for it.
All things go better with Mint. Mint julep, mint jelly, mint gum, candy mints, pillow mints, peppermint, chocolate mints, spearmint,....

ugljesap
Level 1
Level 1
Posts: 13
Joined: Mon May 27, 2013 8:27 am

Re: Copy files to USB problem?

Post by ugljesap » Sun Jul 07, 2013 3:21 pm

I just did a small test. I took two thumbdrives and performed the same thing on both devices. One is NTFS, the other one is FAT32. I choose Warcraft 3 folder cause it has around 700 files and a few of them are above 200mb in size. So i copied it first on NTFS drive, once it finished, i went on properties and it shows correct folder size and number of files. I replugged the device, properties, bump, half the size it was and 300-400 files less. I did the same with FAT32 drive, and the same thing. Is it possible that i maybe didn't properly auto mount my ntfs partitions in fstab, could that be the problem?

User avatar
WharfRat
Level 21
Level 21
Posts: 13142
Joined: Thu Apr 07, 2011 8:15 pm

Re: Copy files to USB problem?

Post by WharfRat » Sun Jul 07, 2013 3:34 pm

ugljesap,

Are you umounting the device before pulling them out of the usb port :?:
ImageImage

User avatar
Spearmint2
Level 16
Level 16
Posts: 6770
Joined: Sat May 04, 2013 1:41 pm
Location: Maryland, USA

Re: Copy files to USB problem?

Post by Spearmint2 » Sun Jul 07, 2013 3:51 pm

ugljesap wrote:I just did a small test. I took two thumbdrives and performed the same thing on both devices. One is NTFS, the other one is FAT32. I choose Warcraft 3 folder cause it has around 700 files and a few of them are above 200mb in size. So i copied it first on NTFS drive, once it finished, i went on properties and it shows correct folder size and number of files. I replugged the device, properties, bump, half the size it was and 300-400 files less. I did the same with FAT32 drive, and the same thing. Is it possible that i maybe didn't properly auto mount my ntfs partitions in fstab, could that be the problem?
sounds like a write-cache problem for external devices. I went looking and ran across this.

http://community.linuxmint.com/idea/view/885

how to turn it off.
http://www.linuxquestions.org/questions/debian-26/how-can-i-permanently-turn-off-%27write-cache%27-debian-424136/

Or google more suggestions on disabling write-cache for USB devices.
All things go better with Mint. Mint julep, mint jelly, mint gum, candy mints, pillow mints, peppermint, chocolate mints, spearmint,....

ugljesap
Level 1
Level 1
Posts: 13
Joined: Mon May 27, 2013 8:27 am

Re: Copy files to USB problem?

Post by ugljesap » Sun Jul 07, 2013 4:40 pm

neither unmount nor safely remove hardware work, i mean they do, but they return some error. i will write the correct error when i get home.

User avatar
MartyMint
Level 5
Level 5
Posts: 953
Joined: Thu Dec 27, 2012 10:50 pm

Re: Copy files to USB problem?

Post by MartyMint » Sun Jul 07, 2013 4:58 pm

ugljesap wrote:neither unmount nor safely remove hardware work, i mean they do, but they return some error. i will write the correct error when i get home.

Could be that you're not waiting long enough for the data to finish being written.

ugljesap
Level 1
Level 1
Posts: 13
Joined: Mon May 27, 2013 8:27 am

Re: Copy files to USB problem?

Post by ugljesap » Sun Jul 07, 2013 6:35 pm

MartyMint wrote:
ugljesap wrote:neither unmount nor safely remove hardware work, i mean they do, but they return some error. i will write the correct error when i get home.

Could be that you're not waiting long enough for the data to finish being written.
I see that now. If i type sync after the copying is completed, i have to wait a while for it to get done, depending on how much data i've copied. This is REALLY a pain in the *ss. And it takes a few minutes to sync 16gb of data that i just copied. This one fills the cup...

User avatar
WharfRat
Level 21
Level 21
Posts: 13142
Joined: Thu Apr 07, 2011 8:15 pm

Re: Copy files to USB problem?

Post by WharfRat » Sun Jul 07, 2013 6:42 pm

ugljesap wrote:I see that now. If i type sync after the copying is completed, i have to wait a while for it to get done, depending on how much data i've copied. This is REALLY a pain in the *ss. And it takes a few minutes to sync 16gb of data that i just copied. This one fills the cup...
Don't trust those progress dialogs, it seems they account for the buffer too meaning when no more data in being copied to the buffer it may report done.

The slowness is the transfer rate of the pendrive.
ImageImage

ugljesap
Level 1
Level 1
Posts: 13
Joined: Mon May 27, 2013 8:27 am

Re: Copy files to USB problem?

Post by ugljesap » Mon Jul 08, 2013 4:52 am

I know that pendrives are slow. I've spent quite a lot time to figure out how to turn off write cache for removable devices, and to make it persistant , cause you can turn it of from terminal, but it will be restored after next reboot. I hope somewhere i will find a guide that explains it.

User avatar
Reorx
Level 11
Level 11
Posts: 3931
Joined: Tue Jul 07, 2009 7:14 pm
Location: SE Florida, USA

Re: Copy files to USB problem?

Post by Reorx » Mon Jul 08, 2013 11:47 am

USB flash drive tips/tricks >>>

1) For any given transfer size - it is always better to transfer a small number of large files than a large number of small files. If your data consists of a larger number of small files - zip the data down to a single (or small number of files) first and then do the transfer - it'll go MUCH faster this way.

2) I believe "safely remove" or "unmount" will cause the write buffer to be flushed but how long it takes or when it is finished might not be obvious... On my Mint 13 machines, when the buffered write is finished, the desktop icon for the flash dirve disappears (mounted drives appear on the desktop automatically).

3) Getting a flash drive with a Drive Activity LED might be helpful.

4) Consider a USB 3 flash drive if your machine has a USB 3 port. USB 3 is approximately 10 times faster than USB 2.
Last edited by Reorx on Mon Oct 23, 2017 9:09 pm, edited 1 time in total.
Full time Linux Mint user since 2011 - Currently running LM19 Cinnamon.

Image Image

ugljesap
Level 1
Level 1
Posts: 13
Joined: Mon May 27, 2013 8:27 am

Re: Copy files to USB problem?

Post by ugljesap » Tue Jul 09, 2013 5:12 am

Reorx wrote:USB flash drive tips/tricks >>>

1) For any given transfer size - it is always better to transfer a small number of large files than a large number of small files. If you data consists of a larger number of small files - zip the data down to a single (or small number of files) first and then do the transfer - it'll go MUCH faster this way.

2) I believe "safely remove" or "unmount" will cause the write buffer to be flushed but how long it takes or when it is finished might not be obvious... On my Mint 13 machines, when the buffered write is finished, the desktop icon for the flash dirve disappears (mounted drives appear on the desktop automatically).

3) Getting a flash drive with a Drive Activity LED might be helpful.

4) Consider a USB 3 flash drive if your machine has a USB 3 port. USB 3 is approximately 10 times faster than USB 2.
Please don't get me wrong, but your suggestions seem as they were pre recorded on some technical support answering machine :) I am just kidding around.

1. your suggestion to zip/rar the files is not an option. Wait for it to compress them, wait for them to copy the files, and wait for it to decompress.... thats about 3 times longer, it might be safer but again....
2. safely remove and eject can not be completed untill write cache is flushed, it just comes up with an error
3. well, it might be helpful but it's not, cause flash drives can blink without a reason, system probing it and so on. And someone mentioned here that LED activity means nothing.
4. USB 3.0? Gladly! But that would mean new mobo, cpu, ram, and a USB 3.0 pendrive, those are pretty expensive here.

I was not trying to be rude or anything, i just wanted to point out that answers like that really don't help anyone, cause that's the stuff you can read on your computer's package.

Thank you all once again, at least i know what my problem is

Volunteer4F
Level 1
Level 1
Posts: 11
Joined: Wed Dec 07, 2011 3:48 am

Re: Copy files to USB problem?

Post by Volunteer4F » Mon Oct 23, 2017 6:51 pm

The only trick I found that really works:
https://unix.stackexchange.com/question ... 00-or-near

If you want to try some power-user tricks, you can reduce the size of the buffer that Linux uses by setting /proc/sys/vm/dirty_bytes to something like 15728640 (15 MB). This means the application can't get more than 15MB ahead of its actual progress.

A side effect is that your computer might have lower data-writing throughput with this setting, but on the whole, I find it helpful to see that a program is running a long time while it writes lots of data vs. the confusion of having a program appear to be done with its job but the system lagging badly as the kernel does the actual work. Setting dirty_bytes to a reasonably small value can also help prevent your system from becoming unresponsive when you're low on free memory and run a program that suddenly writes lots of data.

But, don't set it too small! I use 15MB as a rough estimate that the kernel can flush the buffer to a normal hard drive in 1/4 of a second or less. It keeps my system from feeling "laggy".

Locked

Return to “Newbie Questions”