extremely slow file copy to USB

[Drygar]

Linux Mint 17.1 , XFCE, Mint/Ubuntu kernel 3.16

Hi all,

Recently, I came across this issue:
http://superuser.com/questions/424512/w ... -over-time

When i try to copy files to my USB pen drive (e.g. back-up), it takes about 90 minutes to complete - 13 GB of files.

The issue started with a movie 1.2 GB which I wanted to save to the USB pen drive and transfer it to another computer. It copied about 1GB and has 200 MB pending. It looked like it stopped/froze and stayed this way for minutes. When I attempted to stop it, restart machine, etc. the USB drive file system got corrupted, files inside were all in read-only mode. I tried many recommendations over the Internet, nothing helped and I had to re-format the USB pen drive.

Later, my research led me to these:
http://lwn.net/Articles/467328/

https://bugs.launchpad.net/ubuntu/+sour ... bug/500069

https://bugs.launchpad.net/ubuntu/+sour ... ug/1424443
https://bugzilla.kernel.org/show_bug.cgi?id=12309

Please, correct me if I am wrong, but this seems like known issue since 2008 or earlier ?
It says fixed for the Linux Kernel and not fixed for Ubuntu/Fedora.
Can somebody clarify this for me ? Will this be fixed in Ubuntu/Linux Mint ? Is the cause at least known to the Linux developers ?

[Mute Ant]

''it takes about 90 minutes to complete - 13 GB of files'' => 2.4MB/s sustained write => your pendrive is causing this; that's as fast as it can go.

[Drygar]

''it takes about 90 minutes to complete - 13 GB of files'' => 2.4MB/s sustained write => your pendrive is causing this; that's as fast as it can go.

Hi,

Thank you for your reply.

While I appreciate you stepping in, I can't agree with you.

First, my computer is new one (I bought it less than an year ago).
Second, all my 3 pen drivers (2 Transcend and 1 HP) are also new (bought less than an year ago and used just several times).
Third, this is not happening all the time. Most of the times, it is way fast(er).
Fourth, this looks like known issue in the Linux kernel - all web search engines points to many articles (some posted above) - you didn't comment on them.
Fifth, this is not happening on Windows 7 and 8 machines with the same pen drives. On Windows 7 the speed is a lot faster (e.g. 30 MB/s)

The original question is more of this Linux kernel known issues - my pen drive issue is just an example and points how I came across these articles.
Any feedback is welcome !

P.S. I also read on the Net that the freeze might be caused by some issues on file browsers and nautilus seems more stable.
Since I run LM 17.1 XFCE, Nautilus is not included. I downloaded it and have been using it in the past days. Did tests and at least it doesn't freeze (hasn't frozen yet with Nautilus)

[phd21]

Hi Drygar,

There are many factors that influence the performance of any software or hardware, including USB devices in any computer operating system. Need more specific information, see FYI below, like what type of computer you have, the USB devices you are using, etc...

I have an older PC with USB v2.0 and copying 13 gigabytes of data can take awhile. But, it doesn't take that long on my WD Passport USB 1 tb hard drive, or my Patriot USB Flash drives (very fast & sturdy). On some of my cheaper USB flash drives, or slower SD cards in a USB device (they have various specs), it can take an excruciating long time in MS Windows or Linux. Obviously, if you have a computer with USB 3.0 ports, and USB 3.0 devices, things are going to be much quicker. Some computers have the much faster USB 3.0 ports and USB 2.0 ports, and it is up to you to make sure you are using the right USB ports. I have a 3rd party USB add-on device in one computer that constantly gave me problems in MS Windows; when I finally had the right drivers installed, it would work, then an update would come along and change it back to USB v1. Yet, on Linux Mint it has always worked, go figure?

Sometimes a USB extender cable can be bad or the wrong kind for that device too.

Another problem can be the use of "passive USB hubs" which are USB devices, (4-port, or more), that do not have their own power supply. This can cause all kinds of issues, including intermittent problems which can drive you crazy. I had problems with my printer, webcam, USB flash drives, when using a "passive 4-port hub" that all went away when I quit using the "Passive USB Hub" and plugged my USB devices directly into the computers USB ports. I'm pretty sure that self powered USB hubs would eliminate those problems (USB hubs with their own power supply). These USB hubs also come in USB 3.0 or USB V2.0 as well.

Every new edition of Linux Mint makes improvements over their previous editions, so quoting posts or articles that are years old and speculative in some cases won't help you resolve any issues on current versions and editions.

I am continuously amazed in a good way at how wonderful these free Linux Mint operating systems are, how they are maintained, this superb forum, and most of the helpful people in this forum and other Linux forums. Of course you have to put a little time and effort in choosing and using the correct version of Linux Mint for your system's hardware and that you like the most. If you don't do that, then you might have some issues to work out. Not that there is anything wrong with Linux Mint Xfce, but this version is known to be really good for people using limited hardware. If you have a newer computer system why not create live DVD's or USB flash drives of Cinnamon, Mate, & KDE (my fav), 64-bit, and try them out; you might find that one of these editions works way better for your newer system.

FYI: It would help everyone here to have more information about your system's setup: like what edition and version of Linux Mint are you using, Cinnamon, Mate, KDE (my fav), or Xfce; version 17 or 17.1, or What?; 32-bit or 64-bit? Some more information about your hardware would be useful as well. You can get this information from top of your system menu's Welcome screen, System Information program(s), and the best is by typing in "inxi -Fxz" from a console terminal prompt, which you can then copy and paste back here. This information helps anyone here in this forum to be able to answer any questions you may have better , thanks. You can even add this information (abbreviated) to your signature using the control panel link in the above left of this forum screen; then when you post a question or reply to one, that info will automatically be there, ex: see mine and others below their posts.

[Mute Ant]

Disks accessory will do a proper Read/Write Benchmark test. I get...

• Partition 1 of 16GB SanDisk Cruzer Edge [1.27] [/dev/sdb1]
  Sample Size 10.0MiB
  Average Read Rate 24.2 MB/s
  Average Write Rate 7.5 MB/s

Using the same system as SD cards, it's easily a Class 6 and nearly a Class 8.

''The issue started with a movie 1.2 GB which I wanted to save to the USB pen drive''
Using a simulation of your original problem file, 1.2GB of random bytes, my machine takes 190 seconds to complete a drag-and-drop into that stick. Mean sustained write rate 6.3MB/s.

''The original question is more of this Linux kernel known issues'' Linux will change to correct demonstrated defects...you prove the kernel is defective, they will change it.

[Drygar]

Hi all,

Thanks for the reply. Sorry for the delay.

Just to notify you that the issue is somewhat [resolved] improved. Nautius doesn't give any issue. The file copy speed is still slow (but faster) and so far no freezing with nautilus. I am considering transferring to MATE

[Drygar]

''it takes about 90 minutes to complete - 13 GB of files'' => 2.4MB/s sustained write => your pendrive is causing this; that's as fast as it can go.

Hi,

Just to update you. It's not my pendrive nor my PC - I just reimaged the machine and been testing with Ubuntu15, Ubuntu GNOME 15, Xubuntu 14 and LM Mate 17.2 - no more such an issue. On LM it is still a bit slower than *buntus but way faster than what it was some days ago [when the thread was posted by me]. E.g. copying 20 GB takes less than 2 minutes. Definitely, this was a software issue.

Thanks everyone !

[Mute Ant]

''E.g. copying 20 GB takes less than 2 minutes.'' Oh look! A flying pig!

[Drygar_2016]

System: Host: Toshiba Kernel: 4.2.0-34-generic x86_64 (64 bit, gcc: 4.8.2)
Desktop: Xfce 4.12.2 (Gtk 2.24.23) Distro: Linux Mint 17.2 Rafaela
Machine: System: TOSHIBA product: SATELLITE C50-A
Mobo: TOSHIBA model: Portable PC version: MP Bios: INSYDE version: 1.10 date: 12/03/2013
CPU: Dual core Intel Celeron CPU N2820 (-MCP-) cache: 1024 KB flags: (lm nx sse sse2 sse3 sse4_1 sse4_2 ssse3 vmx) bmips: 8524.8
Clock Speeds: 1: 1424.310 MHz 2: 591.268 MHz
Graphics: Card: Intel Atom Processor Z36xxx/Z37xxx Series Graphics & Display bus-ID: 00:02.0
X.Org: 1.15.1 drivers: intel (unloaded: fbdev,vesa) Resolution: 1366x768@60.0hz
GLX Renderer: Mesa DRI Intel Bay Trail GLX Version: 3.0 Mesa 10.1.3 Direct Rendering: Yes
Audio: Card: Intel Atom Processor Z36xxx/Z37xxx Series High Definition Audio Controller
driver: snd_hda_intel bus-ID: 00:1b.0
Sound: Advanced Linux Sound Architecture ver: k4.2.0-34-generic
Network: Card-1: Qualcomm Atheros QCA9565 / AR9565 Wireless Network Adapter driver: ath9k bus-ID: 01:00.0
IF: wlan0 state: down mac: <filter>
Card-2: Realtek RTL8101/2/6E PCI Express Fast/Gigabit Ethernet controller
driver: r8169 ver: 2.3LK-NAPI port: 1000 bus-ID: 02:00.0
IF: eth0 state: up speed: 100 Mbps duplex: full mac: <filter>
Drives: HDD Total Size: 182.0GB (32.0% used)
1: id: /dev/sda model: Samsung_SSD_850 size: 120.0GB
2: USB id: /dev/sdb model: TF10 size: 61.9GB
Partition: ID: / size: 106G used: 55G (55%) fs: ext4
RAID: No RAID devices detected - /proc/mdstat and md_mod kernel raid module present
Sensors: System Temperatures: cpu: 48.0C mobo: N/A
Fan Speeds (in rpm): cpu: N/A
Info: Processes: 171 Uptime: 11 min Memory: 569.7/3835.9MB Runlevel: 2 Gcc sys: 4.8.4
Client: Shell (bash 4.3.11) inxi: 1.9.17

[Drygar_2016]

Just trying to copy files to external USB to make archive.

It is awful and it seems that in order to back up about 30 GB of data I will need to wait for about 65 hours.

Copying to external USBs no matter the device is with terrible speed on my LM.
Copyting to te very same USBs on Windows 7 SP1 and Windows 8.1 works perfectly fine.

Tried the default way - just plugin the device - 32 hours to complete 30 GB data. USB3 port used.
Tried with sudo mount -o remount,sync /dev/sdb1 - 60 to 65 hours to complete 30 GB of data.

Same problem was kernel 3.13, 3.16, now 4.2 - LM 17, 17.1, now with 17.2
I really can't back up data in case of some failure.

[ALbin75]

I see that this post is still not solved. And it is the only one I can find on the subject.

It's difficult to find any other threads about this subjects since the search engine ignores words with less than 4 characters.
USB 3.0 has to be written USB3 to crate a 4 character word and thus this is the only post I find here.
Slow USB will generate 18.755 hits and they are all excluding USB since it's only 3 characters.

I'm sure there is a way around how to search for slow usb, but so far Your post is the only one I find on the subject. of slow USB 3.0
Or generally slow USB.

[cameraman]

Hi

I've had this problem for a while, it's a bit better at the moment for example yesterday I was transferring about 10Gb's of videos to my USB 3.0 HD , it transfers like 1.5 Gb then it takes pause for a few min and so on.

The same thing is happening when using USB 2.0 devices.

Running LM 18 64-bit, Linux Kernel 4.4.0 and this laptop is only a few months old
Toshiba Satellite S55T-C5164-4K with a 512 SSD

[maks1m]

The same issue on my Linux Mint 18.1
It's a shame to have such bug since 2008.

[Jarew]

Have the same problem with slow transfers to network and usb drives. Transfer fly with windows so don't tell me it's the drives, etc.. How about Mint telling the truth and admitting it's the operating system.

[Citizen229]

The work around for searching the forums for usb is to use google. Make your search terms in quotes. "Linux mint forums" " USB 3.0" Do that all of that in the search bar and it will search the forums. The quotes make it search for exact words only. Its a work around for forums with strict search parameters.

As far as the USB. 17.3 works excellent for me. You could be right for 18.X. But remember 18.X is not a near final product until 18.3. I have a 4 port hdd hub that ports through usb 3.0. If I remember right t did 30 gigs to the 5400 rpmlaptop drive in about an hour and 120 gigs in 45 mins to the ssd that is in there. Source, connection, secondary host, destination. the lowest variable decides the speed limit. for me it is SSD>USB 3.0> USB3.0 hub>5400rpm/SSD.

Dont forget USB ( does it still?) used to split it bandwidth from the devices plugged in. so a usb transfer rate from connection of 10 megs/sec with 4 devices plugged in is now slowed to 2.5megs/sec. I do not think this applies to thing plugged into source, but things such as hubs or things which have storage volume. It was many many moons ago I read a benchmark article on this topic. I believe it was prelaunch/launch testing of usb 2.0. Thats how long ago.

[McSk]

Same Problem here: while copying 2 (two) GB Data to my USB 3 stick (SanDisk Extreme with up to 190 MB/s write speed...) I managed to find this post, read it through, register in this forum and write this answer.
The speed dropped to 2.0 MB/s...

Now trying 660 MB: 1hour 38 minutes remaining...

Has anyone a hint on how I can debug this?

Thanks in advance,
Sven

[phd21]

Hi "Drygar", & Anyone Else,

1.) Do not use Passive USB hubs (4-port or 8-port USB hubs) without their own power supply as they do not work well, or at all, or worse they work intermittently.

2.) When copying large files, or a lot of files, some USB devices can go into a power saving mode or sleep mode because there is no keyboard activity. Linux users can change settings to prevent that.

For example: Grub boot parameter option to disable (turn off) auto suspend of USB devices

Code: Select all

'usbcore.autosuspend=-1'

Another post with good information
viewtopic.php?f=47&t=240569&hilit=usb+grub

3.) Some computer's bios settings can also affect performance, so turn off power saving or sleep modes for hard drives and with USB features.

4.) Sometimes the USB cable is bad or not working well, so try another USB cable.

5.) I have found that typical file operations, like copying or moving files, that the Linux "ext4" file system worked much faster than "NTFS" or "Fat32" file systems.


Hope this helps ...

[Volunteer4F]

The only trick I found that really works:
https://unix.stackexchange.com/question ... 00-or-near

If you want to try some power-user tricks, you can reduce the size of the buffer that Linux uses by setting /proc/sys/vm/dirty_bytes to something like 15728640 (15 MB). This means the application can't get more than 15MB ahead of its actual progress.

A side effect is that your computer might have lower data-writing throughput with this setting, but on the whole, I find it helpful to see that a program is running a long time while it writes lots of data vs. the confusion of having a program appear to be done with its job but the system lagging badly as the kernel does the actual work. Setting dirty_bytes to a reasonably small value can also help prevent your system from becoming unresponsive when you're low on free memory and run a program that suddenly writes lots of data.

But, don't set it too small! I use 15MB as a rough estimate that the kernel can flush the buffer to a normal hard drive in 1/4 of a second or less. It keeps my system from feeling "laggy".

[69Rixter]

Evening:

I believe I should be posting here, rather than creating a new thread, but I'll leave that determination to admin. I too have 5 drives that just recently acquired the "slow writing deficiency" All the drives were 'writing" at minimum 6.5+Mb/s. 2 drives are 2.0, 3 drives 3.0. No drive is writing faster than 4.0Mb/s and the 2.0 drives are 1.8-2.1Mb/s. In LM 18.3 the drives do write at normal/factory speeds, but do so only in short bursts, then lag/stall, write again-lag/stall. In Win7, drives write steadier, but at much lower rates. The "read" speeds all are consistent with what they were originally (around 18Mb/s and faster) The oldest of these drives is merely 2 y/o. And don't suggest warranty replacement, been there, was denied. Drives are just barely 'out-of-warranty". Anyways, I don't think the drives are corrupt. They DO still write and read, just something has caused a major slowdown of the 'write" process. I've done malware tests, chkdsk, gparted, and fdsik, (to take advantage of those programs 'tools"), wiped/rebuilt partition, formatted several times to both FAT32 and NTFS and tests for bad sectors. Every test has returned that the drives are 'healthy"/no malware/virus; no bad/dead sectors! There have been no updates to the settings or security of the PC...as best I can determine. OK, if there is any more information you may wish to have, please request it. Here's what I want to accomplish; #1. Identify and correct whatever caused these drives to slow #2. Restore these drives to "original" operation parameters.
Thanx:

Rick

[rickNS]

Just my observation take it for what it is worth.
It seems that sticks take smaller amounts of data fast enough, but as the amount of data grows larger, the speed gets slower...and slower the larger it is. Any truth in that ?
When transferring multi GB's of data, like doing backups, a usb stick is just the wrong device (unless you don't have a choice). Using a usb HD you can copy about 2 GBs a minute.