Too fast uploading content on a flash drive

Questions about applications and software
Forum rules
Before you post please read how to get help
Post Reply
User avatar
rado84
Level 3
Level 3
Posts: 153
Joined: Sun Aug 19, 2012 9:31 am
Contact:

Too fast uploading content on a flash drive

Post by rado84 » Thu Jul 12, 2018 10:41 am

I noticed this problem in 18.3 and it seems it's still present in 19: when I format a USB flash drive and then I try to upload anything on it (even a simple docx file), the upload speed is 155 MB/second (the normal speed of this FDD is 10 MB tops). If the file is, for instance, 700 MB, uploading reaches 80% instantly and then hangs on this percentage forever.
At first I thought it was the flash drive problem, so I diagnosed it with different software, I even bought a new one and it was still the same - instant upload to a specific percentage and then hanging on that forever. Which is why I can't upload a Linux live ISO on a flash drive doing that in Linux env. I tried that with Mint, Antergos, Debian and even Fedora - in all distroes the upload speed is extremenly high.
I tried the same in Windows and all uploading processes happened normally, including uploading the ISO on a flash drive.

So, before I report this as a bug in Mint, I'd like to know has someone else ever had this problem and how did you fix it?

ajgringo619
Level 6
Level 6
Posts: 1386
Joined: Thu Mar 01, 2018 8:36 pm
Location: San Diego, California

Re: Too fast uploading content on a flash drive

Post by ajgringo619 » Thu Jul 12, 2018 10:52 am

How did you format the drive? What do you mean by "uploading an .ISO"?
Last edited by ajgringo619 on Thu Jul 12, 2018 11:10 am, edited 1 time in total.
Mint 19.2 XFCE
kernel 4.15.0-64-generic
AMD FX-8350 Eight-Core w/16 GB RAM
Nvidia GeForce 960 w/2GB RAM (435.21)
Image

User avatar
rado84
Level 3
Level 3
Posts: 153
Joined: Sun Aug 19, 2012 9:31 am
Contact:

Re: Too fast uploading content on a flash drive

Post by rado84 » Thu Jul 12, 2018 10:55 am

With the integrated formatting tool which allows only FAT32 and no other options.

gm10
Level 18
Level 18
Posts: 8726
Joined: Thu Jun 21, 2018 5:11 pm

Re: Too fast uploading content on a flash drive

Post by gm10 » Thu Jul 12, 2018 10:59 am

This sounds like write caching. What does sudo hdparm -W /dev/sdb say? Replace /dev/sdb with wherever your actual USB stick is.
Tune up your LM 19.x: ppa:gm10/linuxmint-tools

schi2m
Level 1
Level 1
Posts: 7
Joined: Tue Jul 10, 2018 5:11 am

Re: Too fast uploading content on a flash drive

Post by schi2m » Thu Jul 12, 2018 11:14 am

I have the same problem.

Code: Select all

> sudo hdparm -W /dev/sdb1

/dev/sdb1:
SG_IO: bad/missing sense data, sb[]:  70 00 05 00 00 00 00 0a 00 00 00 00 20 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 write-caching = not supported

User avatar
rado84
Level 3
Level 3
Posts: 153
Joined: Sun Aug 19, 2012 9:31 am
Contact:

Re: Too fast uploading content on a flash drive

Post by rado84 » Thu Jul 12, 2018 11:18 am

Code: Select all

rado@Mint19:~$ sudo hdparm -W /dev/sdd1
[sudo] password for rado: 

/dev/sdd1:
SG_IO: bad/missing sense data, sb[]:  70 00 05 00 00 00 00 0a 00 00 00 00 20 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 write-caching = not supported
rado@Mint19:~$ 

gm10
Level 18
Level 18
Posts: 8726
Joined: Thu Jun 21, 2018 5:11 pm

Re: Too fast uploading content on a flash drive

Post by gm10 » Thu Jul 12, 2018 11:53 am

So not write-caching, or at least none configured that way. Was the only idea I had, sorry. Can't reproduce it, either. Files copy with a steady speed to my USB sticks.
Tune up your LM 19.x: ppa:gm10/linuxmint-tools

ajgringo619
Level 6
Level 6
Posts: 1386
Joined: Thu Mar 01, 2018 8:36 pm
Location: San Diego, California

Re: Too fast uploading content on a flash drive

Post by ajgringo619 » Thu Jul 12, 2018 12:15 pm

I just ran a test, using unison to sync my critical /home files to a USB3 stick that was formatted in FAT32. While the data seems to be copying OK, I'm getting tons of timeouts (although no errors that I can see). I've never noticed timeouts when using my USB sticks - slowdowns yes, but never complete stoppages. I've created USB images from ISOs that have all worked as advertised.

Hopefully someone knows of a fix, or at least a workaround. Maybe this is a FAT32 issue; I'll try the same test with an EXT4 USB stick and see what happens.
Mint 19.2 XFCE
kernel 4.15.0-64-generic
AMD FX-8350 Eight-Core w/16 GB RAM
Nvidia GeForce 960 w/2GB RAM (435.21)
Image

gm10
Level 18
Level 18
Posts: 8726
Joined: Thu Jun 21, 2018 5:11 pm

Re: Too fast uploading content on a flash drive

Post by gm10 » Thu Jul 12, 2018 12:18 pm

ajgringo619 wrote:
Thu Jul 12, 2018 12:15 pm
Maybe this is a FAT32 issue;
Doubt it. I had tried to repro with a FAT32 formatted stick. No issues whatsoever.
Tune up your LM 19.x: ppa:gm10/linuxmint-tools

ajgringo619
Level 6
Level 6
Posts: 1386
Joined: Thu Mar 01, 2018 8:36 pm
Location: San Diego, California

Re: Too fast uploading content on a flash drive

Post by ajgringo619 » Thu Jul 12, 2018 12:22 pm

gm10 wrote:
Thu Jul 12, 2018 12:18 pm
ajgringo619 wrote:
Thu Jul 12, 2018 12:15 pm
Maybe this is a FAT32 issue;
Doubt it. I had tried to repro with a FAT32 formatted stick. No issues whatsoever.
Thanks for the quick reply. Just grasping at straws now...
Mint 19.2 XFCE
kernel 4.15.0-64-generic
AMD FX-8350 Eight-Core w/16 GB RAM
Nvidia GeForce 960 w/2GB RAM (435.21)
Image

User avatar
Flemur
Level 17
Level 17
Posts: 7076
Joined: Mon Aug 20, 2012 9:41 pm
Location: Potemkin Village

Re: Too fast uploading content on a flash drive

Post by Flemur » Thu Jul 12, 2018 12:35 pm

rado84 wrote:
Thu Jul 12, 2018 10:41 am
If the file is, for instance, 700 MB, uploading reaches 80% instantly and then hangs on this percentage forever.
I had that too with thunar/Xfce.
(Edit: except "not forever", it "hung" for as long as it took to copy the data).
It's a bug, but it's a bug you can ignore if you feel like it. I "fixed it" by using "cp" in a terminal.

Remember how - for years - when you copied something with windows it would say "3232328445 seconds remaining" then "25 seconds" then "98452394723 seconds" ?
Please edit your original post title to include [SOLVED] if/when it is solved!
Your data and OS are backed up....right?
Mint 19.1 Xfce/fluxbox
Manjaro openbox/fluxbox

ajgringo619
Level 6
Level 6
Posts: 1386
Joined: Thu Mar 01, 2018 8:36 pm
Location: San Diego, California

Re: Too fast uploading content on a flash drive

Post by ajgringo619 » Thu Jul 12, 2018 1:27 pm

Tried another test, this time using rsync. After about 500 MB, the command timed out for almost 2 minutes before kicking back into gear and finishing the copy; same thing happens when I use Thunar.

This never happens with my WD Passport USB drive, only my stick drives (Sandisk 64GB).
Mint 19.2 XFCE
kernel 4.15.0-64-generic
AMD FX-8350 Eight-Core w/16 GB RAM
Nvidia GeForce 960 w/2GB RAM (435.21)
Image

Mute Ant
Level 14
Level 14
Posts: 5135
Joined: Tue Sep 03, 2013 7:45 pm
Location: Norfolk UK

Re: Too fast uploading content on a flash drive

Post by Mute Ant » Thu Jul 12, 2018 8:24 pm

I think it is a write-cache phenomenon, but by Linux saving up data-to-send rather than the drive saving up data-received. Normally a Debian kernel is set to use 10% of your RAM to cache data being sent to a drive. So, if you write more than that, it goes RAM speed, then Drive speed.

You can smooth out transfers by limiting the kernel's write buffer to something similar to that in a drive...

sudo sysctl vm.dirty_bytes=8192000 ### Refuse to cache more than 8MB of new data, until the next boot.

I always eject the drive and insert it again to re-register the hardware and a new cache. Maybe that's not needed, I don't know.

"...when you copied something with windows..." https://xkcd.com/612/
While you're waiting, read the free novel we sent you. It's a Spanish story about a guy named "manual".

User avatar
rado84
Level 3
Level 3
Posts: 153
Joined: Sun Aug 19, 2012 9:31 am
Contact:

Re: Too fast uploading content on a flash drive

Post by rado84 » Fri Jul 13, 2018 9:21 am

Mute Ant wrote:
Thu Jul 12, 2018 8:24 pm
sudo sysctl vm.dirty_bytes=8192000 ### Refuse to cache more than 8MB of new data, until the next boot.
That didn't work either. 128 MB/sec upload speed. The integrated tool in Mint 19 offers other file systems (finally) but the result is the same with NTFS:
https://www19.zippyshare.com/v/YqiEUQbd/file.html
There's also EXT4 FS but the set top box for the TV won't read the flash drive if I format it in EXT4. It supports many codecs and formats but only if they're on a Windows' file system.

Post Reply

Return to “Software & Applications”