Making USB Mass Storage behave like Windows

Questions about hardware,drivers and peripherals
Forum rules
Before you post please read this

Making USB Mass Storage behave like Windows

Postby ranyardm on Tue Aug 19, 2008 5:32 pm

Okay I know, heresy right? Well we live in a world where manufacturers make the assumption that we run Windows.

This can cause problems because the Linux kernel makes the assumption that devices will work sensibly.

Background info : Windows automatically transfers no more than 32k of data to/from usb mass storage devices (usb pendrives, external hdds etc). Linux uses a default of nearly double that. Unfortunately some usb sticks and other devices (yes even USB2 ones) will fall over and corrupt the filesystem on the device and can even make the device unusable.

The solution : make a file (as superuser) in /etc/udev/rules.d/ called something like 99-usbwinsectsize.rules containing the following :
Code: Select all
DRIVERS=="usb-storage", RUN+="/bin/sh -c '/bin/echo 128 > /sys/block/%k/device/max_sectors'"

Now, this is VERY generic and will possibly slow down big transfers on better devices, but at least it won't make expensive devices that work perfectly well with Windows die. (I lost one 8Gb USB Stick to this "issue" in January, and am revisiting it because of Elyssa install on my EEE)

Could we make this sticky, and Clem, any chance of including this in the next release of Mint, as it really makes Mint more reliable with certain hardware.
ranyardm
Level 1
Level 1
 
Posts: 48
Joined: Thu Mar 20, 2008 3:40 pm

Linux Mint is funded by ads and donations.
 

Re: Making USB Mass Storage behave like Windows

Postby miket on Wed Aug 20, 2008 4:12 am

Hi !

ranyardm wrote:Okay I know, heresy right? Well we live in a world where manufacturers make the assumption that we run Windows.

This can cause problems because the Linux kernel makes the assumption that devices will work sensibly.

Background info : Windows automatically transfers no more than 32k of data to/from usb mass storage devices (usb pendrives, external hdds etc). Linux uses a default of nearly double that. Unfortunately some usb sticks and other devices (yes even USB2 ones) will fall over and corrupt the filesystem on the device and can even make the device unusable.

The solution : make a file (as superuser) in /etc/udev/rules.d/ called something like 99-usbwinsectsize.rules containing the following :
Code: Select all
DRIVERS=="usb-storage", RUN+="/bin/sh -c '/bin/echo 128 > /sys/block/%k/device/max_sectors'"

Now, this is VERY generic and will possibly slow down big transfers on better devices, but at least it won't make expensive devices that work perfectly well with Windows die. (I lost one 8Gb USB Stick to this "issue" in January, and am revisiting it because of Elyssa install on my EEE)

Could we make this sticky, and Clem, any chance of including this in the next release of Mint, as it really makes Mint more reliable with certain hardware.


I've never come across this at all ??!!
I've used all sorts of USB storage devices from 256mb to 16GB and have never had this problem ?
Including it as part of a default built of Mint is not a good idea at all, as it would slow down transfers from memory sticks that work perfectly with the larger data transfer setting,
I think documentation is the best option ;)

What make is your USB memory stick ??
I have a huge collection of devices here, from extremely cheap models from Ebay to expensive models and none of them have suffered from this problem ?
What Filesystem are you using on the memory stick ?

Mike.
User avatar
miket
Level 5
Level 5
 
Posts: 524
Joined: Tue Mar 04, 2008 5:50 am
Location: UK & France

Re: Making USB Mass Storage behave like Windows

Postby ranyardm on Wed Aug 20, 2008 9:45 am

Current one is a Emtec (french made) 8Gb usb stick that reports as
idVendor 0x0204
idProduct 0x1976

However, my point is that hardware that "just works" in windows with no drivers should not require digging about in sysfs to "just work" in Linux. I am not the only one with this problem - the bug report that led me to the solution (of which there are many duplicates across the net, even some in ubuntu's bug system) is :
https://bugs.launchpad.net/ubuntu/+source/udev/+bug/61235
Note the comment about the genesis chipset being already set to 64 by the kernel indicates that this is not a single device issue. Quite simply, Linux assumes quality hardware which is not necessarily the case.
I have to completely disagree with you on the documentation thing - usb mass storage devices don't require drivers or documentation on Windows and should not require documentation under linux.

A possibility that corollax_ came up with on irc was to have an option in MintAssistant to use the lower transfer threshold. I agree with this as a possibility, but I think the default option should be to not break hardware by default (no matter how assumptive the manufacturers have been) - i.e. include said udev rule - but to include a "turbo usb transfer" option to use the linux default of 240. Actually a Mint tool that allowed you to pick a threshold per device and add a udev rule would be really nice - that way, the good usb chipsets could have a super turbo of 1024 sectors.

EDIT: Remembered the brand. Added it in.
Last edited by ranyardm on Wed Aug 20, 2008 10:24 am, edited 1 time in total.
ranyardm
Level 1
Level 1
 
Posts: 48
Joined: Thu Mar 20, 2008 3:40 pm

Re: Making USB Mass Storage behave like Windows

Postby miket on Wed Aug 20, 2008 10:04 am

Hi !

ranyardm wrote:Current one is a french made 8Gb usb stick that reports as
idVendor 0x0204
idProduct 0x1976

However, my point is that hardware that "just works" in windows with no drivers should not require digging about in sysfs to "just work" in Linux. I am not the only one with this problem - the bug report that led me to the solution (of which there are many duplicates across the net, even some in ubuntu's bug system) is :
https://bugs.launchpad.net/ubuntu/+source/udev/+bug/61235
Note the comment about the genesis chipset being already set to 64 by the kernel indicates that this is not a single device issue. Quite simply, Linux assumes quality hardware which is not necessarily the case.
I have to completely disagree with you on the documentation thing - usb mass storage devices don't require drivers or documentation on Windows and should not require documentation under linux.

A possibility that corollax_ came up with on irc was to have an option in MintAssistant to use the lower transfer threshold. I agree with this as a possibility, but I think the default option should be to not break hardware by default (no matter how assumptive the manufacturers have been) - i.e. include said udev rule - but to include a "turbo usb transfer" option to use the linux default of 240. Actually a Mint tool that allowed you to pick a threshold per device and add a udev rule would be really nice - that way, the good usb chipsets could have a super turbo of 1024 sectors.


I think the big problem here is that you are thinking as a Windows user, not a UNIX user ... UNIX is not windows, it operates differently to windows, it was designed that way.
Just because it doesn't work with some USB memory sticks doesn't make it at fault, it has the ability to work correctly with these devices once you, the user, have configured it correctly to do so.
Video cards are a classic example of this !

Based on your comments Linux should work with all hardware without ever having to load any kind of driver or changing any of the default settings, windows most certainly doesn't, so why should Linux ??

I think you are just going through a learning process, which once you have completed you will see how much more flexible and powerful Linux is because of the way it is put together.
Adding "Features" to make it "More like windows" will do nothing but restrict the flexibility of the most flexible O/S there is around today ... resist thinking like this and try and embrace
the way Linux works .... sometimes in life we have to let go of how we've done things in the past and accept new and exciting ways of doing things ...

Mike.
User avatar
miket
Level 5
Level 5
 
Posts: 524
Joined: Tue Mar 04, 2008 5:50 am
Location: UK & France

Re: Making USB Mass Storage behave like Windows

Postby ranyardm on Wed Aug 20, 2008 10:42 am

AvanceIT wrote:I think the big problem here is that you are thinking as a Windows user, not a UNIX user ... UNIX is not windows, it operates differently to windows, it was designed that way.
Just because it doesn't work with some USB memory sticks doesn't make it at fault, it has the ability to work correctly with these devices once you, the user, have configured it correctly to do so.

Completely disagree - I'm thinking as an end-user - the USB stick doesn't come with drivers - you plug it in and it works - except on Linux, if you happen to plug in one make, you could lose a lot of data and end up with a stick that won't work in Linux or Windows ever again!
By the way, I'm a Linux user through and through - 8+ Years of linux server admin and constant use, dating back to Slackware 97.

AvanceIT wrote:Video cards are a classic example of this !

Video cards are not "plug and play" to the same extent that Mass Storage devices are

AvanceIT wrote:Based on your comments Linux should work with all hardware without ever having to load any kind of driver or changing any of the default settings, windows most certainly doesn't, so why should Linux ??

No I'm not saying that at all - and with a lot of hardware, you have to take the driver CD and run the software, reboot a few hundred times, and hopefully it'll work, but with USB Mass storage devices, you just plug em in.

AvanceIT wrote:I think you are just going through a learning process, which once you have completed you will see how much more flexible and powerful Linux is because of the way it is put together.

Oh so wrong on so many fronts, sorry, preaching to the choir here, just I don't do the fanboi attitude - 'cos after 8 years, you add a dash of realism and realise that Linux is not perfect and can be improved, hence the OP
AvanceIT wrote:Adding "Features" to make it "More like windows" will do nothing but restrict the flexibility of the most flexible O/S there is around today ... resist thinking like this and try and embrace the way Linux works .... sometimes in life we have to let go of how we've done things in the past and accept new and exciting ways of doing things ...

You have obviously taken the fact that I want to make Linux Mint more compatible with more hardware to mean that I want it to be more like Windows. I don't at all - I just want it to not kill hardware that works flawlessly plug-and-play under Windows.

I'm sorry, but judgement calls have to be made - not my job, but clem's* in this case - if a user comes to linux, plugs in their lovely 8Gb stick with their favorite music, their CV, important stuff they want to see if it's readable under Linux and whilst copying the data to the system, their stick dies, taking their valuable data with it, it's not going to give Linux a good rep. I'd rather see people not have to know that the particular stick manufacturer they have bought from (or worse rebadged by someone else) that they have to go and figure out which scsi device name was used, and then tell linux, "actually, no that stick can't use that many sectors, please use the same value that windows uses", by a command line they don't understand.

*Sorry to embroil you in the discussions clem, but I really think this is a valid fix for a valid problem, it would appear that lots of people (IRC and here) don't, but I feel strongly enough to say that this should be done.
ranyardm
Level 1
Level 1
 
Posts: 48
Joined: Thu Mar 20, 2008 3:40 pm

Re: Making USB Mass Storage behave like Windows

Postby miket on Wed Aug 20, 2008 11:09 am

Hi !

ranyardm wrote:Completely disagree - I'm thinking as an end-user - the USB stick doesn't come with drivers - you plug it in and it works - except on Linux, if you happen to plug in one make, you could lose a lot of data and end up with a stick that won't work in Linux or Windows ever again!


That's strange, I've got 6 brand new USB 8GB Memory sticks here and all of them have windows drivers on them ?
Not that I have any need for them :)
So are you saying that linux actually "Damages the hardware" to a point where it "ceases to function permanently" ??? I find this hard to believe ...

By the way, I'm a Linux user through and through - 8+ Years of linux server admin and constant use, dating back to Slackware 97.


Then I'm surprised you are asking for this change, surley as an experienced UNIX admin you can see this is the wrong way to go ?

Oh so wrong on so many fronts, sorry, preaching to the choir here, just I don't do the fanboi attitude - 'cos after 8 years, you add a dash of realism and realise that Linux is not perfect and can be improved, hence the OP


Yes Linux can be improved on, what O/S can't, but including this will actually be a step backwards as it will affect the performance of many just to make the "Odd one or two" work ...
I've been a UNIX Admin for 20 years or more and I realise that sometimes you can't use old/outdated hardware with new software, it's the way of progress ... we need to move on ....

You have obviously taken the fact that I want to make Linux Mint more compatible with more hardware to mean that I want it to be more like Windows. I don't at all - I just want it to not kill hardware that works flawlessly plug-and-play under Windows.


There are so many hardware items that work under windows and not under Linux though, where do you draw the line ? USB Webcams, USB sound devices, USB TV devices ... the list is endless ....
surely the best route would be to get the hardware manufacturers to support Linux ;)

Mike.
User avatar
miket
Level 5
Level 5
 
Posts: 524
Joined: Tue Mar 04, 2008 5:50 am
Location: UK & France

Re: Making USB Mass Storage behave like Windows

Postby ranyardm on Wed Aug 20, 2008 11:25 am

AvanceIT wrote:That's strange, I've got 6 brand new USB 8GB Memory sticks here and all of them have windows drivers on them ?
Not that I have any need for them :)

If you look at the disk, they will be only for Windows 98, almost guaranteed, or provide extra functionality (encryption with some, backup software with others); However, all the recent ones I've bought don't have driver discs at all
AvanceIT wrote:So are you saying that linux actually "Damages the hardware" to a point where it "ceases to function permanently" ??? I find this hard to believe ...

It may be hard to believe, but it is exactly what happens - the chipset fries, and the device then recognises at best 32mb, even when dd'd from /dev/zero, repartitioned and formatted, or inserted into windows.
AvanceIT wrote:Then I'm surprised you are asking for this change, surley as an experienced UNIX admin you can see this is the wrong way to go ?

Actually, I'm simply thinking of the end-user who has 8Gb of personal data on a stick and inserts it into a Linux machine because their friend says (quite rightly) Linux is cool, look it'll even open your documents you have there on your stick.... Right, just to be safe, I'll copy everything to the hard disk so we don't write to the stick... Oh, hang on, it just told me I should have unmounted before unplugging, but I didn't unplug it, odd... okay, lets try again, oh, it's saying it's not formatted now... oh... erm... sorry, just lost your cv for you.... yeah, anyway, Linux is great.... :oops:
AvanceIT wrote:Yes Linux can be improved on, what O/S can't, but including this will actually be a step backwards as it will affect the performance of many just to make the "Odd one or two" work ...
I've been a UNIX Admin for 20 years or more and I realise that sometimes you can't use old/outdated hardware with new software, it's the way of progress ... we need to move on ....

We're not talking outdated hardware - we're talking about an 8Gb Emtec usb stick (in this instance, but other sticks too) bought at PC World in January, and I believe still on sale today. It's not just "making the odd one or two work", it's not trashing someone's data because they don't know who the manufacturer of their USB stick is.
AvanceIT wrote:There are so many hardware items that work under windows and not under Linux though, where do you draw the line ? USB Webcams, USB sound devices, USB TV devices ... the list is endless ....

I draw the line at those that work perfectly under Windows without drivers and appear to work perfectly under Linux, just physically get bricked if you don't know who the manufacturer is.
AvanceIT wrote:surely the best route would be to get the hardware manufacturers to support Linux ;)

Now that's totally unfair, signing off with a statement you know I'd agree with! BTW, Emtec suck - never even got a reply from them about how to get an RMA number/form.
--
Martyn
ranyardm
Level 1
Level 1
 
Posts: 48
Joined: Thu Mar 20, 2008 3:40 pm

Linux Mint is funded by ads and donations.
 

Return to Hardware Support

Who is online

Users browsing this forum: No registered users and 16 guests