ext4 killed my data

[hoppel]

Hi,

when I uodated from Mint Felicia to Gloria, I chose ext4 as my new file system, because I thought it wouldn't cause any trouble, like updating from ext2 to ext3. But after the installation process all data on my home partition was gone although I didn't format it.

Is there a way I can recover the data?

[emorrp1]

First of all, how exactly did this happen. You say you selected ext4 as new file-system, but you also say you didn't format it. There is a way to do that, but it's an explicit command-line method, and you'd know if you did it. Merely selecting mount-as/use-as ext4 will not change the filesystem, you need to either format it, or find the above method (google for ext3 to ext4). If you just selected the wrong type in this way, replacing ext4 with ext3 in /etc/fstab may recover the data. You should also note that ext4 support is still experimental, hence why it's not chosen as default, you're kind of on your own to a certain extent if you choose to run with experimental features. ext4 will be the default in karmic/mint 8 (helena?), so it's guaranteed to be stable by then.

[hoppel]

Now nothing works anymore. Instead I get a message saying "you don't seem to have a home partition, would you like to use / instead?" But this option makes Mint stop working instantly.

Concerning my installation: I used the graphical installer, I don't know how to use the command-line for this. The installer didn't warn me about possible data loss. I knew that ext4 is new, but seeing it in the list of possible file systems led me to the conclusion that I could use it without greater (!) risk. Since I also know, that upgrading from ext2 to ext3 is no problem for the stored data, I thought it would be the same with ext4.
Maybe the graphical installer has chosen the format option itself - I didn't do that. I'm sure about that, because I willingly selected that option for the other partitions.
This is really bad, and I think that the installer should prevent such actions or should at least warn in any way. I lost very important data...

[emorrp1]

As I said before there is a way to migrate from ext3 to ext4 safely, but it won't be present in the installation gui, as it's not related to installation. Also you can use ext4 without a great amount of risk, but once you've got to the manual partitioning stage it is basically assumed you know what you're doing. You're probably right, maybe we should warn users more, but partitioning will always be non-trivial. I guess my point is that if you started with an ext3 /home, then followed the installer steps, but had an ext4 /home by the end of the process, then you must have made a mistake as the only real way to do that is formatting (you can however mount ext3 as ext4 to get a few extra features)

I'm sorry but without knowing more about the precise steps you made at the partitioning stage, I'm more likely to assume good old human error (we've all done it) rather than a bug, as there're plenty of people who have managed to get ext4 working no problems (though I don't personally know any who have upgraded ext3 to ext4). Additionally, everyone always recommends a complete data backup before making any major changes to your computer, especially a new OS, so if you didn't, then there's really nothing we can do.

Installing linux is basically at the stage where I guess novices and experts can install it easily, but intermediates generally have problems when they do stuff they're not sure of, but it's the only way to learn and become an expert. I'm just sorry that you're particular learning path was more costly than most.

[hoppel]

Well, I'm really sure about the things I have done during installation. I even remember thinking "maybe ext3 would have been safer" just after starting the installation process... Since it is working perfectly now - there is nothing wrong except the missing data - I don't think there's a bug in ext4. But I really know, that I haven't chosen a format process for my home partition. The installer must have done this itself. Maybe it's possible to see this - I should have checked that. But the fact is, the installer has done a format without warning.
And since it looks like you're involved in the making of Mint: When it comes to chose the way of installing, there are only two types of users who would choose the default installation and partitioning way: newbies who are too afraid to touch anything labeled "experts only" and experts who know exactly what will happen. But all others would choose the manual way, because only this way you have full control about partitioning. And usually that's not very dangerous... But the default mode isn't even explained - how should I know what happens when I chose this? Maybe it formats a partition without even telling me. In fact, for me it looks much more dangerous...

[emorrp1]

You're right, and part of the reason that makes you no longer a newbie is the fact you're not happy with the default options. In your case, you already have the /home that you want to re-use under the latest version of your distro, so none of the default options are even appropriate, it's an "advanced" partitioning setup as soon as you even create the seperate /home in the first place. I think the default options are as explained as they can be without assuming more Linux knowledge and they're fairly self-explanatory, though perhaps a help page which details the exact changes would be useful.

By precise steps, I mean almost click-by-click info. For instance, I tried to recreate this in a virtual machine. In the first (felicia) installation I created: 4GB ext3 / (sda1); 1GB swap (sda2); 3GB ext3 /home (sda3). I then started the second (gloria) installation process (using edit partition button):
Use sda1 as ext4, mounted at /, format
Use sda3 as ext4, mounted at /home, no format
sda2 was already marked as swap. Selecting use as ext4 on sda3 did not auto-check the format box, and the test data survived the installation (though I wouldn't recommend to use this method in production systems, as there would be /home conflicts between versions). So without further detail on exactly how you dealt with the /home partition, or what it was like before, I can't reproduce this error. It is possible I suppose, that it's a bug, but I still doubt it.

Anyway, at least you've learnt to be more careful and observant in case some developer thought they'd add a auto-change feature. It's also presumably left you with an effectively clean installation to play with, in which case I recommend a new partitioning scheme. Use the partition you had as /home as a data partition, and just leave /home under /. This way your data is always saved, but config files are removed, giving you a clean slate each install, without conflicting versions. You can even set it up for seemless integration with your Pictures folders and everything in /home (search the forum to see how, I wrote a brief post on it a while back)

p.s. I'm not on the dev team, I just help out, and have implemented the odd feature in mintUpload, that's all.

[hoppel]

Now I'm testing it again - I have to make my home partition ext3 again, to get access from Windows to it. Using the installer again it was like I thought: If I select a different filesystem than the already present, the installer chooses the format option itself, without asking or warning. When installing yesterday I didn't check that, because I did this process many times with ext3 without any problems. So it's not a bug, but a very dangerous behaviour by the installer.

[Carl]

Now I'm testing it again - I have to make my home partition ext3 again, to get access from Windows to it. Using the installer again it was like I thought: If I select a different filesystem than the already present, the installer chooses the format option itself, without asking or warning. When installing yesterday I didn't check that, because I did this process many times with ext3 without any problems. So it's not a bug, but a very dangerous behaviour by the installer.

On the very last part of the installer it asks you to check the options you selected and it states at the bottom what partitions are to be formated....Ubuntu does it exactly the same way

[Fred]

hoppel,

I know it doesn't help to say it now, but in the future, anytime you mess with a file system, especially if you are changing it, alarm bells should go off if there is data on the partition. This would apply regardless of the method being used.

Fred

[hoppel]

May I add one question: Meanwhile I made another installation to get my home partition back to ext3. I use this partition for data storage under Windows, too. With Felicia I had no problems to get access via "Ext2IFS", an ext2/3 driver for Windows. But now it doesn't seem to recognize the file system - does anyone know how to fix that? Because otherwise I'd have to repartition my whole hard disk...

[Fred]

hoppel,

Please do not use Windows drivers to access your Linux install from Windows. This is one of the worst things you can do for your security. You are just asking for some one to take your Linux box away from you. These kinds of security practices not only affect you but are a threat to others on the net too.

To be entirely honest with you, I will not help you in any way to get these drivers working for you. What I will do is help you every way I can to get you in a position where you don't need to access your Linux install from Windows with these drivers to do whatever it is you need to do.

Fred

[emorrp1]

I agree with Fred completely. For instance, if it's a shared data drive you want you're better off formatting it ntfs, as linux support for ntfs is fine.

[hoppel]

Dear Fred,

please accept this as my own decision. I know about the risks, but a.) accessing a NTFS drive with Linux is also risky, b.) I cannot use a NTFS drive as /home (as far as I know) and c.) I used this solution for quite while, and I never got any problem. You might never want to use your computer that way, but as a "normal" user I cannot and certainly never will be able to run a Linux only system. There has to be a way of coexisting, and I believe this to be the best solution.

It's not easy to complain about someone who's trying to help, but actually you are patronizing me. You should accept that there are different philosophies of computer usage. A warning is ok, but telling me to use my computer in a complete different way is not. I can live with that risks, and this forum shouldn't be the place to tell other people why you're NOT helping them.

[rlindsey0]

b.) I cannot use a NTFS drive as /home (as far as I know)

But, with respect, surely you could have created a /data partition in NTFS that both Linux and Windows could have had access to? I'm not as expert as many here, but that strikes me as a better idea. It's really only data that you want both OSs to have access to, correct? Not configuration files and the like? Aside from data, is there any reason to let Windows have access to /home? I can't think of a good one. If you have a separate data partition, you can let both OSs get at it, and you never let Windows anywhere near any other part of your Linux install, at least as I understand it. For example, I've got an outboard USB drive that's formatted to NTFS, and that's where I put anything I want both OSs to have access to. I've got a separate Data partition also on my HDD (/home/myself/Data); I formatted it to ext3, because I don't need/want XP to have access, but as far as I know I could have formatted it to NTFS at the install.

As for the other matter, if you ask a question on a public forum, you are likely to get answers you didn't expect. If someone thinks you are doing something that might be dangerous, why shouldn't they say so? I'm not sure why you would resent that--it's generally meant to be helpful, whether or not it's what you want to hear.

[Fred]

hoppel,

I am sorry you took my post as patronizing. I was just trying to be honest with you and show you how strongly I disagree with your decision regarding the Windows drivers. If your decision affected only you I wouldn't have put it so strongly. But your decision affects many others as well.

Do you remember reading about a bot net of Macs a few months back? It was in the tech press for a couple of weeks and was used to attack Macs as being no more secure than Windows. There were many derisive comments and much FUD was made of it. When people practice poor security practices in Linux it sets Linux up for the same kind of attacks. I care about the reputation of Linux and don't like to see it threatened by the decisions of uncaring or unknowing Linux users.

Give me the opportunity to address your comments and hopefully give you pause to reconsider your decision.

accessing a NTFS drive with Linux is also risky

Actually, accessing an NTFS partition from Linux presents almost zero risk to Linux, and an insignificant risk to Windows. None of the Windows bugs will have any effect on a user account of a Linux install.

I cannot use a NTFS drive as /home (as far as I know)

This is correct, but you don't need to in order to share data between Linux and Windows. There are other much more practical and secure methods available.

I used this solution for quite while, and I never got any problem.

How do you know? The whole idea in taking a box is to conceal the fact so that you can use it for your own purposes at will. It kind of defeats the purpose to give any indication to the rightful owner that the box is compromised. I can't tell you how many Windows boxes have been brought to me to have something done to them and the owners tell me they were clean. But when I hung a packet sniffer on the outputs there was so much activity, it looked like grand central station!

There has to be a way of coexisting, and I believe this to be the best solution.

There are ways to coexist. One's opinions and beliefs on subjective matters are unassailable. On technical objective matters, there is a right and wrong. Your decision here is technically incorrect.

You should accept that there are different philosophies of computer usage. A warning is ok, but telling me to use my computer in a complete different way is not. I can live with that risks, and this forum shouldn't be the place to tell other people why you're NOT helping them.

If your actions only put you at risk I could agree with you. But in this case your actions potentially affect me and many others. Your rights end at the beginning of my nose. I choose not to live with the risks you are willing to impose on yourself and me, and have a perfect right to complain and call attention to that fact. Your computer philosophy has no right to put others at risk.

I see it as a matter of socially acceptable behavior. I would never give a firearm to a child without insuring that qualified, adult supervision and training was in place. I could but I wouldn't. Just as I will not help a computer user to write malware or put in place poor security practices that I know will ultimately give the Linux community a bad name and put others at risk. I could but I wouldn't. That is just the kind of person that I am. I call them the way I see them.

Fred
.

[koss]

I also, agree with Fred on this. Couldn't have said it any better Fred!

[hoppel]

@ rlindsey0:
I have in fact lots of partitions for different purposes:
Linux /
Linux /home (the biggest partition, therefore also storage for Windows)
Linux /usr
Linux swap
Windows c:
Windows swap
Windows programms
Windows home (same function as the Linux home)
Windows movies
Windows data

@Fred:
For a long time the Windows movies partition was my favorite data storage. I changed it, because I have a greater believe in an completely open source based driver under Windows (for ext2/3) than in the result of a reengineering process (NTFS for Linux). To make a really reliable statement about the safety of those two possible solutions we would need a scientific study. And I don't see any verifiable argument in your statement - you're saying, I'm technically wrong but you don't give any reasons for it. Although it sounds as if you are mixing two different arguments: The lacking resistance of Windows against Viruses and its network vulnerabilities on the one hand, and the risk of data loss that can happen with a bad driver for a file system. You seem to be concerned about the first one, for me the second one matters.

I don't see, how I affect you're ways of working with your computer, and I don't understand why you still refuse to help. In a first reaction I thought you might be a kind of "typical" Linux expert, which seemed to be quite interesting, because I write my phd about the relation between experts and laypersons. You seem to have a quite typical expert-layperson way of thinking...
Anyway, I searched a bit and found a lot of help in the Linux community outside Linux Mint. The best help I found was in the official documentation of Ubuntu (with a link to Ext2Fsd, an alternative to Ext2IFS). So now I know that your way of thinking is at least not a very common one, even in the world of Linux.

One last argument from my side: you say you don't patronize me - but you're trying to steer my actions with all available force you can do inside the limited frame of an internet forum. I'd like to present a counter example to you: Maybe you are supporter of a political party which I really dislike - but I think that I don't have the right to force you to vote for a different party, because I should respect your free will. But you're acting like someone who is refusing to show me (virtually) the way to the polling place, because you don't want me to vote in a different manner than you do. In may eyes that counts as patronizing.

[AK Dave]

@ rlindsey0:
I have in fact lots of partitions for different purposes:
Linux /
Linux /home (the biggest partition, therefore also storage for Windows)

Thats the problem right there. Your /home need not, ought not, should not, also be your shared storage location.

Oh, and for the record: ext4 did not kill your data.

[markcynt]

hoppel, the best thing you can do is set up an NTFS partition/Hard Drive for your data. If you use a separate hard drive like moi, your chances of losing data go way, way down compared to the way you do it. Your way is just asking for trouble.

[rlindsey0]

@ rlindsey0:
I have in fact lots of partitions for different purposes:
Linux /
Linux /home (the biggest partition, therefore also storage for Windows)

Thats the problem right there. Your /home need not, ought not, should not, also be your shared storage location.

Oh, and for the record: ext4 did not kill your data.

This is what I was trying to get at, but you (i.e., AK Dave) said it more clearly and incisively.

@hoppel, no matter how many partitions you have, it's hard to see why you would ever want or need to let Windows into /home. Just seems like a bad idea. Why not just shrink /home, and create an NTFS partition for data to which Windows and Linux both have access?