Fixed:Mint13-14 Latest Kernel Version Causing EXT4-fs Errors

Questions about applications and software
Forum rules
Before you post read how to get help. Topics in this forum are automatically closed 6 months after creation.
User avatar
karlchen
Level 23
Level 23
Posts: 18206
Joined: Sat Dec 31, 2011 7:21 am
Location: Germany

Re: Mint13-14 Latest Kernel Version Causing EXT4-fs Errors

Post by karlchen »

Hello, viking777. Hello, folks.

As reported in my previous post, on Ubuntu 12.04.2 / Mint 13, x86 as well as x64, the reported problem has disappeared after installing kernel 3.2.0-37. It has no returned after installing kernel 3.2.0-38 (#61). So I consider the problem fixed with respect to the kernel 3.2.0 series. :D

Coming back to Ubuntu 12.10 / Mint 14 x86 (sorry, no x64 installed): Kernel 3.5.0-25 was installed last night at 21:40.
These are the EXT4-fs entries to be read in /var/log/syslog after having rebooted K3.5.0-25:
syslog:Feb 26 23:19:36 unimatrix0 kernel: [ 1.737622] EXT4-fs (sda6): mounted filesystem with ordered data mode. Opts: (null)
syslog:Feb 26 23:19:36 unimatrix0 kernel: [ 25.588791] EXT4-fs (sda6): re-mounted. Opts: errors=remount-ro
Note:
No "INFO: recovery required on readonly filesystem" and no "recovery complete"

I'll definitely keep an eye on it in order to find out whether this means K3.5.0-25 has finally solved the reported problem completely or whether I have just been lucky once.

<Added>
  1. First hypothesis discarded. I was lucky once, but not twice:
    syslog:Feb 26 23:19:36 unimatrix0 kernel: [ 1.737622] EXT4-fs (sda6): mounted filesystem with ordered data mode. Opts: (null)
    syslog:Feb 26 23:19:36 unimatrix0 kernel: [ 25.588791] EXT4-fs (sda6): re-mounted. Opts: errors=remount-ro
    syslog:Feb 27 00:35:01 unimatrix0 kernel: [ 1.666957] EXT4-fs (sda6): INFO: recovery required on readonly filesystem
    syslog:Feb 27 00:35:01 unimatrix0 kernel: [ 1.666970] EXT4-fs (sda6): write access will be enabled during recovery
    syslog:Feb 27 00:35:01 unimatrix0 kernel: [ 4.399591] EXT4-fs (sda6): recovery complete
    syslog:Feb 27 00:35:01 unimatrix0 kernel: [ 4.409158] EXT4-fs (sda6): mounted filesystem with ordered data mode. Opts: (null)
    syslog:Feb 27 00:35:01 unimatrix0 kernel: [ 28.626838] EXT4-fs (sda6): re-mounted. Opts: errors=remount-ro
  2. And again a new kernel version has invaded my system: K3.5.0-25 (#39).
    Starting from the next reboot a new hypothesis can be setup and verified concerning what will be logged about EXT4-fs.
    Oh, well. :roll:
</Added>

Cheers,
Karl
Image
The people of Alderaan have been bravely fighting back the clone warriors sent out by the unscrupulous Sith Lord Palpatine for 771 days now.
Lifeline
viking777

Re: Mint13-14 Latest Kernel Version Causing EXT4-fs Errors

Post by viking777 »

I am still getting that error with the 3.5.0-25 kenel on both Mint and Ubuntu, but it doesn't appear to be happening quite so often. 2 days ago in Mint, 4 days ago in Ubuntu.
Redondo
Level 1
Level 1
Posts: 41
Joined: Tue Dec 15, 2009 12:19 pm

Re: Mint13-14 Latest Kernel Version Causing EXT4-fs Errors

Post by Redondo »

I'm getting journal errors each and every time I boot. Mint 14 64bit, on Ext4, using Linux m 3.5.0-17-generic.

I have Ubuntu 12.04 on the same computer but different partition, Ext4, running linux 3.5.0-28 and it never fails.

What I do is after rebooting or powering off Mint 14, I then head over to Ubuntu partition and issue a "fsck" on the Mint 14 mount. Each time it has errors.

Code: Select all

cat /var/log/syslog | grep -i recovery
May 15 09:32:11 m kernel: [    3.174232] EXT4-fs (sda7): INFO: recovery required on readonly filesystem
May 15 09:32:11 m kernel: [    3.174236] EXT4-fs (sda7): write access will be enabled during recovery
May 15 09:32:11 m kernel: [    3.776357] EXT4-fs (sda7): recovery complete
May 15 14:10:01 m kernel: [    2.068382] EXT4-fs (sda7): INFO: recovery required on readonly filesystem
May 15 14:10:01 m kernel: [    2.068386] EXT4-fs (sda7): write access will be enabled during recovery
May 15 14:10:01 m kernel: [    3.426669] EXT4-fs (sda7): recovery complete
May 15 19:06:13 m kernel: [    2.048551] EXT4-fs (sda7): INFO: recovery required on readonly filesystem
May 15 19:06:13 m kernel: [    2.048555] EXT4-fs (sda7): write access will be enabled during recovery
May 15 19:06:13 m kernel: [    3.656549] EXT4-fs (sda7): recovery complete
May 15 19:08:30 m kernel: [    2.183427] EXT4-fs (sda7): INFO: recovery required on readonly filesystem
May 15 19:08:30 m kernel: [    2.183431] EXT4-fs (sda7): write access will be enabled during recovery
May 15 19:08:30 m kernel: [    3.433620] EXT4-fs (sda7): recovery complete
I also reported "it effects me too" on the 1105900 bug report.
User avatar
karlchen
Level 23
Level 23
Posts: 18206
Joined: Sat Dec 31, 2011 7:21 am
Location: Germany

Re: Mint13-14 Latest Kernel Version Causing EXT4-fs Errors

Post by karlchen »

Hello, Redondo.

Thank you for confirming it is not my particular Mint 14 system only, currently K3.5.0-28, 32-bit.
On Ubuntu 12.04.2 x64 K3.2.0-41 I do not see the problem, either.
I should cross-check with Ubuntu 12.10 K3.5.0-28, because it is the basis of Mint 14 K3.5.0-28, but I do not really feel like installing Ubuntu 12.10 just for this tests. :oops:
Anyway, your positive experience with Ubuntu 12.04 K3.5.0-28 suggests, there might be something in Mint 14 which prevents the root filesystem from being dismounted cleanly before shtudown/reboot.
If only there were a way of finding out what it is ...
Everything that I tried did not enlighten me. :(

Cheers,
Karl
Image
The people of Alderaan have been bravely fighting back the clone warriors sent out by the unscrupulous Sith Lord Palpatine for 771 days now.
Lifeline
Redondo
Level 1
Level 1
Posts: 41
Joined: Tue Dec 15, 2009 12:19 pm

Re: Mint13-14 Latest Kernel Version Causing EXT4-fs Errors

Post by Redondo »

Ok, I think I just proved its not Ext4 causing the problem, but in fact Mint's mdm. I rebooted to X-only without a display manager. Then rebooted to Ubuntu to check the fs again. fsck was clean.
I wanted to make sure the initramfs was deployed and all the proc devices.

I decided to try another FS, so I installed Mint 14 on ReiserFS. That went smoth enough.

Once installed I booted to the new Mint 14 ReiserFS. Then after reboot, this time fsck had some rather strange messages, which I found out are error messages caused by improper shutdown!

Here's the reiserFS check messages. I did it twice so to compare a good ReiserFS check:

Code: Select all

$ sudo fsck /dev/sda7
fsck from util-linux 2.20.1
reiserfsck 3.6.21 (2009 www.namesys.com)

*************************************************************
** If you are using the latest reiserfsprogs and  it fails **
** please  email bug reports to reiserfs-list@namesys.com, **
** providing  as  much  information  as  possible --  your **
** hardware,  kernel,  patches,  settings,  all reiserfsck **
** messages  (including version),  the reiserfsck logfile, **
** check  the  syslog file  for  any  related information. **
** If you would like advice on using this program, support **
** is available  for $25 at  www.namesys.com/support.html. **
*************************************************************

Will read-only check consistency of the filesystem on /dev/sda7
Will put log info to 'stdout'

Do you want to run this program?[N/Yes] (note need to type Yes if you do):Yes
###########
reiserfsck --check started at Fri May 17 04:41:08 2013
###########
Replaying journal: Trans replayed: mountid 14, transid 1489, desc 3986, len 26, commit 4013, next trans offset 3996

Trans replayed: mountid 14, transid 1491, desc 4124, len 20, commit 4145, next trans offset 4128
Trans replayed: mountid 14, transid 1492, desc 4146, len 20, commit 4167, next trans offset 4150
Trans replayed: mountid 14, transid 1493, desc 4168, len 21, commit 4190, next trans offset 4173
Trans replayed: mountid 14, transid 1494, desc 4191, len 26, commit 4218, next trans offset 4201
Trans replayed: mountid 14, transid 1495, desc 4219, len 6, commit 4226, next trans offset 4209
Trans replayed: mountid 14, transid 1496, desc 4227, len 26, commit 4254, next trans offset 4237
Trans replayed: mountid 14, transid 1497, desc 4255, len 18, commit 4274, next trans offset 4257
Trans replayed: mountid 14, transid 1498, desc 4275, len 19, commit 4295, next trans offset 4278
Trans replayed: mountid 14, transid 1499, desc 4296, len 8, commit 4305, next trans offset 4288
Trans replayed: mountid 14, transid 1500, desc 4306, len 17, commit 4324, next trans offset 4307
Trans replayed: mountid 14, transid 1501, desc 4325, len 8, commit 4334, next trans offset 4317
Trans replayed: mountid 14, transid 1502, desc 4335, len 6, commit 4342, next trans offset 4325
Trans replayed: mountid 14, transid 1503, desc 4343, len 11, commit 4355, next trans offset 4338
Trans replayed: mountid 14, transid 1504, desc 4356, len 9, commit 4366, next trans offset 4349
Trans replayed: mountid 14, transid 1505, desc 4367, len 11, commit 4379, next trans offset 4362
Trans replayed: mountid 14, transid 1506, desc 4380, len 10, commit 4391, next trans offset 4374
Trans replayed: mountid 14, transid 1507, desc 4392, len 6, commit 4399, next trans offset 4382
Trans replayed: mountid 14, transid 1508, desc 4400, len 17, commit 4418, next trans offset 4401
Trans replayed: mountid 14, transid 1509, desc 4419, len 2, commit 4422, next trans offset 4405
Trans replayed: mountid 14, transid 1510, desc 4423, len 22, commit 4446, next trans offset 4429
Trans replayed: mountid 14, transid 1511, desc 4447, len 15, commit 4463, next trans offset 4446
Trans replayed: mountid 14, transid 1512, desc 4464, len 31, commit 4496, next trans offset 4479
Trans replayed: mountid 14, transid 1513, desc 4497, len 8, commit 4506, next trans offset 4489
Trans replayed: mountid 14, transid 1514, desc 4507, len 1, commit 4509, next trans offset 4492
Trans replayed: mountid 14, transid 1515, desc 4510, len 1, commit 4512, next trans offset 4495
Trans replayed: mountid 14, transid 1516, desc 4513, len 17, commit 4531, next trans offset 4514
Trans replayed: mountid 14, transid 1517, desc 4532, len 13, commit 4546, next trans offset 4529
Trans replayed: mountid 14, transid 1518, desc 4547, len 1, commit 4549, next trans offset 4532
Trans replayed: mountid 14, transid 1519, desc 4550, len 1, commit 4552, next trans offset 4535
Trans replayed: mountid 14, transid 1520, desc 4553, len 1, commit 4555, next trans offset 4538
Trans replayed: mountid 14, transid 1521, desc 4556, len 1, commit 4558, next trans offset 4541
Trans replayed: mountid 14, transid 1522, desc 4559, len 1, commit 4561, next trans offset 4544
Trans replayed: mountid 14, transid 1523, desc 4562, len 1, commit 4564, next trans offset 4547
Trans replayed: mountid 14, transid 1524, desc 4565, len 1, commit 4567, next trans offset 4550
Trans replayed: mountid 14, transid 1525, desc 4568, len 1, commit 4570, next trans offset 4553
Trans replayed: mountid 14, transid 1526, desc 4571, len 2, commit 4574, next trans offset 4557
Trans replayed: mountid 14, transid 1527, desc 4575, len 1, commit 4577, next trans offset 4560
Trans replayed: mountid 14, transid 1528, desc 4578, len 1, commit 4580, next trans offset 4563
Replaying journal: Done.                                                        
Reiserfs journal '/dev/sda7' in blocks [18..8211]: 40 transactions replayed
Checking internal tree.. finished
Comparing bitmaps..finished
Checking Semantic tree:
finished
No corruptions found
There are on the filesystem:
	Leaves 6508
	Internal nodes 47
	Directories 15466
	Other files 138797
	Data block pointers 759408 (71 of them are zero)
	Safe links 0
###########
reiserfsck finished at Fri May 17 04:41:17 2013
###########


===second fsck===
$ sudo fsck /dev/sda7
fsck from util-linux 2.20.1
reiserfsck 3.6.21 (2009 www.namesys.com)

*************************************************************
** If you are using the latest reiserfsprogs and  it fails **
** please  email bug reports to reiserfs-list@namesys.com, **
** providing  as  much  information  as  possible --  your **
** hardware,  kernel,  patches,  settings,  all reiserfsck **
** messages  (including version),  the reiserfsck logfile, **
** check  the  syslog file  for  any  related information. **
** If you would like advice on using this program, support **
** is available  for $25 at  www.namesys.com/support.html. **
*************************************************************

Will read-only check consistency of the filesystem on /dev/sda7
Will put log info to 'stdout'

Do you want to run this program?[N/Yes] (note need to type Yes if you do):Yes
###########
reiserfsck --check started at Fri May 17 04:43:04 2013
###########
Replaying journal: Done.
Reiserfs journal '/dev/sda7' in blocks [18..8211]: 0 transactions replayed
Checking internal tree.. finished
Comparing bitmaps..finished
Checking Semantic tree:
finished
No corruptions found
There are on the filesystem:
	Leaves 6508
	Internal nodes 47
	Directories 15466
	Other files 138797
	Data block pointers 759408 (71 of them are zero)
	Safe links 0
###########
reiserfsck finished at Fri May 17 04:43:11 2013
###########
All those "Trans replayed" messages are errors that the fsck found.

Maybe another topic should be opened, and point it to Mint 14 shutdown/Power-off sequence as the problem, and not the fs.
Redondo
Level 1
Level 1
Posts: 41
Joined: Tue Dec 15, 2009 12:19 pm

Re: Mint13-14 Latest Kernel Version Causing EXT4-fs Errors

Post by Redondo »

Guess what? There's nothing wrong (I think).

These errors have lead me down the path of futility. Someone mentioned my hard drive might be at fault. On that note, I tried another hard drive. I remove/replace the HD with another. walla! No errors. I used Min14 LIVECD to test it. I didn't think until much about it until later.

Once again I re-installed Mint14 into partition # 7, and once again it failed. Thinking maybe there's something wrong with the orignal disk,I unloaded the partitions and was in position to reformat the who shebang, or replace it. Then I used e2fsck to test for bad blocks:

Code: Select all

$ sudo e2fsck -cv /dev/sda7
e2fsck 1.42 (29-Nov-2011)
Checking for bad blocks (read-only test): done                                                 
/dev/sda7: Updating bad block inode.
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information

/dev/sda7: ***** FILE SYSTEM WAS MODIFIED *****

  154253 inodes used (16.47%)
      28 non-contiguous files (0.0%)
      93 non-contiguous directories (0.1%)
         # of inodes with ind/dind/tind blocks: 0/0/0
         Extent depth histogram: 125044/3
  893063 blocks used (23.86%)
       0 bad blocks
       1 large file

  107628 regular files
   15462 directories
      57 character device files
      25 block device files
       0 fifos
       0 links
   31070 symbolic links (29114 fast symbolic links)
       2 sockets
--------
  154244 files
As you can see, nothing showed up.

Every time I rebooted to Ubuntu to test Mint14's install I got the following error:

Code: Select all

$ sudo e2fsck /dev/sda7
[sudo] password for vmc: 
e2fsck 1.42 (29-Nov-2011)
/dev/sda7: recovering journal
Setting free inodes count to 782227 (was 782229)
Setting free blocks count to 2848012 (was 2848059)
/dev/sda7: clean, 154333/936560 files, 895152/3743164 blocks
Then it dawned on me. On the other drive install, I tested using Mint14's LIVECD. So I tried once agin using Mint's LIVECD. walla!, again. No errors.
Only this time, I realized its something to do with using Ubuntu's installed OS's fsck. (could this be a Ubuntu conspiracy :shock: ).

The only thing left to do is replace Mint14's LIVECD with Ubuntu's and test again. At least I now know, it was a form of cockpit error.
User avatar
karlchen
Level 23
Level 23
Posts: 18206
Joined: Sat Dec 31, 2011 7:21 am
Location: Germany

Mint13-14 Kernel Versions Causing EXT4-fs Errors [fixed]

Post by karlchen »

Hi, folks.

I know it has been a really long time since I have last posted in this thread which after all has been started by me.

Ubuntu 12.04 Kernel 3.2.0-37

In Launchpad bug report post #17 I confirmed that starting with kernel 3.2.0-37 the reported problem/bug had been fixed.
As far as I can tell this is true for all later 3.2.0-x kernels, too.
On my Ubuntu 12.04, by now Ubuntu 12.04.3, the problem has never occurred again. And currently, I am on kernel 3.2.0-55.
Summary: Bug fixed in kernel 3.2.0-37 and all later kernels.

Ubuntu 12.10 Kernel 3.5.0-23

In Launchpard bug report post #18 I had stated that the problem still persisted. This was right with respect to the symptoms. This was wrong with respect to the cause.
Starting with 3.5.0.23 the "filesystem / busy" during the shutdown and the need to perform an fsck during startup has no longer been caused by the kernel.

As I know by now - shame on me for needing more than 6 months to find out - the remaining symptoms were caused by a shutdown problem which is discussed e.g. in these two Launchpad bug reports:
+ Upstartification of /etc/init.d/networking has lost deconfiguring-networking event causing bad side-effects
+ In Quantal, the root filesystem is not cleanly unmounted at shutdown or reboot

So whoever is still experiencing an unclean root filesystem on each startup should check those two bug reports.

From my point of view: Kernel bug fixed in kernel 3.5.0-23 and all later kernels.

Kind regards,
Karl
--
P.S.:
On my Mint 14 the reported bug(s) [1] [2] have been worked around by now and Nadia will startup with a clean root filesystem. Yet, this is a different story and deserves its own thread.
Image
The people of Alderaan have been bravely fighting back the clone warriors sent out by the unscrupulous Sith Lord Palpatine for 771 days now.
Lifeline
Locked

Return to “Software & Applications”