Mint13-14 Latest Kernel Version Causing EXT4-fs Errors

[karlchen]

Hello, viking777. Hello, folks.

As reported in my previous post, on Ubuntu 12.04.2 / Mint 13, x86 as well as x64, the reported problem has disappeared after installing kernel 3.2.0-37. It has no returned after installing kernel 3.2.0-38 (#61). So I consider the problem fixed with respect to the kernel 3.2.0 series.

Coming back to Ubuntu 12.10 / Mint 14 x86 (sorry, no x64 installed): Kernel 3.5.0-25 was installed last night at 21:40.
These are the EXT4-fs entries to be read in /var/log/syslog after having rebooted K3.5.0-25:

syslog:Feb 26 23:19:36 unimatrix0 kernel: [ 1.737622] EXT4-fs (sda6): mounted filesystem with ordered data mode. Opts: (null)
syslog:Feb 26 23:19:36 unimatrix0 kernel: [ 25.588791] EXT4-fs (sda6): re-mounted. Opts: errors=remount-ro

Note:
No "INFO: recovery required on readonly filesystem" and no "recovery complete"

I'll definitely keep an eye on it in order to find out whether this means K3.5.0-25 has finally solved the reported problem completely or whether I have just been lucky once.

<Added>

1. First hypothesis discarded. I was lucky once, but not twice:

   syslog:Feb 26 23:19:36 unimatrix0 kernel: [ 1.737622] EXT4-fs (sda6): mounted filesystem with ordered data mode. Opts: (null)
   syslog:Feb 26 23:19:36 unimatrix0 kernel: [ 25.588791] EXT4-fs (sda6): re-mounted. Opts: errors=remount-ro
   syslog:Feb 27 00:35:01 unimatrix0 kernel: [ 1.666957] EXT4-fs (sda6): INFO: recovery required on readonly filesystem
   syslog:Feb 27 00:35:01 unimatrix0 kernel: [ 1.666970] EXT4-fs (sda6): write access will be enabled during recovery
   syslog:Feb 27 00:35:01 unimatrix0 kernel: [ 4.399591] EXT4-fs (sda6): recovery complete
   syslog:Feb 27 00:35:01 unimatrix0 kernel: [ 4.409158] EXT4-fs (sda6): mounted filesystem with ordered data mode. Opts: (null)
   syslog:Feb 27 00:35:01 unimatrix0 kernel: [ 28.626838] EXT4-fs (sda6): re-mounted. Opts: errors=remount-ro

2. And again a new kernel version has invaded my system: K3.5.0-25 (#39).
   Starting from the next reboot a new hypothesis can be setup and verified concerning what will be logged about EXT4-fs.
   Oh, well.

</Added>

Cheers,
Karl

[viking777]

I am still getting that error with the 3.5.0-25 kenel on both Mint and Ubuntu, but it doesn't appear to be happening quite so often. 2 days ago in Mint, 4 days ago in Ubuntu.

[Redondo]

I'm getting journal errors each and every time I boot. Mint 14 64bit, on Ext4, using Linux m 3.5.0-17-generic.

I have Ubuntu 12.04 on the same computer but different partition, Ext4, running linux 3.5.0-28 and it never fails.

What I do is after rebooting or powering off Mint 14, I then head over to Ubuntu partition and issue a "fsck" on the Mint 14 mount. Each time it has errors.

Code: Select all

cat /var/log/syslog | grep -i recovery
May 15 09:32:11 m kernel: [    3.174232] EXT4-fs (sda7): INFO: recovery required on readonly filesystem
May 15 09:32:11 m kernel: [    3.174236] EXT4-fs (sda7): write access will be enabled during recovery
May 15 09:32:11 m kernel: [    3.776357] EXT4-fs (sda7): recovery complete
May 15 14:10:01 m kernel: [    2.068382] EXT4-fs (sda7): INFO: recovery required on readonly filesystem
May 15 14:10:01 m kernel: [    2.068386] EXT4-fs (sda7): write access will be enabled during recovery
May 15 14:10:01 m kernel: [    3.426669] EXT4-fs (sda7): recovery complete
May 15 19:06:13 m kernel: [    2.048551] EXT4-fs (sda7): INFO: recovery required on readonly filesystem
May 15 19:06:13 m kernel: [    2.048555] EXT4-fs (sda7): write access will be enabled during recovery
May 15 19:06:13 m kernel: [    3.656549] EXT4-fs (sda7): recovery complete
May 15 19:08:30 m kernel: [    2.183427] EXT4-fs (sda7): INFO: recovery required on readonly filesystem
May 15 19:08:30 m kernel: [    2.183431] EXT4-fs (sda7): write access will be enabled during recovery
May 15 19:08:30 m kernel: [    3.433620] EXT4-fs (sda7): recovery complete


I also reported "it effects me too" on the 1105900 bug report.

[karlchen]

Hello, Redondo.

Thank you for confirming it is not my particular Mint 14 system only, currently K3.5.0-28, 32-bit.
On Ubuntu 12.04.2 x64 K3.2.0-41 I do not see the problem, either.
I should cross-check with Ubuntu 12.10 K3.5.0-28, because it is the basis of Mint 14 K3.5.0-28, but I do not really feel like installing Ubuntu 12.10 just for this tests.
Anyway, your positive experience with Ubuntu 12.04 K3.5.0-28 suggests, there might be something in Mint 14 which prevents the root filesystem from being dismounted cleanly before shtudown/reboot.
If only there were a way of finding out what it is ...
Everything that I tried did not enlighten me.

Cheers,
Karl

[Redondo]

Ok, I think I just proved its not Ext4 causing the problem, but in fact Mint's mdm. I rebooted to X-only without a display manager. Then rebooted to Ubuntu to check the fs again. fsck was clean.
I wanted to make sure the initramfs was deployed and all the proc devices.

I decided to try another FS, so I installed Mint 14 on ReiserFS. That went smoth enough.

Once installed I booted to the new Mint 14 ReiserFS. Then after reboot, this time fsck had some rather strange messages, which I found out are error messages caused by improper shutdown!

Here's the reiserFS check messages. I did it twice so to compare a good ReiserFS check:

Code: Select all

$ sudo fsck /dev/sda7
fsck from util-linux 2.20.1
reiserfsck 3.6.21 (2009 www.namesys.com)

*************************************************************
** If you are using the latest reiserfsprogs and  it fails **
** please  email bug reports to reiserfs-list@namesys.com, **
** providing  as  much  information  as  possible --  your **
** hardware,  kernel,  patches,  settings,  all reiserfsck **
** messages  (including version),  the reiserfsck logfile, **
** check  the  syslog file  for  any  related information. **
** If you would like advice on using this program, support **
** is available  for $25 at  www.namesys.com/support.html. **
*************************************************************

Will read-only check consistency of the filesystem on /dev/sda7
Will put log info to 'stdout'

Do you want to run this program?[N/Yes] (note need to type Yes if you do):Yes
###########
reiserfsck --check started at Fri May 17 04:41:08 2013
###########
Replaying journal: Trans replayed: mountid 14, transid 1489, desc 3986, len 26, commit 4013, next trans offset 3996

Trans replayed: mountid 14, transid 1491, desc 4124, len 20, commit 4145, next trans offset 4128
Trans replayed: mountid 14, transid 1492, desc 4146, len 20, commit 4167, next trans offset 4150
Trans replayed: mountid 14, transid 1493, desc 4168, len 21, commit 4190, next trans offset 4173
Trans replayed: mountid 14, transid 1494, desc 4191, len 26, commit 4218, next trans offset 4201
Trans replayed: mountid 14, transid 1495, desc 4219, len 6, commit 4226, next trans offset 4209
Trans replayed: mountid 14, transid 1496, desc 4227, len 26, commit 4254, next trans offset 4237
Trans replayed: mountid 14, transid 1497, desc 4255, len 18, commit 4274, next trans offset 4257
Trans replayed: mountid 14, transid 1498, desc 4275, len 19, commit 4295, next trans offset 4278
Trans replayed: mountid 14, transid 1499, desc 4296, len 8, commit 4305, next trans offset 4288
Trans replayed: mountid 14, transid 1500, desc 4306, len 17, commit 4324, next trans offset 4307
Trans replayed: mountid 14, transid 1501, desc 4325, len 8, commit 4334, next trans offset 4317
Trans replayed: mountid 14, transid 1502, desc 4335, len 6, commit 4342, next trans offset 4325
Trans replayed: mountid 14, transid 1503, desc 4343, len 11, commit 4355, next trans offset 4338
Trans replayed: mountid 14, transid 1504, desc 4356, len 9, commit 4366, next trans offset 4349
Trans replayed: mountid 14, transid 1505, desc 4367, len 11, commit 4379, next trans offset 4362
Trans replayed: mountid 14, transid 1506, desc 4380, len 10, commit 4391, next trans offset 4374
Trans replayed: mountid 14, transid 1507, desc 4392, len 6, commit 4399, next trans offset 4382
Trans replayed: mountid 14, transid 1508, desc 4400, len 17, commit 4418, next trans offset 4401
Trans replayed: mountid 14, transid 1509, desc 4419, len 2, commit 4422, next trans offset 4405
Trans replayed: mountid 14, transid 1510, desc 4423, len 22, commit 4446, next trans offset 4429
Trans replayed: mountid 14, transid 1511, desc 4447, len 15, commit 4463, next trans offset 4446
Trans replayed: mountid 14, transid 1512, desc 4464, len 31, commit 4496, next trans offset 4479
Trans replayed: mountid 14, transid 1513, desc 4497, len 8, commit 4506, next trans offset 4489
Trans replayed: mountid 14, transid 1514, desc 4507, len 1, commit 4509, next trans offset 4492
Trans replayed: mountid 14, transid 1515, desc 4510, len 1, commit 4512, next trans offset 4495
Trans replayed: mountid 14, transid 1516, desc 4513, len 17, commit 4531, next trans offset 4514
Trans replayed: mountid 14, transid 1517, desc 4532, len 13, commit 4546, next trans offset 4529
Trans replayed: mountid 14, transid 1518, desc 4547, len 1, commit 4549, next trans offset 4532
Trans replayed: mountid 14, transid 1519, desc 4550, len 1, commit 4552, next trans offset 4535
Trans replayed: mountid 14, transid 1520, desc 4553, len 1, commit 4555, next trans offset 4538
Trans replayed: mountid 14, transid 1521, desc 4556, len 1, commit 4558, next trans offset 4541
Trans replayed: mountid 14, transid 1522, desc 4559, len 1, commit 4561, next trans offset 4544
Trans replayed: mountid 14, transid 1523, desc 4562, len 1, commit 4564, next trans offset 4547
Trans replayed: mountid 14, transid 1524, desc 4565, len 1, commit 4567, next trans offset 4550
Trans replayed: mountid 14, transid 1525, desc 4568, len 1, commit 4570, next trans offset 4553
Trans replayed: mountid 14, transid 1526, desc 4571, len 2, commit 4574, next trans offset 4557
Trans replayed: mountid 14, transid 1527, desc 4575, len 1, commit 4577, next trans offset 4560
Trans replayed: mountid 14, transid 1528, desc 4578, len 1, commit 4580, next trans offset 4563
Replaying journal: Done.                                                       
Reiserfs journal '/dev/sda7' in blocks [18..8211]: 40 transactions replayed
Checking internal tree.. finished
Comparing bitmaps..finished
Checking Semantic tree:
finished
No corruptions found
There are on the filesystem:
   Leaves 6508
   Internal nodes 47
   Directories 15466
   Other files 138797
   Data block pointers 759408 (71 of them are zero)
   Safe links 0
###########
reiserfsck finished at Fri May 17 04:41:17 2013
###########


===second fsck===
$ sudo fsck /dev/sda7
fsck from util-linux 2.20.1
reiserfsck 3.6.21 (2009 www.namesys.com)

*************************************************************
** If you are using the latest reiserfsprogs and  it fails **
** please  email bug reports to reiserfs-list@namesys.com, **
** providing  as  much  information  as  possible --  your **
** hardware,  kernel,  patches,  settings,  all reiserfsck **
** messages  (including version),  the reiserfsck logfile, **
** check  the  syslog file  for  any  related information. **
** If you would like advice on using this program, support **
** is available  for $25 at  www.namesys.com/support.html. **
*************************************************************

Will read-only check consistency of the filesystem on /dev/sda7
Will put log info to 'stdout'

Do you want to run this program?[N/Yes] (note need to type Yes if you do):Yes
###########
reiserfsck --check started at Fri May 17 04:43:04 2013
###########
Replaying journal: Done.
Reiserfs journal '/dev/sda7' in blocks [18..8211]: 0 transactions replayed
Checking internal tree.. finished
Comparing bitmaps..finished
Checking Semantic tree:
finished
No corruptions found
There are on the filesystem:
   Leaves 6508
   Internal nodes 47
   Directories 15466
   Other files 138797
   Data block pointers 759408 (71 of them are zero)
   Safe links 0
###########
reiserfsck finished at Fri May 17 04:43:11 2013
###########


All those "Trans replayed" messages are errors that the fsck found.

Maybe another topic should be opened, and point it to Mint 14 shutdown/Power-off sequence as the problem, and not the fs.

[Redondo]

Guess what? There's nothing wrong (I think).

These errors have lead me down the path of futility. Someone mentioned my hard drive might be at fault. On that note, I tried another hard drive. I remove/replace the HD with another. walla! No errors. I used Min14 LIVECD to test it. I didn't think until much about it until later.

Once again I re-installed Mint14 into partition # 7, and once again it failed. Thinking maybe there's something wrong with the orignal disk,I unloaded the partitions and was in position to reformat the who shebang, or replace it. Then I used e2fsck to test for bad blocks:

Code: Select all

$ sudo e2fsck -cv /dev/sda7
e2fsck 1.42 (29-Nov-2011)
Checking for bad blocks (read-only test): done                                                 
/dev/sda7: Updating bad block inode.
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information

/dev/sda7: ***** FILE SYSTEM WAS MODIFIED *****

  154253 inodes used (16.47%)
      28 non-contiguous files (0.0%)
      93 non-contiguous directories (0.1%)
         # of inodes with ind/dind/tind blocks: 0/0/0
         Extent depth histogram: 125044/3
  893063 blocks used (23.86%)
       0 bad blocks
       1 large file

  107628 regular files
   15462 directories
      57 character device files
      25 block device files
       0 fifos
       0 links
   31070 symbolic links (29114 fast symbolic links)
       2 sockets
--------
  154244 files

As you can see, nothing showed up.

Every time I rebooted to Ubuntu to test Mint14's install I got the following error:

Code: Select all

$ sudo e2fsck /dev/sda7
[sudo] password for vmc:
e2fsck 1.42 (29-Nov-2011)
/dev/sda7: recovering journal
Setting free inodes count to 782227 (was 782229)
Setting free blocks count to 2848012 (was 2848059)
/dev/sda7: clean, 154333/936560 files, 895152/3743164 blocks


Then it dawned on me. On the other drive install, I tested using Mint14's LIVECD. So I tried once agin using Mint's LIVECD. walla!, again. No errors.
Only this time, I realized its something to do with using Ubuntu's installed OS's fsck. (could this be a Ubuntu conspiracy ).

The only thing left to do is replace Mint14's LIVECD with Ubuntu's and test again. At least I now know, it was a form of cockpit error.