SOLVED - LM19 xfce "login loop" / read only ext4 partition after crash

[gm10]

1. Doesn't need Internet, no. I don't really think it will do anything, either though. I know I was the one who linked that thread but I'm not convinced it applies in your case. Anyway, can't hurt to try I guess.

2. If you install via ubiquity -b (which will launch the normal graphical installer, don't worry) then no change to your boot configuration will be made at all. GRUB simply won't be installed (which also means you won't get any updates to it until you install it again), so the existing version must handle it. Just make sure that in the installer you don't select to format / root if you go that route or you won't be able to boot.

Please make a backup first.

[fabien85]

+1 on backup, very important.
You should be able to mount and read your Mint partition from a live USB, so copy your files to an external drive (or any other backup method your prefer)

If you do not reformat the / partition, then its uuid will be kept, and as gm10 says grub should then find your root partition.
That said, it wont hurt either if you launch a normal install and reinstall grub. At least I dont see any drawback unless you had particular problems with installing grub last time. (your previous thread was "only" a problem with setting up the config file to have the good entry in the grub menu)

When you get back running (hopefully), you will probably want to set up timeshift, so that in future you can just restore a timeshift snapshot to get your system back. A life saver.

[Girafenaine]

IT WORKED ! even if I am not sure about what worked. I had two documents backups I made yesterday on two external hard drives (from MacOS with carbon copy cloner, that I found very useful to have incremental backups AND bootable backup... for MacOS. I did not find such an easy tool with mint).

That seems strange to me, because the problem was far beyond my understanding... and is always.

I use the chroot process to enter my "real LM ext4 root partition" from a live USB. Thanks to my mobile, I could have an USB cable internet connection.

I could take my own user by su myuser.
I tried an apt-get update, which gave a lot of errors : "no place on the device" and "signature error" with the sources
I mount again the root partition with mount -o rw, remount / (and don't know if it changed anything, I made it because the "no place on the device" message sounded odd, as if the / was read-only)
I made an apt purge lvm2, which seems to go through its whole process, then apt-get update and apt-get upgrade, that was able to process with no big errors and about 50 packages update.
I tried some other apt command, for example to purge a lot of old kernel packages that were useless. All of these apt command seemed to work (but may have been useless for my initial problem).

I don't know if it is the apt purge lvm2 or the apt-get upgrade that changes something in the system file, but after the exit from chroot process, I reboot, and my normal session is now running with my normal user. I will perhaps end in reinstalling because of some others problems... but not because of this one.

Thanks very much gm10 and Fabien85 for your time and wise advice. I am sorry not to have seen the one command line that saved my system, and not to be sure whether lvm2 was responsible for something. I guess that because of the crash there was at least one broken package that plays a role in the login process, and in the file system stability. Either removing lvm2 or updating makes the system right again. But I could not make this update from the recovery mode, that was deeply bothered by the unstability of the file system. In chroot mode, I could upgrade "from the inner side" without any problem.

thanks again,

Girafenaine

[gm10]

Good news. Shame we didn't figure out the exact cause, I would have liked to know, but as long as you're stable again that's good enough (did you check your dmesg whether those Dependency failed messages are actually gone?).

[Girafenaine]

Hello,

With dmesg I found following errors:

Code: Select all

[    2.621433] Couldn't get size: 0x800000000000000e
[    2.621438] MODSIGN: Couldn't get UEFI db list
[    2.649734] Couldn't get size: 0x800000000000000e
[    2.649736] MODSIGN: Couldn't get UEFI MokListRT
[    2.678024] Couldn't get size: 0x800000000000000e

and at the end :

Code: Select all

[   60.140695] cfg80211: Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7'
[   60.257207] PKCS#7 signature not signed with a trusted key
[   60.257227] nvidia: loading out-of-tree module taints kernel.
[   60.257238] nvidia: module license 'NVIDIA' taints kernel.
[   60.257239] Disabling lock debugging due to kernel taint
[   60.270116] nvidia: module verification failed: signature and/or required key missing - tainting kernel
[   60.287244] ACPI: PCI Interrupt Link [LPMU] enabled at IRQ 18
[   60.287266] nvidia: probe of 0000:00:03.5 failed with error -1
[   60.287291] nvidia 0000:02:00.0: enabling device (0002 -> 0003)
[   60.287627] ACPI: PCI Interrupt Link [LGPU] enabled at IRQ 17
[   60.287648] nvidia 0000:02:00.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=none:owns=io+mem
[   60.317859] [drm] Initialized nvidia-drm 0.0.0 20150116 for 0000:02:00.0 on minor 0
[   60.321364] NVRM: loading NVIDIA UNIX x86_64 Kernel Module  340.107  Thu May 24 21:54:01 PDT 2018
[   60.327507] intel_powerclamp: No package C-state available
[   60.342390] PKCS#7 signature not signed with a trusted key
[   60.361025] snd_hda_intel 0000:00:08.0: enabling device (0000 -> 0002)
[   60.361571] ACPI: PCI Interrupt Link [LAZA] enabled at IRQ 20
[   60.361582] snd_hda_intel 0000:00:08.0: Disabling MSI
[   60.548836] input: bcm5974 as /devices/pci0000:00/0000:00:04.0/usb3/3-6/3-6:1.2/input/input8
[   60.548990] usbcore: registered new interface driver bcm5974
[   60.679514] PKCS#7 signature not signed with a trusted key
[   60.680193] nvidia_uvm: Loaded the UVM driver, major device number 243
[   60.714258] ACPI Warning: \_SB.PCI0.IXVE.IGPU._DSM: Argument #4 type mismatch - Found [Buffer], ACPI requires [Package] (20170831/nsarguments-100)
[   60.793271] applesmc: key=271 fan=1 temp=14 index=14 acc=1 lux=2 kbd=1
[   60.899478] PKCS#7 signature not signed with a trusted key

With "dmesg | grep dependency" I had no output. Is there something else I should check ?

For the "size error" and the "nvidia error" above, do you have a clue whatabout I could repair, if it is of some importance ?

Girafenaine

[gm10]

With dmesg I found following errors:

This is about secure boot keys. Check your secure boot setting in your BIOS (disable it) and any particular settings about how it stores the keys. If you care, that is, this error probably won't affect your operation.

I'll need to look something up about the nvidia error, you may have a missing package but I don't recall its name right now. But disabling secure boot should also take care of the errors.

[fabien85]

Congrats !

The dmesg message about MODSIGN is a useless warning. I got it also, and it's fixed in some of the newer 4.15 kernel. You can just ignore it.
I dont know about the nvidia stuff. Maybe make a new thread for it.

I found very useful to have incremental backups AND bootable backup... for MacOS. I did not find such an easy tool with mint

This is exactly what timeshift does ! Incremental backups of your system, can be restored easily, even if the system doesnt boot (with a live USB). It has been made central in Mint 19 : when you first open it, the update manager should advise you to set timeshift up.

[Girafenaine]

I use timeshift, but only as a system backup system, on the same hard disk than the root partition. So in my case (and when system doesn't boot) it is useless.

But timeshift does not give bootable backups, does it ?

And is it well suited for documents backup too ?

OK for the error messages.

[gm10]

I use timeshift, but only as a system backup system, on the same hard disk than the root partition. So in my case (and when system doesn't boot) it is useless.

But timeshift does not give bootable backups, does it ?

And is it well suited for documents backup too ?

1. You can restore a timeshift snapshot from a live USB no matter if your system boots or not. Doing it that way is even the safest way to do it. Timeshift is pre-installed in the live USB.

2. Theoretically you can chroot into a snapshot but that would modify it so it's not something I would recommend or even give you instructions on how to do.

3. Not really, because restoring a system snapshot would also restore your documents to the point in time the snapshot was taken, which is something you usually do not wan to happen.

[Girafenaine]

Wow, thanks, I was not aware of Timeshift abilities.

If one's wants to restore a timeshift snapshot from a live USB, one's has to boot on its live USB, and then find the snapshot file in its file system ? or from an external backup drive ?

For documents, I can figure out another snapshot from time to time, not everyday like for system files snapshot.

[gm10]

If one's wants to restore a timeshift snapshot from a live USB, one's has to boot on its live USB, and then find the snapshot file in its file system ? or from an external backup drive ?

For documents, I can figure out another snapshot from time to time, not everyday like for system files snapshot.

No, just start timeshift, select the partition the snapshots are stored on and get ready to restore.

Regarding documents, personally I think the risk of making a mistake is too great. Better use a another, if similar, tool for documents, e.g. https://github.com/bit-team/backintime (use the PPA version if you choose to use this).

[Girafenaine]

OK, so... I could have solved my problem in this much easier way...

OK for documents, even if I don't understand well why it should be safer with another tool, it this one does the same kind of backup. Is it to have different snapshots for documents and for system ? Can't you do two diffents snapshots with the same tool, timeshift for instance ?

Girafenaine

[gm10]

OK for documents, even if I don't understand well why it should be safer with another tool, it this one does the same kind of backup. Is it to have different snapshots for documents and for system ? Can't you do two diffents snapshots with the same tool, timeshift for instance ?

Yes, it's only to have different snapshots. It's possible to do with timeshift if you really want to. You'd need two different /etc/timeshift.json files, one for snapshot location A including only the system, and one for snapshot location B containing only your user data. Then a cronjob that switches them out every other hour (timeshift's own cronjob runs hourly) so timeshift can take automated snapshots for both.

[Girafenaine]

OK, thank your for your explanations... It seems not as easy as I thought and I understand why you suggest to use another tool for another snapshot.

Girafenaine

[Girafenaine]

Hello,

Actually after a few happy days I got aware that I had my fstab with some commented lines. I could not access my documents, which are on my shared hfsplus partition.

So I tried to uncomment these lines. And then at the rebooting, I had the same « login loop » than before ! Argh.

Thanks to what you taught to me, I was able to use timeshift and came back to a recent snapshot. After I tried a few snapshots, the system booted normally. Strangley in this system the fstab was not the same than before, with only one line for the root partition and nothing for swap, efi and documents.

But i had still my documents partition to mount. I can mount it after my desktop is loaded with the « disks » application, but the partition is still in read only mode. All my attempts to add a line in fstab to automount the documents partition at the startup ended with a « login loop », preventing me to enter my own session.

So this Documents partition seems to have problems. Fsck gives no error. With fdisk -l I get only « partition does not start on the physical sector » or something like that. Any ideas to let it be mounted without corrupting my login ? By the way I can not figure out why this partition has an effect on my login, since documents are not involved at this step.

Girafenaine

[fabien85]

Hum, strange stuff, and I guess quite annoying.

I can mount it after my desktop is loaded with the « disks » application, but the partition is still in read only mode.

Do you get a message that the partition is mounted read-only or is it a permission problem ?
Like if you become root, can you write to the partition ?
Please provide the output of

Code: Select all

mount | grep sd

after having mounted the partition.

If it's not a permission problem, I personally dont know.
I already said that my experience with sharing an hfs+ partition is bad, so I would recommend backing up all the data there, then (from macOS) reformat the partition as FAT32 and copy back the data.
Note that FAT32 has a limit of 4G per file. That's normally fine, but in case you have larger files try exFAT instead of FAT32 (Mint supports read-write etc on exFAT, so most things work perfectly. It's just that gparted doesnt support exFAT for copyright reasons I dont understand).

[Girafenaine]

Hello Fabien85,

Actually when I mount this partition with the "disks" application, It can be written, but only with a root access. I am the user who owns all the documents folder, yet in the file manager (thunar) I cannot see the folder content without a root access. With a root access I have no problem. So it could be a permission matter as you suggested.

Here is the requested output :

Code: Select all

joe@joe-MacBook:~$ mount | grep sd
/dev/sda3 on / type ext4 (rw,relatime,errors=remount-ro,data=ordered)
cgroup on /sys/fs/cgroup/unified type cgroup2 (rw,nosuid,nodev,noexec,relatime,nsdelegate)
gvfsd-fuse on /run/user/501/gvfs type fuse.gvfsd-fuse (rw,nosuid,nodev,relatime,user_id=501,group_id=501)
/dev/sda5 on /media/joe/Documents type hfsplus (rw,nosuid,nodev,relatime,umask=22,uid=501,gid=501,nls=utf8,uhelper=udisks2)
/dev/sda1 on /media/joe/EFI type vfat (rw,nosuid,nodev,relatime,uid=501,gid=501,fmask=0022,dmask=0022,codepage=437,iocharset=iso8859-1,shortname=mixed,showexec,utf8,flush,errors=remount-ro,uhelper=udisks2)
/dev/sda2 on /media/joe/Systeme type hfsplus (ro,nosuid,nodev,relatime,umask=22,uid=501,gid=501,nls=utf8,uhelper=udisks2)
gvfsd-fuse on /root/.gvfs type fuse.gvfsd-fuse (rw,nosuid,nodev,relatime,user_id=0,group_id=0)

And for fdisk -l :

Code: Select all

joe@joe-MacBook:~$ sudo fdisk -l
[sudo] password for joe: 
Disk /dev/sda: 465,8 GiB, 500107862016 bytes, 976773168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: 9D37B812-7731-4501-BACA-5FBE283AD1D9

Device         Start       End   Sectors   Size Type
/dev/sda1         40    409639    409600   200M EFI System
/dev/sda2     409640  61962704  61553065  29,4G Apple HFS/HFS+
/dev/sda3  895170560 970754047  75583488    36G Linux filesystem
/dev/sda4  970756096 976771071   6014976   2,9G Linux swap
/dev/sda5   61962705 895170559 833207855 397,3G Apple HFS/HFS+

Partition 5 does not start on physical sector boundary.
Partition table entries are not in disk order.

I don't yet understand what's wrong with this documents partition, and how it can block the login process.

Right now I am on my LM xfce session, after a new timeshift recovery.

Girafenaine

[Girafenaine]

I saw that for the last timeshift snapshots ( after the crash and my login loop), there are data to write on /dev/sda3 (root partition) only. /boot is on dev/sda3 and location for /efi/boot is not detailed.

For the older snapshots, there are data to write on /dev/sda1 (EFI partition) for /efi/boot. /boot is still on /dev/sda3. By the way I was not able to restore these snapshots. At the end the process I have a « completed with error » message, and then I cannot boot. The boot leads me to a tty shell.

Girafenaine

[Girafenaine]

Because of this efi/boot location clue, I made a boot-info file :

http://paste.ubuntu.com/p/ttr3Sj2fww/

The disk type is "gpt".
On /dev/sda1 there are the following files :

Code: Select all

Boot files:  /EFI/ubuntu/grub.cfg
		/EFI/BOOT/fbx64.efi 
        	/EFI/tools/gptsync_ia32.efi
        	/EFI/ubuntu/fwupx64.efi 
        	/EFI/ubuntu/grubx64.efi
        	/EFI/ubuntu/mmx64.efi 
        	/EFI/ubuntu/shimx64.efi

Could you have a look there and tell me whether the suggested repair looks good ? It is about "reinstall the grub-efi-amd64-signed of sda3, using the following options: sda1/boot/efi". I guess it could help getting back an efi/boot on sda1. I don't know if it can explain why my fstab file prevented my login.

Girafenaine

[fabien85]

I dont think a boot repair will have any effect. It will reinstall grubx64.efi on the EFI partition. But grub is already present, you already boot with it and you (now) have no problem booting.

Looking at your mount output, the partition is mounted read-write. Furthermore you can write if you are root.
So this looks to me like a permission problem.

Looking at your mount output and your boot-info log, I am getting a lot of confusion :
- in your mount output, your user is joe, but in your boot-info, there are references to /home/etienne
- in your mount output sda5 is mounted with uid=gid=501 (which is usually the first user on macOS), but in the boot-info it's with uid=gid=0 (which is root)
- in the boot-info, sda5 is mounted 5 times at different locations. Not sure what' going on there.

To have permission to write on the partition, you must use a user with uid compatible with the mounting option.
If the partition is mounted with uid=gid=0, then only root can write. If the partition is mounted with uid=gid=501, then root can write but also a user with uid=501. Normally the first user you create in Mint has uid=1000 (second has 1001, etc). So your Mint user should not be able to write to the partition in any case. Unless you made the funny thing to change your uid. (I remember I tried that once, it failed miserably as I was unable to boot afterwards)
Could you give your uid and gid ?

Code: Select all

id
id etienne
id joe