Page 1 of 1

LMDE2 > LMDE3: Systemd boots in recovery mode - a start job is running for...

Posted: Sun Apr 21, 2019 6:29 pm
by Ottavio
Hi all,

I've upgraded my LMDE2 MATE to LMDE3 Cinnamon+Mate using this tutorial and some great tips here

Upon booting, systemd gets stuck in a loop for about 3 minutes and complains that "a start job is running for" ... a few devices.

This is all the information I can give you:

Code: Select all

oc@e130 ~ $ dmesg|grep sda
[    2.697546] sd 0:0:0:0: [sda] 976773168 512-byte logical blocks: (500 GB/466 GiB)
[    2.697551] sd 0:0:0:0: [sda] 4096-byte physical blocks
[    2.697614] sd 0:0:0:0: [sda] Write Protect is off
[    2.697619] sd 0:0:0:0: [sda] Mode Sense: 00 3a 00 00
[    2.697646] sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[    2.764401]  sda: sda1 sda2 sda3 sda4 sda5 sda6
[    2.765558] sd 0:0:0:0: [sda] Attached SCSI disk
[    4.407140] EXT4-fs (sda5): mounted filesystem with ordered data mode. Opts: (null)
[    7.933042] EXT4-fs (sda5): re-mounted. Opts: errors=remount-ro

Code: Select all

oc@e130 ~ $ systemd-analyze
Startup finished in 2.606s (firmware) + 1.787s (loader) + 5.307s (kernel) + 2min 11.807s (userspace) = 2min 21.510s

Code: Select all

oc@e130 ~ $ journalctl -b 0|grep -i dependen
Apr 21 23:09:20 e130 systemd[1]: Dependency failed for File System Check on /dev/disk/by-uuid/BD49-37B5.
Apr 21 23:09:20 e130 systemd[1]: Dependency failed for /home/oc/storage.
Apr 21 23:09:20 e130 systemd[1]: Dependency failed for Local File Systems.
Apr 21 23:09:20 e130 systemd[1]: Dependency failed for Clean up any mess left by 0dns-up.
Apr 21 23:09:20 e130 systemd[1]: dns-clean.service: Job dns-clean.service/start failed with result 'dependency'.
Apr 21 23:09:20 e130 systemd[1]: local-fs.target: Job local-fs.target/start failed with result 'dependency'.
Apr 21 23:09:20 e130 systemd[1]: local-fs.target: Triggering OnFailure= dependencies.
Apr 21 23:09:20 e130 systemd[1]: home-oc-storage.mount: Job home-oc-storage.mount/start failed with result 'dependency'.
Apr 21 23:09:20 e130 systemd[1]: systemd-fsck@dev-disk-by\x2duuid-BD49\x2d37B5.service: Job systemd-fsck@dev-disk-by\x2duuid-BD49\x2d37B5.service/start failed with result 'dependency'.
Apr 21 23:09:20 e130 systemd[1]: Dependency failed for File System Check on /dev/disk/by-uuid/1A32-F445.
Apr 21 23:09:20 e130 systemd[1]: Dependency failed for /boot/efi.
Apr 21 23:09:20 e130 systemd[1]: boot-efi.mount: Job boot-efi.mount/start failed with result 'dependency'.
Apr 21 23:09:20 e130 systemd[1]: systemd-fsck@dev-disk-by\x2duuid-1A32\x2dF445.service: Job systemd-fsck@dev-disk-by\x2duuid-1A32\x2dF445.service/start failed with result 'dependency'.

Code: Select all

oc@e130 ~ $ journalctl -b 0|grep -i timeout
Apr 21 23:09:20 e130 systemd[1]: dev-disk-by\x2duuid-BD49\x2d37B5.device: Job dev-disk-by\x2duuid-BD49\x2d37B5.device/start failed with result 'timeout'.
Apr 21 23:09:20 e130 systemd[1]: dev-disk-by\x2duuid-1A32\x2dF445.device: Job dev-disk-by\x2duuid-1A32\x2dF445.device/start failed with result 'timeout'.
Apr 21 23:09:55 e130 NetworkManager[910]: <info>  [1555884595.2392] dhcp4 (wlan0): activation: beginning transaction (timeout in 45 seconds)
oc@e130 ~ $ 

Code: Select all

oc@e130 ~ $ cat /etc/fstab
# UNCONFIGURED FSTAB FOR BASE SYSTEM
proc	/proc	proc	defaults	0	0

# /dev/sda2
UUID=1A32-F445	/boot/efi	vfat	defaults	0	2

# /dev/sda5
UUID=1d37629d-4503-4c46-b8c0-450f29587610	/	ext4	rw,errors=remount-ro	0	1



#Added sda6 to storage
# changed to fsck=2
# Added fmask=111
UUID=BD49-37B5  /home/oc/storage        vfat iocharset=iso8859-1,uid=1000,gid=1000,fmask=111    0       2

Code: Select all

oc@e130 ~ $ blkid 
/dev/sda1: LABEL="Recovery" UUID="D2882FD7882FB93D" TYPE="ntfs" PARTLABEL="Basic data partition" PARTUUID="99d83961-f03b-4410-b081-279b47143740"
/dev/sda2: UUID="1A32-F445" TYPE="vfat" PARTLABEL="EFI system partition" PARTUUID="a86e7ab5-a10c-407c-bce6-0210f18cde72"
/dev/sda4: UUID="368637918637509B" TYPE="ntfs" PARTLABEL="Basic data partition" PARTUUID="49ba3b9c-17cd-4dee-a00b-ecccfeb35bd6"
/dev/sda5: UUID="1d37629d-4503-4c46-b8c0-450f29587610" TYPE="ext4" PARTUUID="2d814e00-17ef-4e6f-9e86-777c53a2c769"
/dev/sda6: UUID="BD49-37B5" TYPE="vfat" PARTUUID="4dd2b147-5426-465a-8deb-a62083ed4fbe"
It looks to me I have used the correct device id for all partitions in /etc/fstab and I'm not using any swap.

Any input will be appreciated.

Re: LMDE2 > LMDE3: Systemd boots in recovery mode - a start job is running for...

Posted: Mon Apr 22, 2019 2:55 pm
by Ottavio
I made a fresh install from the Cindy live cd and all has been sorted.

Yet, I'd be glad if somebody had a clue on why the manual upgrade didn't go as planned.

Re: LMDE2 > LMDE3: Systemd boots in recovery mode - a start job is running for...

Posted: Mon Apr 22, 2019 4:38 pm
by KBD47
systemd got itself worked up as swap got re-labeled somehow. I've had that happen to me with systemd on Debian, LMDE, and just happened on Mint 19 Cinnamon when I removed a second hard drive and systemd could no longer find swap on that drive.
Here is the solution for future reference:

Use sudo blkid to get your correct UUID's and look closely at the swap number. In the same terminal: sudo (whatever your text editor is)
then open your root, etc, fstab
compare the swap numbers. Your fstab swap number probably doesn't match the swap UUID that blkid turned up. Replace the exact UUID number that blkid turned up inside of fstab with your text editor, save it. Alternatively, if you have an extra swap showing up in fstab that's not in blkid numbers, then remove it and save.
Then run:
sudo update-initramfs -u
Reboot.

Re: LMDE2 > LMDE3: Systemd boots in recovery mode - a start job is running for...

Posted: Mon Apr 22, 2019 5:35 pm
by Ottavio
I don't have swap, so that's not the culprit.

I compared the UUID and they were correct. If there had been a UUID mismatch, I'd have found traces in dmesg, but there were none.

Something must have gone wrong during "mintupgrade install".

Lesson learned; a fresh install is always the best.

Now I have a working system with Cinnamon and Mate, although Mate doesn't look as good as in Betsy, because it's plain vanilla from Debian.

Re: LMDE2 > LMDE3: Systemd boots in recovery mode - a start job is running for...

Posted: Wed May 01, 2019 8:55 pm
by axisofevil
Have you tried creating a directory, say /media/oc/storage that you can mount sda6 on?
It's a bit confusing mounting a vfat filesystem onto a subdirectory of a ext4 filesystem.

If the present setup worked then /home/oc/storage is simultaneously an ext4 and a vfat file system.

Re: LMDE2 > LMDE3: Systemd boots in recovery mode - a start job is running for...

Posted: Thu May 02, 2019 4:27 am
by Ottavio
axisofevil wrote:
Wed May 01, 2019 8:55 pm
Have you tried creating a directory, say /media/oc/storage that you can mount sda6 on?
It's a bit confusing mounting a vfat filesystem onto a subdirectory of a ext4 filesystem.

If the present setup worked then /home/oc/storage is simultaneously an ext4 and a vfat file system.
One can perfectly mount a vfat filesystem on any directory. It's just a mount point. This is not the problem.

I then gave up and made a fresh install of Cindy with systemd. Fstab is the same (only the UUID changes). It's working perfectly now. I think the problem was with the manual update process as outlined here:
https://community.linuxmint.com/tutorial/view/2419