story of multi-boot problems

Questions about Grub, UEFI,the liveCD and the installer
Forum rules
Before you post read how to get help. Topics in this forum are automatically closed 6 months after creation.
Locked
Dirkoir
Level 4
Level 4
Posts: 324
Joined: Wed Jul 30, 2014 12:43 pm

story of multi-boot problems

Post by Dirkoir »

Booting with liberty has become so barricaded nowadays. It is so sad and frustrating. Here are my experiences:

Back in the 80's, our operating system (OS) sat on a floppy disk. If you wanted to boot with a different OS you simply switched the floppy disk. Done. No problems whatsoever.

When hard drives came around, it could get more complex (in theory) because internal hard drives could not be switched so easily. (you had to open the computer, often with a screwdriver, and switch the cables). External hard drives were still quite easy to switch. And me, back then focusing on Macintosh computers (BEFORE the OSX), had it easy, too, to switch versions of the Macintosh OS. They were merely folders that could be easily copied in the Finder (the equivalent of our Nemo on Linux Mint), both to other drives and even the very same drive, very same partition. And you could easily tell your computer which of these folders you wanted to get used as the OS to boot from. With SCSI numbers you could arrange the device priorities.

In the 2000's I was forced to use Windows and used it as a single OS. Hence no possible trouble with competing operating systems. One problem I was frustrated by was the registry that didn't keep installed applications fully separate from the installed operating system (like it had been in the pre-OSX Macintoshs (not sure how it is now on OSX Macs). So, an OS update or switch left the previously installed applications unharmed, running fine as long as they were compatible with the OS version. Liberty had still existed back then. Much less on Windows. (Linux, of course, has a similar problem by merging apps with the OS -- sure, I think, this also gets used for letting applications interact with each other usefully (pipes that can be handy; still having to reinstall and reconfigure all applications after upgrading or switching an OS version is a lot of hassle, though)

Still, when I switched to Linux Mint in 2014, multi-boot was still no problem. I installed it on an external drive (YES! One connected via a USB port!) and could boot from it on two laptops which internally had Windows installed. In the BIOS (no UEFI mess, yet, on one laptop) and even the UEFI interface (modern BIOS equivalent) on the newer laptop (the one I still use as my daily super-necessary main laptop), I could simply set up to prefer booting from any attached external drive, and things went well like they had in the 80's and 90's. I was very happy. And, yes, I could even have multiple Linux OSs on that external drive, and when the laptop booted from the attached external drive automatically, I was given the startup menu from which to pick the Linux OS I wanted to boot from (there was even a tool to manually adjust the GRUB menu usefully). Liberty. Happiness. (and an instant solution after the internal drive died on the older laptop (that is gone now) -- simply boot it from the external USB drive)

But after I installed LM 17 in 2014 or 2015 on my current main laptop's internal drive alongside the pre-installed Windows 8 (a multiboot which also worked well), I soon ran into first problems. Namely cloning the internal hard drive onto an external USB hard drive specially bought for having a backup, I could not test the clone via booting from it. On the same laptop the UUID mess interferes with it (the laptop ignoring devices, only looking for UUIDs (partition IDs) which on a Clonezilla disk-clone are OF COURSE identical (and should probably stay that way for the partitions' interactions and for cloning backwards when restoring a corrupted drive or setting up a replacement drive with the already well configured Linux Mint one backed up as a clone) -- what were the developers thinking when they skipped over ports and especially devices so easily used to distinguish clones from originals?!?). More sadly, booting from this new external drive didn't even work on a new secondary laptop bought, despite the UEFI IDs being different, of course. It would only boot Windows from it (from the first clone; when I made a second clone on a second new external USB hard drive, even that would not work -- Why?!?).

Then, when I booted from the old external USB hard drive -- the one from 2014/2015 -- the one that had always worked, it worked fine the first time, but then suddenly something happened to it in that process, so it can no longer boot from it anywhere. (I suspect it had still an MBR rather than GPT partition table and perhaps no UEFI setup, but how that got it attacked and messed up by one of the more modern laptops, no idea)

I then experimented on the second laptop (the "travel laptop" which is not daily needed by me, and which originally had only one OS installed (LM 19 Cinnamon)). I shrank its LM 19 partition and installed LM 20.1 MATE alongside on its internal HD. The GRUB2 menu then had set the new experimental LM 20.1 MATE installation as the default boot (called "ubuntu", I think), something I managed to switch back to the LM 19 already configured for family members -- with a manual GRUB2 list items shift using the boot-repair tool (the Lm 20.1 MATE OS oddly named "Ubuntu", by the way, and the Lm 19 suitably named without my having to try to name it that way, as far as I recall -- unsure if I could rename these names).

I tried to get the external clones to work. Booting the "travel laptop" from the internal drive into either one of the LM OSs (19, 20.1) and then in the terminal give the command update-grub successfully added the external USB OSs (Windows & and LM 19) from a main laptop clone (plus an unnecessary bunch of ancient other drives/OSs the previous laptop owner(s) must have had (yes, I bought the laptop used), OSs that no longer exist on my end). Sadly, neither the Windows 8 nor LM 17 OS existing on the clone could actually boot when picked by me in the boot menu, the laptop giving me the response that the 'device' (disk) does not exist (despite of it being attached (even on the same port!)). (repeating update-grub and wriggling with UEFI interface settings (commonly still called BIOS when I read it no longer is BIOS) fixed nothing; copying some of the /boot/.../EFI contained boot loader files to the /boot/.../BOOT (or /boot/.../GRUB??? not sure anymore) folder did not help either (something I had read during my online searches had helped someone else) --- btw. these folder locations seem to be link-wise backdoor entrances into the contents of the ESP/efi partition from within a successfully booted Linux OS rather than some duplicate of ESP/efi partition contents)

To return to the good age of utter liberty (in other words, wishing to start any laptop and get a boot menu with ALL the operating systems from ALL its internal and external hard drives and be able to boot from any I pick), I then installed rEFInd on this "travel laptop" (via LM's Software Manager) hoping it could let me also boot from external USB hard drives to finally become able to again test or use OS clones and to possibly be able to avoid the mess of having to try to manually set up boot opportunities for additional OSs -- these setups being complex and time consuming efforts that sadly can result in getting previous OS installations unwillingly blocked from booting or later make drives unbootable when their ESP/efi partition was placed on another drive that has afterwards died or been removed. But, sadly, I now got mysteriously hit with a MOK boot blockade even though I have Secure Boot disabled. Another unsolved mystery. I had to de-prioritize rEFInd in the listing of boot OSs listed in the UEFI shell interface ("InsydeH2O Setup Utility", a modern replacement of traditional "BIOS" apparently) in order to make the "travel laptop" bootable again. *siiiiigh*

I am still unsure how to best add a new OS (LM 20.1 MATE) to my "main laptop" that I need to use daily for important work, the laptop that therefore needs to maintain its already existing OSs (Windows 8 and LM 17 Cinnamon) (Windows 8 only in case I must ever use it or even update it to Windows 10 -- let's hope neither, but LM 17 because of the many tweaks and installations that took months over years and cannot be quickly re-created in a new OS installation (LM 20.1 MATE)). I can only install a new Linux Mint on an added hard drive (already added via replacement of the optical drive) since the original internal hard drive has no room. PLUS, it may die soon. Infos I have found and tips been given so far still leave me unsure. On one hand, installing LM 20.1 MATE with the ESP/efi partition on the original internal drive being used for its boot loader would give me, when I start the laptop, a GRUB2 menu hopefully having all three OSs listed and bootable (and a future 4th one, as well). But when the old drive dies, even the LM 20.1 MATE OS on the added new drive would no longer boot, because its boot loader would have been lost. And would the "InsydeH2O Setup Utility" (today's BIOS equivalent) let me do anything? Or boot-repair run from a live CD/DVD (my LM 20.1 MATE installer, for example)? Alternatively, might it be better to get the boot loader installed on the new added hard drive (it getting its own ESP/efi partition), then never getting a GRUB2 Menu that would give me all three options, though? (plus some online posts warn against having more than one ESP/efi) --- Or could I update that GRUB2 menu somehow to successfully include the older drive's OSs as boot options (maybe by using the "boot-repair" tool when booted from LM 20.1 MATE on the added drive which most likely would be the default boot event after the installation -- or possibly by somehow copying the ESP/efi partition's contents over to the new ESP/efi partition to merge both GRUB2 menu listings -- two actions with which so far I have only had marginal successes in my experiments)? --- I also recall having read that if I wish to get the boot loader for the new OS installation (LM 20.1 MATE) -- that I install on the added new drive rather than the older drive -- likewise installed in an ESP/efi partition on that same new drive, I must first remove the esp & boot flags from the older drive's ESP/efi partition because of a current bug in the LM installer making it pick the first ESP/efi partition it finds (which would be the old drive), rather than the one selected by me (another complexity I could possibly forget to do resulting in the end of our universe). *sigh*

Yes, I am really worried that finally getting around to install the current LM version (which is highly necessary) will either kill the old one or make it unbootable, or the new installation won't be bootable, or it will become unbootable after the old drive dies. Further worries: even if the new OS installation (LM 20.1 MATE) gets to boot, AND then running the "boot-repair" tool I may even get bootability of the older OSs from the older drive merged into the added new drive's ESP/efi partition and GRUB2 menu, something else may get messed up again. One "Ubuntu" may replace another one in the BIOS-equivalent (NVRAM and/or "InsydeH2O Setup Utility") rather than having both still as options, for example (plus former owner's OS references being copied over to possibly create trouble behind the curtains even when not cluttering the GRUB2 Menu at least).

The sad thing is: I can clone (backup) my internal hard drive (and have already done so). But I cannot properly test the clone (by booting from it; since that boot has somehow become impossible even on another laptop). And I cannot back up the NVRAM (the UEFI/InsydeH2O/BIOS hardware storage of... uhm... whatever little the hardware knows... like references of where on which drive to find the boot loaders) in case the NVRAM gets messed up by the LM 20.1 MATE installer or perhaps the "update-grub" or "boot-repair" tools, or even "efibootmgr"). And the splintering of (these hardwired *sigh*) setups across the NVRAM plus an ESP/efi partition plus perhaps a second ESP/efi partition seems to create confusion not only for me but also my laptops, regularly removing available OSs from the boot menu (GRUB2 menu) and InsydeH2O boot listings or having them listed BUT not letting them get booted. So, when these things get messed up, my clone (internal older hard drive backup) will surely not help me. So, if things go wrong, there is no way to go back. *siiiigh*

And then -- *ugh* -- another warning I came across: that kernel upgrades can also mess up multi-boot even after it has been made to work.

Anyway, I am not the only one having suffered from this modern chaos. (see for example viewtopic.php?p=1801963 and viewtopic.php?p=1979681)

Someone coming up with a nice solution would be great (but seems impossible to me by now, there seeming too many chaotic hurdles and bugs in hardwares and softwares nowadays). So, this post is not a question but a story that others could add their own stories to. Maybe this would eventually even lead to recoveries or great tips from insightful experts stumbling across this. Who knows. At the very least, when I ask a small question in the future to help me fill the gaps in my UEFI & MOK chaos jungle exploration, and someone wonders what the bigger picture is on my end, I can refer to this story that summarizes my events otherwise scattered across several posts.

;-) dirkoir
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Important vocabulary as far as I understand it so far:

Hardware thingies:
  • BIOS = traditional (and dependable) startup code and prioritized list-storage firmware now apparently replaced with (chaotic) UEFI firmware, in my case calling itself "InsydeH2O Setup Utility" -- but often still referred to as BIOS by people
  • NVRAM = the hardware RAM storage of what little or too much the UEFI firmware stores on the hardware to then interact or conflict with what's stored on the boot drives
  • mobo = some people's abbreviation of motherboard
Boot drive thingies:
  • MBR = partitions table on old smaller drives, now usually no longer used
  • GPT = the now common partitions table (replacing MBR)
  • ESP also known as efi partition (hence referred to by me as "ESP/efi partition") = "EFI System Partition", a FAT32 formatted partition on a boot drive containing boot loaders (or even a boot manager like rEFInd, as well) plus potential other utilities used by modern UEFI firmware and stored there on some drive rather than its own NVRAM (*shudder* UEFI firmware talk: "Which drive is it again that has my stuff? This one? Or that one? Or the dead one that's long gone? ..." Yep, confusion preprogrammed by design...)
  • UUID = "universally unique identifier" (a joke when considering clones) an id given to a partition to address it and distinguish it from other partitions
  • GUID = some other partition ID apparently; no idea how they differ from UUID and which of the two I typically see in tools.
Tools:
  • update-grub
  • boot-repair
  • efibootmgr
Last edited by LockBot on Wed Dec 28, 2022 7:16 am, edited 2 times in total.
Reason: Topic automatically closed 6 months after creation. New replies are no longer allowed.
______
Linux Mint 17 Cinnamon 64-bit | LM19 Cinnamon | LM20.1 MATE | LM20.3 Cinnamon
Bertelh09
Level 1
Level 1
Posts: 5
Joined: Wed May 13, 2020 11:30 am

Re: story of multi-boot problems

Post by Bertelh09 »

Hi Dirkoir,

right - multiboot IS a never ending story...
(but you never thought about virtualization or even used it ?)

You asked for a "nice solution" --> here's my version (how I did it) - today:

the basics:
a) only use a mainstream PC that supports Linux - NO gaming PC, no home series...
(all major business PC manufacturers do, like HP / Lenovo / Fujitsu or Dell)
b) plan - what you need NOW, what you probably need in the next software generation (2-3 years)
and how to implement this - before creating the disk structures
c) all OS tests are virtual machines
d) every PC (and 1 Laptop also) has 2 physical disks:
SSD for (multi) OS, 256 or 512 GB
HDD for Data (incl. virt. machines), 1 TB...
e) no user data (except thunderbird mail profile) is stored on OS device
-> all data is on data HDD
f) no secure boot of any WIn installation
g) avoid GPT part. and UEFI boot - as long as possible

Just looking at my "private" PCs (I have also business PC & Laptop): 2 PCs and one Laptop
1. productivity PC
a) 256 GB SSD, MBR, (3 prim. + 4 extended, incl. swap)
Win 8.1 (loader & system), 100 GB - just because of tax / banking etc.
Mint 18.3 Mate work (= my system for all), 40 GB; will be upgraded soon (= new installed) to Mint 20.1
Mint 20.1 Cinnamon virt (no office etc., VMware Workstation pro), ~100 GB with 2 virt. machines
b) 1 TB HDD, MBR (only prim)
Win software (ntfs), 50 GB, programs & some drivers
Win data (ntfs), 100 GB, tax & banking data
linux data (ext4), 800 GB, doc, images, audio, video, ~ 5-6 virt. machines / used = production ones
first-backup (ext4), 50 GB, linux only (daily backup of main data)

2. Test-PC (virtualization platform)
a) 500 GB SSD, MBR (4 prim incl. swap)
MS Win Server 2016 std (loader & system) with HyperV and 2 virt. machines, 250 GB
Mint 20.1 Cinnamon (VMware player, free), 250 GB, 6-7 virt. machines
b) 500 GB HDD with MBR (3 prim incl. swap)
HyperV machines (ntfs), 250 GB (test vms)
VMware machines (ext4), 250 GB (test & old vms)

Laptop
...very similar to production PC
data (if needed) is transferred between PC and laptop

therefore I can use for tests by now - independent from my live systems:
Win 3.1, 95, 98, NT, 2000
Win XP pro (production: sw for small telephone switchboard)
Win 7 pro (production: old tax sw)
Win 8.1 pro (testing Win sw)
Win 10 pro (testing...)
Ubuntu (KDE, Gnome...)
Mint Cinnamon, XFCE...
Debian 8,9,10,11
SLE...

-- my rolling system...
lasts at least 3 years (max. lifetime of Win or Linux)
then reinstall (one os in one partition) is needed (the disc structures remain)
after os installation test on Test-PC in a virt. machine
-- if WIn install. --> boot repair from live stick afterwards (grub repair)...
if Linux install --> boot repair...
-- as there is no user data on the OS SSD, there is very lttle to be backup-ed (thunderbird mail profile & program cfg.s);
/home directory is on os partition.

If you'd like to know more details - just ask.
(there is also a dualboot on 1 hd running: Win10 - Mint 19.3 UEFI config on laptop;
and a tripleboot Win8.1, Win10, Mint 19.3, MBR, ssd + hdd, PC)
bendipa
Level 5
Level 5
Posts: 579
Joined: Sat Jan 04, 2020 8:02 pm
Location: NW London. UK

Re: story of multi-boot problems

Post by bendipa »

Dirkoir wrote: Sun Jun 13, 2021 12:56 pm But after I installed LM 17 in 2014 or 2015 on my current main laptop's internal drive alongside the pre-installed Windows 8 (a multiboot which also worked well), I soon ran into first problems. Namely cloning the internal hard drive onto an external USB hard drive specially bought for having a backup, I could not test the clone via booting from it.
Well, there's one issue you could avoid altogether. When backing up your system, don't clone anything. A lot of people here like to use Timeshift which seems to work well for them. I don't happen to like it much as it can fill your drive up pretty quickly if you are not familiar with it, or don't configure it properly. But Disks which comes with Mint allows you to make system images and works very well. You will need to make one image file (.img) of your root partition on your back-up drive. If you have a separate Home partition that needs to be imaged too. You also need to make a one-off back-up of your ESP (efi system partition, if on UEFI) and that's basically it. You shouldn't need to test what you have backed up. You can't directly boot up from an .img file AFAIK, but you can mount it and examine the files within and transfer them to a USB, make an entry for your USB in your grub (on your hdd/sdd), and boot from the USB entry if you're that earnest. But you're just making unnecessary work for yourself there. To date I've never tested an image file and I've restored from them several times over the years.
Computer: Dell Vostro 470
Systems: Linux Mint 21.2 Xfce (Victoria), Manjaro 23.1 Xfce (Vulcan), Windows 10 (22H2) Pro.
Dirkoir
Level 4
Level 4
Posts: 324
Joined: Wed Jul 30, 2014 12:43 pm

Re: story of multi-boot problems

Post by Dirkoir »

Thanks for these nice responses (that I am currently too busy to dig deep enough into and respond to). :-)

Today I followed well-given (I think) instructions (viewtopic.php?p=2023951#p2023951) as much as possible, but sadly was stabbed in the back nevertheless (and am so far helped only by rEFInd which on my main laptop did NOT result in a block via blue screen, like it sadly had resulted on the travel laptop --- so much unpredictability nowadays). :?

I have placed my report of this in my older thread where I asked for multi-boot suggestions but remained quite confused: viewtopic.php?p=2027976#p2027976

At least that thread now can show some success, but one definitely imperfect since I was harmed despite careful attempts to avoid the harm and only found a backdoor rescue that could have failed like it failed on my other laptop before. :?
______
Linux Mint 17 Cinnamon 64-bit | LM19 Cinnamon | LM20.1 MATE | LM20.3 Cinnamon
Dirkoir
Level 4
Level 4
Posts: 324
Joined: Wed Jul 30, 2014 12:43 pm

Re: story of multi-boot problems

Post by Dirkoir »

Bertelh09 wrote: Wed Jun 16, 2021 3:11 pm right - multiboot IS a never ending story...
(but you never thought about virtualization or even used it ?)
Yes, Bertelh09, I sometimes think about that option, too, actually having used VirtualBox for almost as long as Linux Mint. It has given me its own trouble, though. Things like: online video chats not working in them, smooth interaction between host and guest OS (like clipboard crossover and shared folder) not working in every case, and my old Windows XP clone needing new MS registration when used on a different computer, ...

In addition, the virtual machines (VMs) can be too slow on a not very new and expensive host computer and/or need more RAM on it. So some of these issues tend to push me onto the multi-boot path. If I had more money to spend and didn't at least in some cases make things like online video chatting work, I'd probably stick with VMs like you do (heck, backups of VMs can actually be tested :) ). It's a juggling with pluses and minuses.


(Funny, or rather not, that it took so long before I could review these thread comments with a bit more time.)
______
Linux Mint 17 Cinnamon 64-bit | LM19 Cinnamon | LM20.1 MATE | LM20.3 Cinnamon
Dirkoir
Level 4
Level 4
Posts: 324
Joined: Wed Jul 30, 2014 12:43 pm

Re: story of multi-boot problems

Post by Dirkoir »

bendipa wrote: Wed Jun 16, 2021 5:38 pm Well, there's one issue you could avoid altogether. When backing up your system, don't clone anything. ... Disks which comes with Mint allows you to make system images and works very well.
Hmm... interesting suggestion. The popular use of disk/partition "images" nowadays is one of my mysteries since I was skipped out of major computer work when this new thing became a new tradition. I really would like to come across an explanation some day of what has made these "image" backups so popular.

I suppose that if the "Disks" program, when making an "image", would check the correctness of its data copies, then testing via booting would probably be quite useless. And, OTOH, perhaps a boot or virtual run of such an image might even be a possibility?

Being able to boot from a clone, btw, is not only fairly useful for checking the backup success, but a great quick help when one's drive or OD breaks down at a time when one has to quickly finish one's work or chores before one can take time to restore the broken computer.


Thanks to both fellows for your comments. :-)
______
Linux Mint 17 Cinnamon 64-bit | LM19 Cinnamon | LM20.1 MATE | LM20.3 Cinnamon
Locked

Return to “Installation & Boot”