Legacy+UEFI USB with installed system + live ISOs capabilities

[fabien85]

This guide is NOT to create a live USB. In fact you will need to already have a live USB, plus a computer where Linux is already installed.
This is to achieve something similar to a live USB with persistence, but in some way better.
The main goal is to have a portable system that you can keep in your pocket e.g. for travel or general mobility, you can also install tools to diagnostic/repair boot problems, drive problems etc. Live capability is a side bonus, e.g. when you want to save someone from their outdated windows XP
Characteristics :
- Bootability : USB will be able to boot both in Legacy/BIOS mode, useful for older computers in particular, and in UEFI mode.
- Live capabilities : it will be possible to boot ISOs installed on the USB, in order to install your favourite distro(s) to some computer
- Advantage wrt to live USB + persistence : a full linux system is installed, as on a standard hard drive. So you can upgrade anything, including the kernel, xorg etc, or the whole system (e.g. LM18.2->18.3 when 18.3 comes out). Also if I understand correctly, live USB + persistence is broken for Mint>17.2 (or at least it does not work out of the box).
- Drawbacks : you need a larger stick than for a simple live USB. 16GB will do (8GB might if you dont install many programs and/or use a lightweight distro like lubuntu), more will give you more space for your files and programs.
The system on the live ISOs will be used only for installing linux on other computers, so you can consider it's a bit of a waste of space, and you need to reboot to get to the live ISO.

Table of contents :
A) Create a Legacy+UEFI USB
B) Fix some stuff
C) Add live ISOs : grub method
D) Add live ISOs : UEFI/refind method
E) Installing UEFI bootloaders manually
F) Credits/references

Foreword: to use this guide, you need to be familiar with command line, the linux boot process, and drive partitioning. If any of these are unfamiliar, start by reading the references gathered at the end (plus you will probably need to read more).

A) Create a Legacy+UEFI USB
The content of this section is drawn for the most part out of a thread by pbear on this forum : viewtopic.php?f=46&t=252351
You start booted on Linux on your computer, and plug the USB stick

A.1) USB partitioning
This will completely wipe the content of the USB, so backup any important data that was there. Then
- open gparted (in Mint : Menu > Administration > gparted. If you dont have it, you need to install it from the standard repositories)
- partition the USB as GPT (GUID Partition Table, where GUID = globally unique identifier)
in gparted select the USB in the right drop-down menu, make sure any partition there is unmounted (Partition > Unmount), then Device > Create Partition Table, Select new partition table type : gpt, Apply
- Make a bios_grub partition
Partition > New, Free space preceding : 1MiB, New size : 2MiB, File system : unformatted, Add, click the arrow Apply All operations > Apply, then select the partition, Partition > Manage flags, check "bios_grub"
- Make an EFI System Partition (ESP)
Partition > New, Free space preceding : 0MiB, New size : 10-50MiB, File system : FAT32, Add, click the arrow Apply All operations > Apply, then select the partition, Partition > Manage flags, check "esp" (that will also check "boot")
you can now quit gparted
- Install refind on the ESP at the fallback location
I like to do it manually by mounting the USB ESP (e.g. at /mnt), creating the directory EFI/BOOT/ and copying the files from the ESP of my computer onto the USB ESP, and renaming refind_x64.efi as bootx64.efi. This is described in greater details in section E.1.
Simpler option (or if you dont have refind installed on your computer) is to download refind, extract it, cd to its directory, and then

Code: Select all

./refind-install --alldrivers --usedefault /dev/sdc2

where sdc2 is the USB ESP (adapt if necessary). Optionally you can also install an EFI shell (if you dont know what this is, then you shouldnt try to install it). Result should look like this :

Code: Select all

$ ls -R /mnt
/mnt:
EFI  Shell.efi

/mnt/EFI:
BOOT

/mnt/EFI/BOOT:
bootx64.efi  drivers_x64  icons  keys  refind.conf

/mnt/EFI/BOOT/drivers_x64:
ext4_x64.efi

/mnt/EFI/BOOT/icons:
arrow_left.png       os_crunchbang.png  os_mageia.png     os_zesty.png
arrow_right.png      os_debian.png      os_mandriva.png   README
boot_linux.png       os_devuan.png      os_netbsd.png     tool_apple_rescue.png
boot_win.png         os_elementary.png  os_network.png    tool_fwupdate.png
func_about.png       os_fedora.png      os_opensuse.png   tool_memtest.png
func_csr_rotate.png  os_freebsd.png     os_redhat.png     tool_mok_tool.png
func_exit.png        os_frugalware.png  os_refind.png     tool_netboot.png
func_firmware.png    os_gentoo.png      os_refit.png      tool_part.png
func_hidden.png      os_gummiboot.png   os_slackware.png  tool_rescue.png
func_reset.png       os_haiku.png       os_suse.png       tool_shell.png
func_shutdown.png    os_hwtest.png      os_trusty.png     tool_windows_rescue.png
mouse.png            os_kubuntu.png     os_ubuntu.png     transparent.png
os_arch.png          os_legacy.png      os_unknown.png    vol_external.png
os_centos.png        os_linuxmint.png   os_win8.png       vol_internal.png
os_chakra.png        os_linux.png       os_win.png        vol_net.png
os_chrome.png        os_lubuntu.png     os_xenial.png     vol_optical.png
os_clover.png        os_mac.png         os_xubuntu.png

/mnt/EFI/BOOT/keys:
altlinux.cer                microsoft-pca-public.der               refind_local.cer
canonical-uefi-ca.der       microsoft-uefica-public.der            refind_local.crt
centos.cer                  openSUSE-UEFI-CA-Certificate-4096.cer  SLES-UEFI-CA-Certificate.cer
fedora-ca.cer               openSUSE-UEFI-CA-Certificate.cer
microsoft-kekca-public.der  refind.cer

why refind ? You could just go for grub,as in pbear's guide. I find refind useful because it auto-detects all bootloader on all drives attached to the computer. So it can help troubleshoot boot problems on some computer, see what bootloaders are there. For Macs, it provides many boot options to boot macOS (32-bit, single user mode, hardware test...) which are otherwise more difficult to trigger (you need to know the key combinations to be pressed at the right moment during boot) ; and it allows to manage System Integrity Protection (SIP). For UEFI PCs it allows to manage Machine Owner Keys (MOKs), and allows to boot systems on the hard drive which are not first in the boot order (e.g. if windows boots by default instead of the installed grub, or if grub fails to provide a windows entry). It also allows to load an EFI shell if you have one. It's like a swiss army knife for bootloading. And in general it provides a nicer graphical boot menu than grub.

A.2) Install linux to your USB
This is standard. You need a computer with at least two USB ports.
- Boot a live USB in Legacy/BIOS mode.
If the computer is UEFI-capable, make sure you are in Legacy mode by inspecting /sys/firmware, there should not be any efi/ folder there
- plug the target USB and find it's device name (eg with gparted, or inspecting /dev/sd* before and after plugging)
- if you want a particular partition setup on the USB (e.g. separate /home), now is the time to make it with gparted, alternatively you can do it from the installer
- launch the installer and install to the USB via the "Something Else" route, paying attention that "Device for bootloader installation" should point to the target USB (whole device, e.g. /dev/sdd)
I suggest to encrypt the system, at least your home, since it's easier to lose a USB stick than a whole computer.

You now have a USB which can boot in both modes : Legacy through BIOS-mode grub, and UEFI through refind.
If you inspect it with gparted, it should look like this :

GParted_USB.jpg

(swap is optional, and partition names and labels may be absent, I have put them manually for clarity)

B) Fix stuff
- Boot the USB in UEFI mode
- Make a /boot/refind_linux.conf file, in order to get a refind linux entry with nice boot options. Mine looks like this

Code: Select all

"Boot with standard options"  "root=UUID=the-uuid ro quiet splash"
"Boot in compatibility mode"   "ro root=UUID=the-uuid ro nomodeset"

where the-uuid is the UUID of the / partition on the USB (you can find this UUID with gparted or blkid or lsblk).
First option gives a graphical splash screen during boot (nicer), second disables GPUs for computers which are harder to boot from
- correct the fstab to have the ESP mounted at /boot/efi
Add the following lines to /etc/fstab as root

Code: Select all

# /boot/efi 
UUID=the-uuid-bis  /boot/efi       vfat    noatime,utf8,fmask=0133,errors=remount-ro      0       1

where the-uuid-bis is the UUID of the ESP on the USB.
- actually mount the ESP by reloading the fstab

Code: Select all

sudo mount -a

- Install grub UEFI to the USB
I prefer to do this manually, by copying grubx64.efi of some computer to /boot/efi/EFI/ubuntu/grubx64.efi, same for the grub.cfg (the one that is on the ESP, not the one on /boot/grub/) and correcting the UUID in grub.cfg to point to the correct / partition. This is described in greater details in section E.2.
Alternatively you can just go with installing grub-efi-amd64 from the repositories, problem is that this will modify the NVRAM of the computer where you booted from (and it will do so everytime you get a grub update). So you will find that said computer may not boot correctly anymore once you unplug the USB. This can be corrected, but it's a bit painful if you need to do this everytime you get a grub update on the USB. You might get away with this by installing manually, something like

Code: Select all

sudo grub-install /dev/sdc --target=x86_64 --no-nvram

where the important point is the --no-nvram option. I dont guarantee this will work, I haven't tried as I did stuff manually.
- Disable os-prober on the USB

Code: Select all

chmod -x /etc/grub.d/30_os-prober

otherwise everytime grub is updated on the USB, you will get new entries in the GRUB menu corresponding to stuff on the hard drive of the computer where you booted from.

C) Add live ISOs : grub method
You are still booted on the USB (either Legacy or UEFI mode now it doesnt matter)
- Get the ISO of your favourite distro (example below is with lubuntu and Mint 18.2) and put it at /boot/ISOs/
(or put it elsewhere eg just your Downloads folder, if you didnt encrypt the installed sytem)
- Modify /etc/grub.d/40_custom to make grub entries to boot the ISOs
To do this I used this link and some trial and error. Here is how my 40_custom file looks like, with two ISOs : one for Lubuntu, the other for Mint

Code: Select all

#!/bin/sh
exec tail -n +3 $0
# This file provides an easy way to add custom menu entries.  Simply type the
# menu entries you want to add after this comment.  Be careful not to change
# the 'exec tail' line above.

menuentry "Live ISO: Lubuntu-14.04" {
	 search --no-floppy --fs-uuid --set=isopart the-uuid
         set isofile="/boot/ISOs/lubuntu-14.04.5-desktop-amd64.iso"
         loopback loop ($isopart)/$isofile
         linux (loop)/casper/vmlinuz.efi boot=casper iso-scan/filename=$isofile noprompt noeject
         initrd (loop)/casper/initrd.lz
 }

menuentry "Live ISO: Linux Mint 18.2 Cinnamon" {
         search --no-floppy --fs-uuid --set=isopart the-uuid
         set isofile="/boot/ISOs/linuxmint-18.2-cinnamon-64bit.iso"
         loopback loop ($isopart)/$isofile
         linux (loop)/casper/vmlinuz boot=casper iso-scan/filename=$isofile noprompt noeject
         initrd (loop)/casper/initrd.lz
 }

where the-uuid is the UUID of the partition containing /boot on the USB. (as previously, you can find said UUID with gparted or blkid or lsblk...)
Remark : the entries look exactly similar, except that the lubuntu one has the kernel named vmlinuz.efi while the Mint one has the kernel named vmlinuz. In general you will have to inspect the ISO beforehand to find out what the kernel's name and initrd image's name are, then adapt the entry accordingly.
- update the grub menu

Code: Select all

sudo update-grub

Now the grub menu will provide entries to boot the ISOs, whether you booted in Legacy mode or in UEFI mode

D) Add live ISOs : UEFI/refind method
There is another simpler possibility to boot ISOs, but it only works in UEFI mode. This is simpler so I describe it nevertheless.
- Make an empty partition on the USB with size the size of the ISO
It can be formatted in any filesystem, doesnt matter. If you had previously let your installed system take the whole USB, you want to resize partitions to leave some unallocated space.
- dd the ISO to the partition

Code: Select all

sudo dd if=/path/to/iso of=/dev/sdc5 bs=1M status=progress

where sdc5 is the empty partition you just created, adapt the name if necessary.

That's it !
If your computer is sufficiently smart, it will provide an entry to boot the ISO in the firmware boot menu, something like "UEFI:USB-name,Part5" (assuming the ISO partition is the 5th one on the USB). Otherwise dont worry, refind will allow you to boot the ISO. Indeed it will detect the ISO and give two entries with a CD badge : one for grubx64.efi, other for Fallback bootloader (=shim which chainloads to grub), both entries are equivalent.
Drawback : this works only for UEFI mode.

E) Installing UEFI bootloaders manually
If you like to do things manually and understand things deeper, or if you are afraid to overwrite the NVRAM of the computer you are working with, then this section is for you.

E.1) Refind
This happens in section A.1, before installing Linux on the USB, so you are booted on the host computer.
The starting point is that either
- you have refind already installed on the computer on the EFI partition, so /path/to/refind=/boot/efi/EFI/refind/ which should contain something like

Code: Select all

$ ls /boot/efi/EFI/refind/
BOOT.CSV   icons   refind.conf   drivers_x64   keys   refind_x64.efi

and configfile=refind.conf
- you download refind (e.g. to /home/username/Downloads/) and extract the zip file, so /path/to/refind=/home/username/Downloads/refind-bin-0.11.2/refind (for version 0.11.2, the latest at time of writing), which should contain something like

Code: Select all

$ ls Downloads/refind-bin-0.11.0/refind
drivers_aa64  drivers_x64  refind_aa64.efi     refind_ia32.efi  tools_aa64  tools_x64
drivers_ia32  icons        refind.conf-sample  refind_x64.efi   tools_ia32

and configfile=refind.conf-sample

Then the steps are :
1 ) get root with su or sudo -i
2 ) mount the EFI Partition of the USB (I'm assuming this is sdc2)

Code: Select all

mount /dev/sdc2 /mnt

3 ) make the directory structure

Code: Select all

mkdir -p /mnt/EFI/BOOT

4 ) copy the refind UEFI binary and rename it as the fallback bootloader

Code: Select all

cp /path/to/refind/refind_x64.efi /mnt/EFI/BOOT/bootx64.efi

5 ) copy the EFI drivers and refind icons

Code: Select all

cp -r /path/to/refind/drivers_x64 /mnt/EFI/BOOT/
cp -r /path/to/refind/icons /mnt/EFI/BOOT/

(take care of presence/absence of ultimate / if you use bash autocompletion)
6 ) copy the configuration file

Code: Select all

cp /path/to/refind/configfile /mnt/EFI/BOOT/

7 ) if you are a seasoned refind user, you may then want to edit the configfile to tweak it to your needs, add a banner, or add an EFI shell or other EFI utilities (gptsync, KeyTool, ...)
8 ) unmount the ESP and exit root

Code: Select all

umount /mnt
exit

E.2) Grub
This happens on section A.2, after installing Linux to the USB. So I'm going to assume you are booted on the USB, although you could as well do it when booted on the computer (which will only change the mount points of the source and target ESPs, so you would need to swap the mount points in the command below).

1 ) Locate the (source) ESP from which you want to copy grub. Normally that should be the first partition of the hard drive on the computer, so sda1, but adapt if you have a different setup
2 ) get root with su or sudo -i
3 ) mount the source ESP

Code: Select all

mount /dev/sda1 /mnt

(the target ESP, i.e. that of the USB, should be already mounted at /boot/efi
4 ) make the good directory structure on the target ESP

Code: Select all

mkdir -p /boot/efi/EFI/ubuntu

5 ) copy grub and its config file

Code: Select all

cp /mnt/EFI/ubuntu/grubx64.efi  /boot/efi/EFI/ubuntu/
cp /mnt/EFI/ubuntu/grub.cfg  /boot/efi/EFI/ubuntu/

(alternatively you could copy the whole /mnt/EFI/ubuntu directory in a single command, but that will normally get you the EFI programs fbx64.efi and fwupx64.efi which are useless, if not dangerous to have on a USB)
6 ) Find the UUID of the Linux partition containing /boot on the USB, e.g. with blkid. (it's normally the / partition, unless you made a particular setup with a separate /boot, e.g. whole disk encryption)
7 ) correct the grub config file to point to that partition

Code: Select all

nano /booot/efi/ubuntu/grub.cfg

which should give something like

Code: Select all

search.fs_uuid the-uuid root hdX,gptY 
set prefix=($root)'/boot/grub'
configfile $prefix/grub.cfg

correct the-uuid into the one found in step 6), optionally correct also the hdX,gptY (put X=0, Y=number of the /boot partition) though uuid should have precedence
8 ) unmount the ESP and exit root

Code: Select all

umount /mnt
exit

E.3) Updates
With these manual methods, you wont get any update for refind or grub, since they were not installed by apt. But it's a good thing in my opinion.
You can install refind/grub via apt (for refind you will need to add the PPA, see this page), but if you do, when you get an update you will overwrite the NVRAM UEFI entries of the computer you are booted from. This is unwanted.
One could repair said NVRAM entries after each update, but frankly this is tedious and you have to pay attention everytime you see updates are available whether grub or refind is concerned.
Instead, I advise either not to care about refind/grub updates, or to update manually by downloading the files and copying then to the USB's ESP.

F) Credits/References :
http://www.rodsbooks.com in particular the refind pages, but also pages about GPT, UEFI booting etc.
a thread by pbear : viewtopic.php?f=46&t=252351
Installation & Boot subforum and people contributing there
https://help.ubuntu.com/community/Grub2 ... GRUB_files

[kukamuumuka]

Could you make a downloadable small image with base configs?

PS. An excellent tutorial.

[fabien85]

Thanks.
I have added a screenshot of the partition layout on the USB (I understood that was your question, tell me otherwise), and added more detailed instructions on how to install refind or grub manually.