Modify Live Iso to use any kernel version.

[bodge99]

Iso modification to allow any required kernel to be booted.
UEFI and Legacy boot.

(Edit: This specifically applies to the Mint 20.1 Iso's. Other Mint Iso's will be similar. You would need to examine the 'initrd.lz' file with something like 'binwalk' to identify the end address of the microcode archive/start address of the "real" 'initrd'.)

Note: The flashdrive is **writable** now. (Things can be added, pre-boot, if required.)
I "tasked myself" to do this... Yes, I really am that sad.. Actually, I've used this as a learning exercise.

A worked example that uses the Ubuntu Mainline kernel version 5.12.0-051200-generic.
This is installed onto the host system using 'mainline-gtk'. You can uninstall this after this process is completed if required.
See https://code.launchpad.net/~cappelikan/ ... ubuntu/ppa
NOTE: The 5.12.0 (and onwards) kernel headers are not installable in Mint 20.1 at this time (May 2021) due to dependency problems. 'libc6' 2.33 or greater is now required. This does not affect booting or normal system operation.

You can use any kernel that you wish.. I'm copying files from a host system that has the required kernel installed.
You can certainly use kernel '.deb' files if you so wish.

I use 'Gparted' and 'Xed' (run as 'sudo xed' from a non-Root terminal.).
Format a flashdrive to fat32 and set the 'boot' and 'esp' flags. Edit: 'msdos' partition table.. **NOT** 'gpt'.

Edit 10/05/21 . I've found that with some flashdrives that you cannot set the 'boot' and 'esp' flags unless the partition & filesystem has been created with Gparted.

IMPORTANT: Use 'gparted' for this... 'Device', 'Create partition table'. Chooses 'msdos' and 'Apply'. 'Partition', 'New'. Select 'Fat32' and click 'Add'.
'Edit', 'Apply Operations'. Click on the partition graphic (the box with the text '/dev/sda1') to highlight it. 'Partition', 'Manage flags'. Select 'boot' & 'esp'
and click on close. Close Gparted.

Overview:
The Iso contents are extracted. From here, the squashfs filesystem & initrd are extracted and uncompressed.
The unsquashed file system is modified by replacing the firmware and kernel modules. The initramfs.conf is altered to use 'gzip' compression.
The initrd is now extracted. Firstly, the initial Intel & AMD microcode archive is separated off. The remaining 'initrd' is expanded.
The Initrd firmware & modules are replaced and the Initrd's 'initramfs.conf' is altered. The complete 'initrd.lz' is rebuilt in two stages.
Recompress the filesystem using 'mkinitramfs' and replacing the microcode archive at the beginning of the file.

Other minor changes: Alter the system's 'grub.cfg' and replace the boot kernel. Update the 'MD5SUMS' and Squashfs 'filesystem.size' files.
On boot, the system starts a "checking integrity" check. I'm not sure what initiates this as the only thing that I can't reconcile is the Squashfs 'filesystem.size' value. The original Iso version's file content doesn't match the calculated size of its unsquashed filesystem. Just cancel the check.
Yes, I have performed an installation using a flashdrive prepared by this method.

Extract ISO:
Open a terminal, become Root and mount the Iso. Create two working directories. I'll use '/mntwork' & '/mntflash' here.
'/mntflash' contains the extracted Iso files. Files here are either modified directly or copied over from those created in '/mntwork'.

Extract the Iso and unpack the squashfs file:

Code: Select all

sudo su
mkdir /mntwork /mntflash
mount /path-to-iso/iso-name.iso /mnt
rsync -r --info=progress2 /mnt/ /mntflash {Copy Iso contents to /mntflash.}
umount /mnt
cd /mntwork {Working directory. Modified files are stored here and then written to their correct location within /mntflash.}

Next, expand the compressed filesystem and the initrd:

Code: Select all

unsquashfs -d ./unsquashed /mntflash/casper/filesystem.squashfs {Expand the filesystem to ./unsquashed/ directory.}

dd bs=3492864 count=1 if=/mntflash/casper/initrd.lz of=./initrd_part1.bin  {Remove the header archive (this contains Intel & AMD microcode).}
dd bs=3492864 skip=1 if=/mntflash/casper/initrd.lz of=./compressedinit.bin  {Remove the compressed initrd}

{Ignore the two 'cpio' warnings here. These refer to the parent directory that contains the expanded filesystem.)
unlz4 -d ./compressedinit.bin expandedinitrd.bin {Decompress the original initrd.}
cpio -id -D NewInitrd < ./expandedinitrd.bin {Extract filesystem to ./NewInitrd/}

Edit the system's 'grub.cfg':
From a non-Root terminal:

Code: Select all

sudo xed /mntflash/boot/grub/grub.cfg

Delete everything and replace with the following:

Code: Select all

if loadfont /boot/grub/font.pf2 ; then
	set gfxmode=auto
	insmod efi_gop
	insmod efi_uga
	insmod gfxterm
	terminal_output gfxterm
fi

set menu_color_normal=white/black
set menu_color_highlight=black/light-gray


menuentry "Start Linux Mint 20.1 64-bit ***NEW KERNEL***" --class linuxmint {
	set gfxpayload=keep
	linux	/casper/vmlinuz  file=/usb/preseed/linuxmint.seed boot=casper iso-scan/filename=${iso_path} quiet splash --
	initrd	/casper/initrd.lz
}

menuentry "Start Linux Mint 20.1 64-bit ***NEW KERNEL*** (compatibility mode)" {
	linux	/casper/vmlinuz  file=/usb/preseed/linuxmint.seed boot=casper iso-scan/filename=${iso_path} noapic noacpi nosplash irqpoll nomodeset --
	initrd	/casper/initrd.lz
}

Save & Exit.

From a non-Root Terminal:

Code: Select all

sudo xed /mntwork/unsquashed/etc/initramfs-tools/initramfs.conf

Change line 53. From 'COMPRESS=lz4' to 'COMPRESS=gzip'. Save & Exit.

Modify the expanded squashfs filesystem:
From a Root terminal:

Code: Select all

cp /boot/System.map-5.12.0-051200-generic /mntwork/unsquashed/boot
cp /boot/config-5.12.0-051200-generic /mntwork/unsquashed/boot
rm /mntwork/unsquashed/boot/System.map-5.4.0-58-generic
rm /mntwork/unsquashed/boot/config-5.4.0-58-generic

rm -r /mntwork/unsquashed/lib/firmware/*
cp -R /lib/firmware/* /mntwork/unsquashed/lib/firmware {Update firmware.}

cp -R /lib/modules/5.12.0-051200-generic /mntwork/unsquashed/lib/modules {Replace modules.}
rm -r /mntwork/unsquashed/lib/modules/5.4.0-58-generic

ln -s vmlinuz-5.12.0-051200-generic vmlinuz {Replace symlinks to point to the new kernel & initrd.}
ln -s vmlinuz-5.12.0-051200-generic vmlinuz.old
ln -s initrd.img-5.12.0-051200-generic initrd.img
ln -s initrd.img-5.12.0-051200-generic initrd.img.old
mv vmlinuz* /mntwork/unsquashed/boot
mv initrd.* /mntwork/unsquashed/boot

Rebuild the 'filesystem.squashfs' and copy to the final location:

Code: Select all

mksquashfs /mntwork/unsquashed/* /mntwork/filesystem.squashfs
cp /mntwork/filesystem.squashfs /mntflash/casper

Modify the uncompressed Initrd:
From a non-Root Terminal:

Code: Select all

sudo xed /mntwork/NewInitrd/conf/initramfs.conf

Change line 53. From 'COMPRESS=lz4' to 'COMPRESS=gzip'. Save & Exit.

From the Root terminal:

Code: Select all

rm -r /mntwork/NewInitrd/lib/firmware/*
cp -R /lib/firmware/* /mntwork/NewInitrd/lib/firmware {Update firmware.}

cp -R /lib/modules/5.12.0-051200-generic /mntwork/NewInitrd/lib/modules {Replace modules.}
rm -r /mntwork/NewInitrd/lib/modules/5.4.0-58-generic

{Rebuild initrd and copy it to the required location.}
cd /mntwork/NewInitrd
mkinitramfs -d ./conf -o /mntwork/initrd_part2.bin
cd ..
cat initrd_part1.bin initrd_part2.bin > /mntflash/casper/initrd.lz
cp /boot/vmlinuz-5.12.0-051200-generic /mntflash/casper/vmlinuz {Replace kernel.)
cd /mntflash
find -type f -exec md5sum '{}' \; > MD5SUMS {Update the MD5SUMS list.}
du /mntwork/unsquashed -sx --block-size=1 | cut -f1 > /mntflash/casper/filesystem.size

Copy everything to the flashdrive:

Code: Select all

mount /dev/sda1 /mnt
shopt -s dotglob {Temporary change of your bash shell environment to allow the copying of a specific file type}
rsync -r --info=progress2 /mntflash/ /mnt {Copy modified Iso contents to /mnt.}
sync  {Wait for the prompt to return.}

Legacy Boot:

Code: Select all

grub-install --boot-directory=/mnt/boot --target=i386-pc /dev/sda {This assumes that your flashdrive is seen as 'sda'.}
sync
umount /mnt

Done.

Bodge99

[AndyMH]

Useful, not for the faint-hearted.

[bodge99]

Hi,

I forgot to add..

You can use multiple kernels if you so wish.. All you need is the specific kernel(s) and modules etc. copied over. You will also need to alter the flashdrive's 'grub.cfg'. to suit.

The Cinnamon Edge Iso is slightly different. This uses a newer kernel (and modules etc.). The process is identical, just examine the expanded 'initrd' and the unsquashed filesystem ('/lib/modules/') to see the differences. Cinnamon-Edge has '5.4.0-58-generic' and '5.8.0-33-generic' installed.

Bodge99

[bodge99]

Hi,

This morning I received a quite nasty PM (Not from this forum) from someone that I've interacted with before..

He had seen this thread and was giving me "grief" about it.

I decided to write this thread as a result of previous installation problems with brand new kit. In most cases, the stock Mint Live flashdrives wouldn't boot successfully.
It took booting from an external installation (that had a newer kernel than the Cinnamon-Edge has) to prove that a certain minimum kernel version was required.

Although installing the 5.12.0 (or newer) Ubuntu kernel won't allow the relevant headers to be installed on Mint 20.1 (dependency problems), it's still useful to see if the hardware will actually boot.

IMHO this is a valid problem solving technique..

Any opinions..?

Bodge99

[bodge99]

Hi,

I've now uploaded a compressed file that contains the modified Iso files.. If anybody wants this, then please PM me for a link.

You would need to download the archive and extract it/copy the contents to a formatted flashdrive.
Use Gparted to set the 'boot' and 'esp' flags on the flashdrive filesystem.

If you wish to use this for a Legacy boot, then repeat the final Grub Legacy boot stage. This will ensure that the flashdrive's boot mechanism is correctly set.
I've repeated this here:
Legacy Boot:

Code: Select all

sudo su
mount /dev/sda1 /mnt  {Mount the flashdrive to /mnt.}
grub-install --boot-directory=/mnt/boot --target=i386-pc /dev/sda {This assumes that your flashdrive is seen as 'sda'.}
sync
umount /mnt

Bodge99

[bodge99]

Hi,

I've uploaded a new version of the archive to Google Drive. This has 32-bit UEFI Grub installed in addition to the 64-bit version.

If you are installing to a 32-bit UEFI system (with a 64-bit CPU/SOC) then you'll need to manually install 32-bit UEFI Grub **BEFORE** rebooting.
To do this:
You have to update the flashdrive Grub version to the latest available before you can install 32-bit UEFI Grub.
Open a terminal.

Code: Select all

sudo su
apt update
apt install --reinstall grub-common {Trigger an update of Grub.} 
apt install grub-efi-ia32-bin   {Install 32-bit UEFI Grub to the live system.}

Now identify the computer's EFI and system partition:

Code: Select all

fdisk -l {The relevant partitions should be obvious..}

Here, I'll assume that 'sdb1' is the EFI partition and 'sdb2' is the Linux system partition.

Code: Select all

mount /dev/sda2 /mnt
mount /dev/sda1 /mnt/boot/efi
grub-install --efi-directory=/mnt/boot/efi --boot-directory=/mnt/boot --target=i386-efi /dev/sda
umount /mnt/boot/efi /mnt

I've also added the 32-bit and 64-bit UEFI shell executables for those UEFI Bios that don't have a built in shell.
These are in the root directory of the flashdrive.

Three files :

• shell.efi
  shell32.efi
  shell64.efi

The files 'shell.efi' and 'shell64.efi' are the same file, just renamed.
If you need the 32-bit executable, just copy 'shell32.efi' as 'shell.efi', overwriting the 64-bit version.
Most UEFI Bios (if they don't have an internal shell) will search for a file named 'shell.efi'.

Bodge99

[bodge99]

Hi,

The link to the latest archive, if anybody wants it is:
https://drive.google.com/file/d/1NHjCoS ... sp=sharing

For those who need 32-bit UEFI Grub:
Install 32-bit UEFI Grub to the flashdrive's live system and to the computer:
You have to update the flashdrive Grub version to the latest available. This eliminates the risk of any version mismatch.

Boot your computer from the flashdrive.. Install as normal but don't reboot.
Open a terminal.

Code: Select all

sudo su
apt update
apt install --reinstall grub-common {Trigger an update of Grub.} 
apt install grub-efi-ia32-bin   {Install 32-bit UEFI Grub to the live system.}

Now identify the computer's EFI and system partition:

Code: Select all

fdisk -l {The relevant partitions should be obvious..}

Here, I'll assume that sdb1 is the EFI partition and 'sdb2' is the Linux system partition.

Code: Select all

mount /dev/sdb2 /mnt
mount /dev/sdb1 /mnt/boot/efi  {Mount the EFI partition to it's correct location within the filesystem.}
grub-install --efi-directory=/mnt/boot/efi --boot-directory=/mnt/boot --target=i386-efi /dev/sda
umount /mnt/boot/efi /mnt

For All Installations:

When you have rebooted, fully update the system and reboot.
Now install 'Mainline'.
See https://ubuntuhandbook.org/index.php/20 ... inux-mint/

The instructions to install this are:
In a terminal:

Code: Select all

sudo su
add-apt-repository ppa:cappelikan/ppa {Add the repo. to the system.}
apt update
apt install mainline
mainline-gtk 

Install kernel version 5.11.16 . This is the latest Ubuntu kernel version (package set) that can be fully installed into Mint 20.1
This is due to dependency problems with other files at this time.

Watch the screen as the files are downloading. If you see a message that includes the text "critical error" then cancel the installation and restart from the main screen. This error indicates that there is file corruption within the downloaded files..
This is normally due to transient internet problems.

When the kernel packages have been correctly installed, close 'mainline-gtk' and reboot.
On restart, run 'mainline-gtk' again. This time uninstall kernel version '5.12.0'

The reason for uninstalling is this. When I created the modified Iso files archive, I used the latest Ubuntu kernel version that was available at the time.
This was primarily done for hardware testing purposes.. The newest hardware requires the newest (at least a newer) kernel.
Until Mint updates the required dependencies, proper system configuration requires that all of the kernel package versions match the installed system fully.
The kernel header packages for any newer Ubuntu mainline kernels don't, due to them requiring some newer system library file versions (libc6 etc.).

Bodge99