script to install all your programs again on a fresh install

About writing shell scripts and making the most of your shell
Forum rules
Topics in this forum are automatically closed 6 months after creation.
Locked
User avatar
Blankman
Level 2
Level 2
Posts: 59
Joined: Tue May 05, 2020 10:18 am
Location: the world

script to install all your programs again on a fresh install

Post by Blankman »

Hi everyone, I tried moving my linux mint 20 install from one drive to another. I googled it and there was a command to move the installed app list to the new computer and then have it install, dpkg something which then creates the text file which you move to the new computer and run the update. When i ran the one line command it installed something but not the programs on that list. So I've decided to create a script that i can run to install all the programs I want through the terminal. I see a lot of programs say "sudo apt-get update" before "sudo apt-get install program_name". So the question is do i have to type that after each line or can i just type it once at the top and install all programs after that? eg.

Code: Select all

sudo apt-get update
sudo apt-get install program_name
sudo apt-get install program_name
sudo apt-get install program_name
sudo apt-get install program_name
Is there maybe an easier way to install the programs i want? I want to install at least about 20 programs so I'm trying to make a script to do that that will run on linux mint. Thanks in advance.
Last edited by LockBot on Wed Dec 28, 2022 7:16 am, edited 1 time in total.
Reason: Topic automatically closed 6 months after creation. New replies are no longer allowed.
Moved from windows to linux and enjoying every minute of linux. It's a learning curve but well worth it.
User avatar
deck_luck
Level 7
Level 7
Posts: 1577
Joined: Mon May 27, 2019 6:57 pm
Location: R-4808 North

Re: script to install all your programs again on a fresh install

Post by deck_luck »

Blankman wrote: Thu Oct 21, 2021 1:34 am ...
I see a lot of programs say "sudo apt-get update" before "sudo apt-get install program_name". So the question is do i have to type that after each line or can i just type it once at the top and install all programs after that?
...
You only need to run the sudo apt-get update at the beginning of your script.

The update will resynchronize the package index files from their sources. The indexes of available packages are fetched from the location(s) specified in /etc/apt/sources.list.
🐧Linux Mint 20.3 XFCE (UEFI - Secure Boot Enabled) dual boot with Windows 11

Give a friend a fish, and you feed them for a day. Teach a friend how to fish, and you feed them for a lifetime. ✝️
vanadium
Level 4
Level 4
Posts: 325
Joined: Sun Dec 27, 2015 1:07 pm

Re: script to install all your programs again on a fresh install

Post by vanadium »

To add here, you can further simplify by listing all your packages on a single "sudo apt install" line, e.g.

Code: Select all

sudo apt install gimp inkscape kdenlive
There is no easier way than writing it out in a script: you cannot avoid telling your system what to install.
User avatar
AndyMH
Level 21
Level 21
Posts: 13572
Joined: Fri Mar 04, 2016 5:23 pm
Location: Wiltshire

Re: script to install all your programs again on a fresh install

Post by AndyMH »

What is being proposed will only work for stuff in the standard repositories, it will not work if you have software installed from ppa, deb file, additional repos or anything you have to build. Nor is program name necessarily the same as package name. If you are migrating from an earlier version of LM, you also have to contend with the absence of python2 for any software you used to use that depended on it.

I have a script that takes a vanilla install of LM20 and installs all the software I want, sets up my network shares, installs printers, etc. I benefit from having a separate home partition so I don't need to worry about all my config files. The setup folder is over 1GB, the script runs to some +1000 lines (okay, probably 50% comments). It took me a couple of weeks to do on and off.

How complicated do you want to make it? :D

If you just want stuff from the standard repos, you don't need to write your own script, the mint backup tool will do what you want.
Screenshot from 2021-10-21 11-37-32.png
Or you could simply clone the old drive to the new drive - foxclone, rescuezilla, clonezilla.
Thinkcentre M720Q - LM21.3 cinnamon, 4 x T430 - LM21.3 cinnamon, Homebrew desktop i5-8400+GTX1080 Cinnamon 19.0
User avatar
Blankman
Level 2
Level 2
Posts: 59
Joined: Tue May 05, 2020 10:18 am
Location: the world

Re: script to install all your programs again on a fresh install

Post by Blankman »

deck_luck wrote: Thu Oct 21, 2021 4:09 am
Blankman wrote: Thu Oct 21, 2021 1:34 am
I see a lot of programs say "sudo apt-get update" before "sudo apt-get install program_name". So the question is do i have to type that after each line or can i just type it once at the top and install all programs after that?
You only need to run the sudo apt-get update at the beginning of your script.

The update will resynchronize the package index files from their sources. The indexes of available packages are fetched from the location(s) specified in /etc/apt/sources.list.
Thanks ye i tried that, i updated the list, moved the sources.list to new computer and then ran the update command to install the programs in the sources.list file. It ran the updates but never installed the programs. I'm not too sure how to see what it actually did because the update took a while to do. Maybe there's a log somewhere.
vanadium wrote: Thu Oct 21, 2021 6:10 am To add here, you can further simplify by listing all your packages on a single "sudo apt install" line, e.g.

Code: Select all

sudo apt install gimp inkscape kdenlive
There is no easier way than writing it out in a script: you cannot avoid telling your system what to install.
Thanks yeah I'm looking for the simplist way, that's why I was thinking of making a script, that way it's automated. I like what you said, you can have everything on one line, that will make it super simple and less code in the script.
AndyMH wrote: Thu Oct 21, 2021 6:34 am What is being proposed will only work for stuff in the standard repositories, it will not work if you have software installed from ppa, deb file, additional repos or anything you have to build. Nor is program name necessarily the same as package name. If you are migrating from an earlier version of LM, you also have to contend with the absence of python2 for any software you used to use that depended on it.

I have a script that takes a vanilla install of LM20 and installs all the software I want, sets up my network shares, installs printers, etc. I benefit from having a separate home partition so I don't need to worry about all my config files. The setup folder is over 1GB, the script runs to some +1000 lines (okay, probably 50% comments). It took me a couple of weeks to do on and off.

How complicated do you want to make it? :D

If you just want stuff from the standard repos, you don't need to write your own script, the mint backup tool will do what you want.
Screenshot from 2021-10-21 11-37-32.png
Or you could simply clone the old drive to the new drive - foxclone, rescuezilla, clonezilla.

Thanks ye i was considering having a backup drive strictly with all programs I want installed and then just clone it over. The reason I want to make a script is so that I can add programs to it. I don't want to have to put my backup drive back in and then install the programs and then take it out again. If that's the easiest way then I will do that.

Would you mind sharing your script you made here ? I would love to see it.

Regarding the picture you sent showing the backup, i see it backs up programs as well as settings, is that right?
Moved from windows to linux and enjoying every minute of linux. It's a learning curve but well worth it.
User avatar
AndyMH
Level 21
Level 21
Posts: 13572
Joined: Fri Mar 04, 2016 5:23 pm
Location: Wiltshire

Re: script to install all your programs again on a fresh install

Post by AndyMH »

Would you mind sharing your script you made here ? I would love to see it.

Regarding the picture you sent showing the backup, i see it backs up programs as well as settings, is that right?
Too big to post on the forum, send me an email (click on me to get my profile page, there is a 'send an email' link).

The mint backup tool - it creates a list of all the installed programs, think it puts this as a tar.gz in your documents folder. When you restore it reads the list and installs what's there (as long as it is in the repos). Don't think it does anything for your settings (config files). I don't use it.

What I do for backup - all my PCs have an internal/removable 2TB drive. On that I have one large ext4 'backup' partition and one small, 20GB partition. I have timeshift and backintime running automatically every day saving snapshots to the 'backup'. I have another copy of mint installed to the small partition. Inside that I have installed foxclone. I take image backups of my system drive every few months with foxclone.

So if I bork my system or the drive fails, I can boot from the backup drive and try and fix it. Failing that, I can do a foxclone restore, which gets me back a working system current at the date of backup and then do timeshift/backintime restores from the latest snapshots to get me back to yesterday.

The script was put together to cover fresh installs. The recommendation on a major version change, e.g. LM19 to LM20 is a fresh install. Here you have to re-install all your software and set everything up from scratch. A separate home partition means you don't have to worry about all your configs, appearance of desktop, etc. but you still need to install your software, printers, network shares, etc. - that's what the script does.
Thinkcentre M720Q - LM21.3 cinnamon, 4 x T430 - LM21.3 cinnamon, Homebrew desktop i5-8400+GTX1080 Cinnamon 19.0
Kendoori
Level 5
Level 5
Posts: 749
Joined: Thu Jul 09, 2009 12:51 pm
Location: Sanibel, FL USA

Re: script to install all your programs again on a fresh install

Post by Kendoori »

I realize the OP was looking for a script, but I have found that Aptik does an exceptional job of this, to the point where I no longer fret about a reinstall, new install, or migrating hardware. https://teejeetech.com/2021/07/18/aptik-21-07/ There is a 1x cost for this, but IMHO worth it as it is professional and reliable software that is elegant and will save tons of time. FYI, this is authored by the developer of Timeshift. I realize that anything that has a cost will be objectionable to many here, But I personally don’t feel bad paying fair value to a developer who does good work.
User avatar
deck_luck
Level 7
Level 7
Posts: 1577
Joined: Mon May 27, 2019 6:57 pm
Location: R-4808 North

Re: script to install all your programs again on a fresh install

Post by deck_luck »

Instead of paying for aptik you can create your own package lists, and it might be a good little script project.

The dpkg --get-selections | cut -f1 > dpkg__get-selections_original.txt will allow you to keep a list of installed package of your original computer.

Before doing an installation to a new version of LinuxMint you would keep a copy of the output file to your favorite removable media. After the new LinuxMint installation you can copy the original package list from your removable media to the new installation. You can differentiate it with the new installation package list. You can use the diff and awk utilities to differentiate the two package list files.

I think I hear Andy's keyboard clicking now. :lol:
🐧Linux Mint 20.3 XFCE (UEFI - Secure Boot Enabled) dual boot with Windows 11

Give a friend a fish, and you feed them for a day. Teach a friend how to fish, and you feed them for a lifetime. ✝️
User avatar
AndyMH
Level 21
Level 21
Posts: 13572
Joined: Fri Mar 04, 2016 5:23 pm
Location: Wiltshire

Re: script to install all your programs again on a fresh install

Post by AndyMH »

I heard that :D

I assume dpkg --get-selections... will just give you a list of packages, I've been tripped up in the past - where and how did I install stuff? I now keep a simple text file on my backup drive, launcher on the desktop. Whenever I make any change, I make a date/timestamped note of it, if I install stuff - how - software manager, ppa, deb, etc.

Aptik - played with it briefly a few years ago, but don't use it. I assume it simply backs up all the debs downloaded to install stuff. I would expect it to fall over with a major change, e.g. dropping python2 support in LM20. While aptik may be able to install the program, it will be missing the dependencies (no longer in the repos)? This caused me some fun moving to LM20, fslint was one, there were a couple of others. But then, how often do we get changes of this magnitude...

I am also happy to pay for software, crossover and vuescan being two examples, not forgetting the monthly patreon contribution to mint.
Thinkcentre M720Q - LM21.3 cinnamon, 4 x T430 - LM21.3 cinnamon, Homebrew desktop i5-8400+GTX1080 Cinnamon 19.0
Kendoori
Level 5
Level 5
Posts: 749
Joined: Thu Jul 09, 2009 12:51 pm
Location: Sanibel, FL USA

Re: script to install all your programs again on a fresh install

Post by Kendoori »

@AndyMH Aptik covers a lot of bases. It handles backing up configurations, and it's been my experience that it literally made my new install operate like the old, both on the same machine or another. A handful of things didn't properly come across, but nothing significant. What I like about the approach is it encourages you to start fresh, and then it does its thing on top of fresh. Tony writes good software and he is a Linux guy through and through.

I love FoxClone and still use it as a safety (I have multiple external drives that I rotate), but those are really for DR for me. Aptik is just an excellent migration tool, that's thought through the whole migration process (and also any easy way to replicate an existing config across lots of machines).
User avatar
farkas
Level 5
Level 5
Posts: 555
Joined: Mon Sep 21, 2015 6:10 pm
Location: Oregon

Re: script to install all your programs again on a fresh install

Post by farkas »

This suggestion is not to replace previous solutions posted above. It is in case all else fails. The belt buckle has failed and the suspender is about break

Another possibility is to to create a markings file using the Synaptic Package Manager. (SPM)
It will save list of your currently installed programs that are available in SPM.
Open SPM select Installed (manual) > File>Save Markings You should see some ting like this
SPM example.png
Be sure to check the box," Save full state, not just changes" in lower left hand corner.
I usually save it to my Desktop to easily find it then move it to a folder of my choice.
Select location to save to by
Other location>Computer>Home>'Your PC name'>Desktop
Click Save
You should have a text file on your desktop with the file name you chose.
Copy file to a USB stick
After a fresh install LM copy the markings file to Desktop.
Open SPM and select File>Read markings> navigate to Desktop> select the file and click Open.
Select options as as presented.
The above should install all the programs that you had in previous version of LM.
It will NOT restore your settings or programs not in the SPM.
If your query has been resolved, edit your first post and add [SOLVED] to the subject line.
If you found a solution on your own please post it.
A generation which ignores history has no past and no future.
User avatar
Michael_Hathaway
Level 4
Level 4
Posts: 329
Joined: Sat Oct 09, 2021 2:27 am
Location: Shebang, USA
Contact:

Re: script to install all your programs again on a fresh install

Post by Michael_Hathaway »

You can also create a list of apps in a file, example.list, and run

Code: Select all

cat example.list | xargs sudo apt-get -y install
Enterprise Dual Xeon 8081 (112) @3.8Ghz, 16TB NVMe Raid, 387Gb ECC, AMD Pro W7700 16Gb
Debian Support. Deb 12/13 Trixie 6.7.9
Image
User avatar
farkas
Level 5
Level 5
Posts: 555
Joined: Mon Sep 21, 2015 6:10 pm
Location: Oregon

Re: script to install all your programs again on a fresh install

Post by farkas »

How do you create?

Code: Select all

cat example.list
If your query has been resolved, edit your first post and add [SOLVED] to the subject line.
If you found a solution on your own please post it.
A generation which ignores history has no past and no future.
User avatar
Michael_Hathaway
Level 4
Level 4
Posts: 329
Joined: Sat Oct 09, 2021 2:27 am
Location: Shebang, USA
Contact:

Re: script to install all your programs again on a fresh install

Post by Michael_Hathaway »

You can create your example.lst with any text editor. Use Xed, it comes with Mint.
Enterprise Dual Xeon 8081 (112) @3.8Ghz, 16TB NVMe Raid, 387Gb ECC, AMD Pro W7700 16Gb
Debian Support. Deb 12/13 Trixie 6.7.9
Image
User avatar
deck_luck
Level 7
Level 7
Posts: 1577
Joined: Mon May 27, 2019 6:57 pm
Location: R-4808 North

Re: script to install all your programs again on a fresh install

Post by deck_luck »

AndyMH wrote: Thu Oct 21, 2021 1:35 pm ...
I assume dpkg --get-selections... will just give you a list of packages, I've been tripped up in the past - where and how did I install stuff?
...
Yes, if the software installation method is other than dpkg it will not appear.
AndyMH wrote: Thu Oct 21, 2021 1:35 pm ...
I now keep a simple text file on my backup drive, launcher on the desktop. Whenever I make any change, I make a date/timestamped note of it, if I install stuff - how - software manager, ppa, deb, etc.
...
I agree with updating your change log for any hardware or software change. If you work in an enterprise environment, updating the change log is standard operating procedure.
🐧Linux Mint 20.3 XFCE (UEFI - Secure Boot Enabled) dual boot with Windows 11

Give a friend a fish, and you feed them for a day. Teach a friend how to fish, and you feed them for a lifetime. ✝️
User avatar
Michael_Hathaway
Level 4
Level 4
Posts: 329
Joined: Sat Oct 09, 2021 2:27 am
Location: Shebang, USA
Contact:

Re: script to install all your programs again on a fresh install

Post by Michael_Hathaway »

Will give you a list of your applications and create applications.txt and save to your desktop. But this is everything and you will have to identify the post installed apps.

Code: Select all

find /usr/share/applications -maxdepth 1 -type f -exec basename {} .desktop \; | sort > ~/Desktop/applications.txt
Enterprise Dual Xeon 8081 (112) @3.8Ghz, 16TB NVMe Raid, 387Gb ECC, AMD Pro W7700 16Gb
Debian Support. Deb 12/13 Trixie 6.7.9
Image
User avatar
Michael_Hathaway
Level 4
Level 4
Posts: 329
Joined: Sat Oct 09, 2021 2:27 am
Location: Shebang, USA
Contact:

Re: script to install all your programs again on a fresh install

Post by Michael_Hathaway »

Maybe this will help you. You can modify to your needs. Check prerequisites add/remove as needed. I have tested on a virtual machine before posting.

Code: Select all

#!/bin/bash


## If a sub-command fails, the entire pipeline command fails, terminating the script 
set -eu -o pipefail # fail on error and report it, debug all lines

sudo -n true
test $? -eq 0 || exit 1 "you should have sudo privileges to run this script"

echo installing the must-have pre-requisites
while read -r p ; do sudo apt-get install -y $p ; done < <(cat << "EOF"
    perl
    htop
    curl
    git
    wget
    
EOF
)

echo "========================================================================"
echo installing the nice-to-have pre-requisites
echo you have 5 seconds to proceed ...
echo or
echo hit Ctrl+C to quit
echo -e "\n"
echo "========================================================================"
sleep 6

## Don't forget to place your *.list file in directory
cat applications.list | xargs sudo apt-get -y install

clear
echo "========================================================================"
echo "Packages installed"
echo "========================================================================"

read -p "Press enter to close..."
Enterprise Dual Xeon 8081 (112) @3.8Ghz, 16TB NVMe Raid, 387Gb ECC, AMD Pro W7700 16Gb
Debian Support. Deb 12/13 Trixie 6.7.9
Image
User avatar
ibm450
Level 5
Level 5
Posts: 650
Joined: Sun Jan 11, 2009 6:56 am
Location: Hamilton Hill, Western Australia

Re: script to install all your programs again on a fresh install

Post by ibm450 »

Michael_Hathaway wrote: Mon Oct 25, 2021 4:19 am Maybe this will help you. You can modify to your needs. Check prerequisites add/remove as needed. I have tested on a virtual machine before posting.

Code: Select all

#!/bin/bash


## If a sub-command fails, the entire pipeline command fails, terminating the script 
set -eu -o pipefail # fail on error and report it, debug all lines

sudo -n true
test $? -eq 0 || exit 1 "you should have sudo privileges to run this script"

echo installing the must-have pre-requisites
while read -r p ; do sudo apt-get install -y $p ; done < <(cat << "EOF"
    perl
    htop
    curl
    git
    wget
    
EOF
)

echo "========================================================================"
echo installing the nice-to-have pre-requisites
echo you have 5 seconds to proceed ...
echo or
echo hit Ctrl+C to quit
echo -e "\n"
echo "========================================================================"
sleep 6

## Don't forget to place your *.list file in directory
cat applications.list | xargs sudo apt-get -y install

clear
echo "========================================================================"
echo "Packages installed"
echo "========================================================================"

read -p "Press enter to close..."
thanks
HP EILITE FOLIO 9470M i7-3667u x 4
GitHub: tolgaerok
Image Image
rickNS
Level 9
Level 9
Posts: 2914
Joined: Tue Jan 25, 2011 11:59 pm

Re: script to install all your programs again on a fresh install

Post by rickNS »

EDIT: seems this no longer works as it did in Mint 19. so removed. (at least till I see what went wrong)
Mint 20.0, and 21.0 MATE on Thinkpads, 3 X T420, T450, T470, and X200
rickNS
Level 9
Level 9
Posts: 2914
Joined: Tue Jan 25, 2011 11:59 pm

Re: script to install all your programs again on a fresh install

Post by rickNS »

An alternative to dpkg --get-selections which is a pretty long list, I prefer (and I got it fixed to leave out the libs) the following command which gives a pretty short list, and "think" it includes PPA packages as I installed an older version of kodi (but may have installed reg. version first ?) anyway.

comm -23 <(apt-mark showmanual | sort -u) <(gzip -dc /var/log/installer/initial-status.gz | sed -n 's/^Package: //p' | sort -u) | grep -Ev ^lib
Mint 20.0, and 21.0 MATE on Thinkpads, 3 X T420, T450, T470, and X200
Locked

Return to “Scripts & Bash”