mintUpdate pulling 4.9 kernel ??

[Hansl]

I have several machines happily running LMDE2 x64 Mate.  Yesterday morning, on one of them mintUpdate pulled a lot of updates (≈325 MB), including a kernel update to 4.9. (Which crashed my Nvidia dual screen setup and my VirtualBox; I had to manually update them.  Now all seems to run fine again.)  Wow, I thought, Mint is upgrading to Debian Stretch?

But none of the other machines did pull theses Updates…  what was happening here?

On all machines, I have security and kernel updates and layers 1-4 enabled, layer 5 visible.
On two machines, including this one, I have the backports repo (for LibreOffice). But I think backports must be explicitly requested?
The enabled repos on this machine were (from all lists together):

Code: Select all

deb http://ftp-stud.hs-esslingen.de/pub/Mirrors/packages.linuxmint.com betsy main upstream import 
deb ftp://ftp-stud.hs-esslingen.de/debian/ jessie main contrib non-free
deb ftp://ftp-stud.hs-esslingen.de/debian/ jessie-updates main contrib non-free 
deb http://security.debian.org/ jessie/updates main contrib non-free
deb http://www.deb-multimedia.org jessie main non-free
deb http://extra.linuxmint.com betsy main 
deb http://repository.spotify.com stable non-free
deb http://httpredir.debian.org/debian jessie-backports main contrib non-free 

(To get VirtualBox up again, I had to add

Code: Select all

deb http://download.virtualbox.org/virtualbox/debian jessie contrib 

afterwards to install the latest virtualbox-5.1 .  sgfxi took care of my Nvidia.)

[xenopeek]

4.9 is in backports. You must have previously installed a kernel from backports. You only get offered upgrades for packages from backports after you have installed such a package in the past explicitly from backports.

[Hansl]

Sorry, but I did not have a backport kernel installed on this machine (only LibreOffice backports). The previous kernel was LMDE standard 3.16.
Also, there was a lot of other updates — I just remember the most prominent, the kernel update to 4.9.30 and the non-functional VirtualBox update to 5.08. It really looked like a big upgrade, therefore I was thinking of Stretch.

Looking in /var/log/apt/history.log I see hundreds of updates for this day; arbitrarily picking a few of them:
Install: python-funcsigs:amd64 (1.0.2-3~bpo8+1, automatic), libgphoto2-port12:amd64 (2.5.12-1~bpo8+1, automatic), libldap-common:amd64 (2.4.44+dfsg-5~bpo8+1, automatic),
Upgrade: gnome-commander-data:amd64 (1.4.3-1, 1.4.7-1~bpo8+1), octave-common:amd64 (3.8.2-4, 4.0.3-1~bpo8+1), libreoffice-avmedia-backend-gstreamer:amd64 (4.3.3-2+deb8u7, 5.2.7-1~bpo8+1), libdrm-intel1:amd64 (2.4.58-2, 2.4.74-1~bpo8+1),

So this now looks like quite all available backports were applied — by the daily mintUpdate routine!
How could this happen??
If I would disable the backports repo I wouldn't get their updates any more.
BTW, on one other machine I have the backports repo active and had deliberately installed the 4.9 kernel from there. On that machine nothing like this has happened.
Looks like I suddenly got a nearly cutting-edge distri here

[xenopeek]

What is in the files in the /etc/apt/preferences.d/ directory?

And for example what is the output for this command on one of the upgraded package you shared:
apt policy gnome-commander-data

[Hansl]

What is in the files in the /etc/apt/preferences.d/ directory?
And for example what is the output for this command on one of the upgraded package you shared:
apt policy gnome-commander-data

Looks quite normal so far:

Code: Select all

root@cad1:/etc/apt/preferences.d# ls -l
total 4
-rw-r--r-- 1 root root 61 Nov 25  2015 official-extra-repositories.pref

root@cad1:/etc/apt/preferences.d# cat  official-extra-repositories.pref 
Package: *
Pin: origin build.linuxmint.com
Pin-Priority: 700

root@cad1:/etc/apt/preferences.d# apt policy gnome-commander-data
gnome-commander-data:
  Installed: 1.4.7-1~bpo8+1
  Candidate: 1.4.7-1~bpo8+1
  Version table:
 *** 1.4.7-1~bpo8+1 0
        100 http://httpredir.debian.org/debian/ jessie-backports/main amd64 Packages
        100 /var/lib/dpkg/status
     1.4.3-1 0
        500 ftp://ftp-stud.hs-esslingen.de/debian/ jessie/main amd64 Packages
root@cad1:/etc/apt/preferences.d# 

What would happen if, in the moment of apt's / mintUpdate's refreshing of the repo lists, the ftp://ftp-stud.hs-esslingen.de/ wasn't available?
I had only noticed mintUpdate's blue icon after logging in that morning and had clicked on Install as usual without thinking much, don't remember any error messages.

Just now, while experimenting, I have noticed: If I change sources via mintUpdate, for instance from stud.hs-esslingen.de to netcologne.de, but do not click Refresh, then apt policy does show the backports repo only, but nor the newly selected neither the previous repo! But I didn't meddle with the sources before since many weeks.

[Hansl]

Comparing to other LMDE machines, /etc/apt/preferences.d/official-package-repositories.pref is missing here.
Digging in the backups I see that it never was there — this one is an LMDE installation from 2012, upgraded to Betsy in 2015, updated continually.
Should I copy official-package-repositories.pref from another newer installation? Never noticed problems before.

[Dr. Octagon]

Hello,

I ve got the 4.x kernel two weeks ago, too... and a few days later it was "downgraded" by an automatic update to 3.1x again as it was in the initial state. First, I thought it was a "hack"... then I forgot to write it down here in the forum.

Now I am still running on 3.1x kernel... and no update shows up since this "interrupt".

Btw.
Backports are just enabled for Firejail and LibreOffice here... so the "temporally unavailability of the regular source" really seems to be the "crosspoint" in this case.


Cu
Dr. Octagon

[Hansl]

Now I've heard of another identical incident:  By chance at the same time and from the same ftp-stud.hs-esslingen.de mirrors, my sister's notebook also upgraded everything from backports.  My Start-Date (from /var/log/apt/history.log) was 2017-09-12 10:08:44 ;  hers was 2017-09-12 10:01:43 .  There must have been a problem with that mirror at that time, I presume.  Bad luck?  Or —

I see this as a bug in mintUpdate, isn't it?  Not aborting when the official mirrors are unavailable.

[xenopeek]

This is unfortunate but it's not an issue in Update Manager specifically. The same happens in all other package managers if the user ignores errors they are shown and applies upgrades despite having been shown errors about their repository servers.

I did some testing by setting the Debian repository servers in /etc/apt/sources.list.d/official-package-repositories.list to non existent servers or to servers that would give a timeout. Here's the error you'd have to dismiss and ignore in Update Manager in such cases:


The exact same error is shown if you use apt update on the terminal:

Code: Select all

Err http://ftp.us.debi1an.org jessie InRelease
  
Err http://ftp.us.debi1an.org jessie-updates InRelease
  
Err http://ftp.us.debi1an.org jessie Release.gpg                                                                              
  Could not resolve 'ftp.us.debi1an.org'
Err http://ftp.us.debi1an.org jessie-updates Release.gpg                                                                      
  Could not resolve 'ftp.us.debi1an.org'
Hit http://ftp.debian.org jessie-backports InRelease                                                                          
Get:1 http://ftp.debian.org jessie-backports/main amd64 Packages/DiffIndex [27.8 kB]                                          
Ign http://packages.linuxmint.com betsy InRelease                                                                             
Ign http://extra.linuxmint.com betsy InRelease                                       
Get:2 http://ftp.debian.org jessie-backports/main i386 Packages/DiffIndex [27.8 kB]  
Get:3 http://ftp.debian.org jessie-backports/main Translation-en/DiffIndex [27.8 kB]                                 
Hit http://packages.linuxmint.com betsy Release.gpg                                                        
Hit http://extra.linuxmint.com betsy Release.gpg                                                           
Hit http://packages.linuxmint.com betsy Release                
Hit http://extra.linuxmint.com betsy Release                                          
Hit http://packages.linuxmint.com betsy/main amd64 Packages                                                           
Hit http://extra.linuxmint.com betsy/main amd64 Packages                                                                      
Hit http://packages.linuxmint.com betsy/upstream amd64 Packages                                                               
Hit http://extra.linuxmint.com betsy/main i386 Packages                                                                       
Hit http://packages.linuxmint.com betsy/import amd64 Packages                                                                 
Hit http://packages.linuxmint.com betsy/main i386 Packages                                                                    
Hit http://packages.linuxmint.com betsy/upstream i386 Packages                                                                
Hit http://packages.linuxmint.com betsy/import i386 Packages                                                         
Ign http://extra.linuxmint.com betsy/main Translation-en_US                                                                   
Ign http://extra.linuxmint.com betsy/main Translation-en                                                             
Ign http://packages.linuxmint.com betsy/import Translation-en_US                               
Ign http://packages.linuxmint.com betsy/import Translation-en  
Ign http://packages.linuxmint.com betsy/main Translation-en_US 
Ign http://packages.linuxmint.com betsy/main Translation-en    
Ign http://packages.linuxmint.com betsy/upstream Translation-en_US
Ign http://packages.linuxmint.com betsy/upstream Translation-en
Err http://security.debi1an.org jessie/updates InRelease                                                                      
  
Err http://security.debi1an.org jessie/updates Release.gpg                                                                    
  Could not resolve 'security.debi1an.org'
Fetched 83.4 kB in 10s (8,304 B/s)                                                                                            
Reading package lists... Done
W: Failed to fetch http://ftp.us.debi1an.org/debian/dists/jessie/InRelease  

W: Failed to fetch http://ftp.us.debi1an.org/debian/dists/jessie-updates/InRelease  

W: Failed to fetch http://security.debi1an.org/dists/jessie/updates/InRelease  

W: Failed to fetch http://ftp.us.debi1an.org/debian/dists/jessie/Release.gpg  Could not resolve 'ftp.us.debi1an.org'

W: Failed to fetch http://ftp.us.debi1an.org/debian/dists/jessie-updates/Release.gpg  Could not resolve 'ftp.us.debi1an.org'

W: Failed to fetch http://security.debi1an.org/dists/jessie/updates/Release.gpg  Could not resolve 'security.debi1an.org'

W: Some index files failed to download. They have been ignored, or old ones used instead.

Doing some further testing and looking at the output of /usr/bin/apt list --upgradable before and after mangling the Debian repository servers in /etc/apt/sources.list.d/official-package-repositories.list, I see that indeed afterwards all installed packages that have a newer version in backports then have a candidate upgrade from backports. So whether you use Update Manager, apt or any other APT based package manager—the result would have been the same: ignore errors about repository servers and unexpected things may happen.

Picking one package as an example, libreoffice-writer, taking a look with apt policy libreoffice-writer before mangling the Debian repository servers we see that backports are at priority 100 but there is a repository with a higher (500) priority that has the same package and so that repository is the one from which the candidate upgrade is picked instead. This is as per the apt preferences way of handling priority.

Code: Select all

libreoffice-writer:
  Installed: 1:4.3.3-2+deb8u5
  Candidate: 1:4.3.3-2+deb8u7
  Version table:
     1:5.2.7-1~bpo8+1 0
        100 http://ftp.debian.org/debian/ jessie-backports/main amd64 Packages
     1:4.3.3-2+deb8u7 0
        500 http://ftp.us.debian.org/debian/ jessie/main amd64 Packages
        500 http://security.debian.org/ jessie/updates/main amd64 Packages
 *** 1:4.3.3-2+deb8u5 0
        100 /var/lib/dpkg/status

Running the same command after mangling shows that because there is no repository with a higher priority that has the same package, candidate upgrades come from backports.

Code: Select all

libreoffice-writer:
  Installed: 1:4.3.3-2+deb8u5
  Candidate: 1:5.2.7-1~bpo8+1
  Version table:
     1:5.2.7-1~bpo8+1 0
        100 http://ftp.debian.org/debian/ jessie-backports/main amd64 Packages
 *** 1:4.3.3-2+deb8u5 0
        100 /var/lib/dpkg/status

TL;DR: don't ignore errors package managers give you.

[Hansl]

xenopeek:  Thank you for your time!  You're right, of course.  This would explain what happened. Only, I can't remember seeing such an error popup with many lines when this happened.  I see them sometimes, mostly with a single line, then postponing the update.  My sister also thinks there were no such errors.

Could you have a look, after your tests, whether these errors appear in apt's logs?  Or elsewhere maybe?  Does mintUpdate do logging? I do not see them in my /var/log/apt/history.log, /var/log/apt/term.log or /var/log/aptitude.

Do I understand this right:  Those errors appear while fetching the lists (apt update).  When this fails, apt uses the last saved state which should not be dangerous, just not up-to-date.  But if you switch to another mirror (like you tested with your deliberate typo) and this new mirror fails, then there are no old lists, leaving only the additional low-priority repos — triggering this problem.  I did not switch mirrors then.  Hm.

[xenopeek]

No, when a repository server isn't reachable that repository is ignored entirely. Meaning if you install or upgrade packages, you do so with the package manager applying those changes with no knowledge of that repository. This isn't a two step thing; when a repository server isn't reachable, it is ignored. I tested before and after making a repository server unreachable to compare its behavior. That I also tested before making the repository server unreachable has no impact on this and plays no part in it, other than giving me information to compare

mintupdate writes a log in /tmp/mintUpdate/ but it doesn't log this error. I also didn't find any messages about this in the logs under /var/log. It is a runtime error for the user and doesn't appear to get logged.

[Hansl]

Huh?  You're sure?  The last line in that error message reads — 

Code: Select all

W: Some index files failed to download. They have been ignored, or old ones used instead.

…old ones used instead…  At that point, I can only close the message and mintupdate lists the proposed updates. (Well, at least here I should have noticed the “bpo” in all the new version numbers!  And NOT clicked on “Install Updates”…)
In my experience, after closing such an error message, mintupdate never proposed a mass update to backports.  It just didn't list all new updates.  But then, the mirror had never been completely unreachable, only single indices had been unavailable.

[xenopeek]

…old ones used instead…

No, "they have been ignored, or ...". You skipped the first part

There's a difference between some index files of a server not downloading and the entire server being unreachable. Sure, we can agree it might be better if the APT backend used by the various package managers refused to do anything if one or more errors show while updating. But it doesn't. Instead if issues the user an error and lets the user decide how to proceed (fix the error or ignore it).

Like I showed, if the entire Debian repository server is unreachable the way APT is designed makes it so that as a side effect if you do proceed that upgrades from backports suddenly become candidates for installation because there is not another repository server active with the same packages at a higher priority.

[killer de bug]

When your Mint Update window is being full of new updates all of a sudden, you have to investigate carefully what you are going to install and not click blindly on the install button.
Especially if you realize that lot of these updates are from the backports. It can only be a problem.