Linux Mint 18 journal / system log primer

Write tutorials here
There are more tutorials here http://community.linuxmint.com/tutorial/welcome
Forum rules
Please don't add support questions to tutorials,start your own thread in the appropriate sub-forum instead. Before you post please read this
User avatar
xenopeek
Level 24
Level 24
Posts: 21367
Joined: Wed Jul 06, 2011 3:58 am
Location: The Netherlands

Linux Mint 18 journal / system log primer

Postby xenopeek » Sun Jun 05, 2016 1:51 pm

Linux Mint 18 uses systemd's system logging service called the journal. Previous versions of Linux Mint used rsyslog. The journal offers more functionality but if you don't care about that the journal is by default configured to also write log message to the old familiar file /var/log/syslog so you may continue to use that.

For Linux Mint 18 Cinnamon you'll find the System Log program in your menu which shows you the log messages from the file /var/log/syslog. If you want a graphical program to instead show you the log messages from the journal directly (and offer more ways to filter and search the logs), install the package gnome-logs. It will show up as the program called Logs in your menu. It will show you important messages from the current boot by default. This is a handy program to have installed, to quickly see if any error messages are reported in your logs. For most users having this program will suffice.

You would otherwise access the journal with the command journalctl, which has more ways to filter messages, and that is what this tutorial will focus on. By default it uses less as its pager but just like with the /var/log/syslog file you can pipe the output of journalctl into other programs. For example to grep for something in its output. But the journal has a lot of contextual information that you can use with journalctl to filter log messages. I'll give some practical examples below. You can find more information in the journalctl manpage.

Filter by boot or time
systemd keeps track of each boot and the journal can filter on that. In its simplest form you can use journalctl -b to only show log messages from the current boot. More likely you are interested in the log messages from the previous boot after you had a problem that caused you to reboot. You access that with journalctl -b -1 and you can give a more negative number to go back to earlier boots (e.g., journalctl -b -2 to go back 2 boots before the current). You can also see the start time of each boot that is in the journal with the command journalctl --list-boots. In the first column this shows you the negative number you can use with the -b option. You can also the the 32 character id after that with the -b option, to uniquely identify a specific boot.

But perhaps you rebooted a couple of times to try and fix some issue and you want to see the log messages of all those boots. You can pass the -b option multiple times to select multiple boots. But you can also filter by time. You can filter from a specific time with journalctl -S timestamp (or alternatively journalctl --since=timestamp) and you can filter to a specific time with journalctl -U timestamp (or alternatively journalctl --until=timestamp). For timestamp you can use an ISO timestamp like "YYYY-MM-DD hh24:mm:ss" (if you don't specify the date, the current day is assumed; if you don't specify the time, 00:00:00 is assumed; if you don't specify the seconds :00 is assumed). You can also use the keywords "yesterday" and "today" which assume 00:00:00 on that day, and "now" which is the current day and time. However it understands many more keywords so you can do journalctl -S -5weeks+2days -U -2days which would show you all log messages from 5 weeks and 2 days ago till 2 days ago. See the systemd.time manpage for a full description of the keywords you can use in the timestamp.

Filter by priority
Each log message is logged with a priority: emerg (0), alert (1), crit (2), err (3), warning (4), notice (5), info (6), and debug (7). You can filter log messages by priority with journalctl -p priority or you can indicate a range with journalctl -p priority..priority. For priority you may use either the name or the number from the list. In the form where you give only one priority it would also show you more important messages. So journalctl -p err (or alternatively journalctl -p 3) would show you emerg, alert, and crit messages as well. In the form where you give two priorities only message with a priority in that range are shown. So journalctl -p warning..notice (or alternatively journalctl -p 4..5) would only show you warning and notice messages.

BTW, notice that journalctl colors err (and worse) messages in red and highlights both warning and notice messages. Messages of priority info or debug are not colored or highlighted.

Filter by service, other fields, and programs
If there is a specific service you're having trouble with you may only be interested in messages from that service. With systemd init the services each have a so called unit file but that goes beyond the scope of this tutorial (I'll cover systemd init and the systemctl command in a next tutorial). Anyway, you can filter by service unit with the command journalctl -u name or journalctl UNIT=name.service. In the first form the .service extension is assumed if you don't specify it. For example to see all messages from the cups (printer) service you would use journalctl -u cups or journalctl -u cups.service or journalctl UNIT=cups.service.

You may be able to guess the unit name to use but another method is to use journalctl -o verbose to display all the fields of information that the journal has. Find the log message you are interested in, note the value of the UNIT field and then use that to filter messages only from that service. You'll see a lot more fields in the verbose output and you can filter on each field here exactly as you can with the UNIT field.

You may also give the path of an executable as an argument to journalctl and it will display all the entries that involve the executable in question (you can find these also as the _EXE field in the verbose output).

Some other options
You can use dmesg to show kernel messages. While the dmesg command has various ways to filter the messages it knows only messages from the current boot. You can use journalctl -k to display the same kernel messages. The journal also has the kernel messages from previous boots so you can use the above mentioned options to filter by boot or time to see kernel messages from before the current boot. It will also color and highlight kernel messages based on priority.

If you want to investigate some issue as it is happening you may find journalctl -f useful. It will display the 10 most recent messages and then wait for new messages and print them as they arrive.
Image

Habitual
Level 13
Level 13
Posts: 4866
Joined: Sun Nov 21, 2010 8:31 pm
Location: 0.0.0.0

Re: Linux Mint 18 journal / system log primer

Postby Habitual » Mon Jun 06, 2016 9:24 am

You so rock.

Thank you.
Image

User avatar
Flemur
Level 13
Level 13
Posts: 4540
Joined: Mon Aug 20, 2012 9:41 pm
Location: Potemkin Village

Re: Linux Mint 18 journal / system log primer

Postby Flemur » Mon Jun 06, 2016 3:24 pm

Great info, thanks!
Mint 18.2 Xfce/fluxbox/pulse-less - Xubuntu 16.10/fluxbox/pulse-less
Please edit your original post title to include [SOLVED] if/when it is solved!
Your data and OS are backed up....right?

JusTertii
Level 3
Level 3
Posts: 171
Joined: Tue Jul 29, 2014 11:21 pm

Re: Linux Mint 18 journal / system log primer

Postby JusTertii » Wed Jun 08, 2016 7:12 am

Fantastic! Thanks.

Hopefully I won't need it too much when LM 18 comes out, but good to know just in case.


Return to “Tutorials”

Who is online

Users browsing this forum: No registered users and 5 guests