Linux Mint 18 journal / system log primer

Write tutorials for Linux Mint here
More tutorials on https://github.com/orgs/linuxmint/discu ... /tutorials and (archive) on https://community.linuxmint.com/tutorial
Forum rules
Don't add support questions to tutorials; start your own topic in the appropriate sub-forum instead. Before you post read forum rules
Post Reply
User avatar
xenopeek
Level 25
Level 25
Posts: 29507
Joined: Wed Jul 06, 2011 3:58 am

Linux Mint 18 journal / system log primer

Post by xenopeek »

Linux Mint 18 uses systemd's system logging service called the journal. Previous versions of Linux Mint used rsyslog. The journal offers more functionality but if you don't care about that the journal is by default configured to also write log message to the old familiar file /var/log/syslog so you may continue to use that.

For Linux Mint 18 Cinnamon you'll find the System Log program in your menu which shows you the log messages from the file /var/log/syslog. If you want a graphical program to instead show you the log messages from the journal directly (and offer more ways to filter and search the logs), install the package gnome-logs. It will show up as the program called Logs in your menu. It will show you important messages from the current boot by default. This is a handy program to have installed, to quickly see if any error messages are reported in your logs. For most users having this program will suffice.

You would otherwise access the journal with the command journalctl, which has more ways to filter messages, and that is what this tutorial will focus on. By default it uses less as its pager but just like with the /var/log/syslog file you can pipe the output of journalctl into other programs. For example to grep for something in its output. But the journal has a lot of contextual information that you can use with journalctl to filter log messages. I'll give some practical examples below. You can find more information in the journalctl manpage.

Filter by boot or time
systemd keeps track of each boot and the journal can filter on that. In its simplest form you can use journalctl -b to only show log messages from the current boot. More likely you are interested in the log messages from the previous boot after you had a problem that caused you to reboot. You access that with journalctl -b -1 and you can give a more negative number to go back to earlier boots (e.g., journalctl -b -2 to go back 2 boots before the current). You can also see the start time of each boot that is in the journal with the command journalctl --list-boots. In the first column this shows you the negative number you can use with the -b option. You can also the the 32 character id after that with the -b option, to uniquely identify a specific boot.

But perhaps you rebooted a couple of times to try and fix some issue and you want to see the log messages of all those boots. You can pass the -b option multiple times to select multiple boots. But you can also filter by time. You can filter from a specific time with journalctl -S timestamp (or alternatively journalctl --since=timestamp) and you can filter to a specific time with journalctl -U timestamp (or alternatively journalctl --until=timestamp). For timestamp you can use an ISO timestamp like "YYYY-MM-DD hh24:mm:ss" (if you don't specify the date, the current day is assumed; if you don't specify the time, 00:00:00 is assumed; if you don't specify the seconds :00 is assumed). You can also use the keywords "yesterday" and "today" which assume 00:00:00 on that day, and "now" which is the current day and time. However it understands many more keywords so you can do journalctl -S -5weeks+2days -U -2days which would show you all log messages from 5 weeks and 2 days ago till 2 days ago. See the systemd.time manpage for a full description of the keywords you can use in the timestamp.

Filter by priority
Each log message is logged with a priority: emerg (0), alert (1), crit (2), err (3), warning (4), notice (5), info (6), and debug (7). You can filter log messages by priority with journalctl -p priority or you can indicate a range with journalctl -p priority..priority. For priority you may use either the name or the number from the list. In the form where you give only one priority it would also show you more important messages. So journalctl -p err (or alternatively journalctl -p 3) would show you emerg, alert, and crit messages as well. In the form where you give two priorities only message with a priority in that range are shown. So journalctl -p warning..notice (or alternatively journalctl -p 4..5) would only show you warning and notice messages.

BTW, notice that journalctl colors err (and worse) messages in red and highlights both warning and notice messages. Messages of priority info or debug are not colored or highlighted.

Filter by service, other fields, and programs
If there is a specific service you're having trouble with you may only be interested in messages from that service. With systemd init the services each have a so called unit file but that goes beyond the scope of this tutorial (I'll cover systemd init and the systemctl command in a next tutorial). Anyway, you can filter by service unit with the command journalctl -u name or journalctl UNIT=name.service. In the first form the .service extension is assumed if you don't specify it. For example to see all messages from the cups (printer) service you would use journalctl -u cups or journalctl -u cups.service or journalctl UNIT=cups.service.

You may be able to guess the unit name to use but another method is to use journalctl -o verbose to display all the fields of information that the journal has. Find the log message you are interested in, note the value of the UNIT field and then use that to filter messages only from that service. You'll see a lot more fields in the verbose output and you can filter on each field here exactly as you can with the UNIT field.

You may also give the path of an executable as an argument to journalctl and it will display all the entries that involve the executable in question (you can find these also as the _EXE field in the verbose output).

Some other options
You can use dmesg to show kernel messages. While the dmesg command has various ways to filter the messages it knows only messages from the current boot. You can use journalctl -k to display the same kernel messages. The journal also has the kernel messages from previous boots so you can use the above mentioned options to filter by boot or time to see kernel messages from before the current boot. It will also color and highlight kernel messages based on priority.

If you want to investigate some issue as it is happening you may find journalctl -f useful. It will display the 10 most recent messages and then wait for new messages and print them as they arrive.
Image
Habitual

Re: Linux Mint 18 journal / system log primer

Post by Habitual »

You so rock.

Thank you.
User avatar
Flemur
Level 20
Level 20
Posts: 10097
Joined: Mon Aug 20, 2012 9:41 pm
Location: Potemkin Village

Re: Linux Mint 18 journal / system log primer

Post by Flemur »

Great info, thanks!
Please edit your original post title to include [SOLVED] if/when it is solved!
Your data and OS are backed up....right?
JusTertii

Re: Linux Mint 18 journal / system log primer

Post by JusTertii »

Fantastic! Thanks.

Hopefully I won't need it too much when LM 18 comes out, but good to know just in case.
Post Reply

Return to “Tutorials”