Log confusion, help needed.

[JayBird707]

Last week I had an incident where ufw.log went crazy and grew to 50GB. Also kern.log and syslog were filling up with same messages. I got into a panic mode because machine was getting ready to lock and booted with a live cd. I read that logrotate when forced should rebuild any missing log files so I deleted ufw.log, kern.log and syslog and executed logrotate -f /etc/logrotate.conf.

I rebooted my system and it runs fine. But I don't see new ufw.log, kern.log and syslog files in /var/log. So for days I've been searching for answers and spending endless hours trying to figure out if I have any logging now at all. I read some posts that say the files will be rebuilt when needed. So I tried to use the logger command and still can't get any log action.

Here's what I get when I try to see if rsyslog is active:

xxx@xxxy-Studio-xxx ~ $ pidof rsyslogd
1141
xxx@xxx-Studio-xxx ~ $ /etc/init.d/rsyslog status
* rsyslogd is running
xxx@xxx-Studio-xxx ~ $ sudo service rsyslog status
[sudo] password for xxx:
rsyslog start/running, process 1141
xxx@xxx-Studio-xxx ~ $ sudo initctl show-config rsyslog
rsyslog
start on filesystem
stop on runlevel [06]

When I try the logger command I've done things like this but never get a log entry (as user with SUDO and as Root):
logger -p syslog.crit test

Any advice will be greatly appreciated.

[WharfRat]

All indications show that the log daemon is up and running. Does this show anything

Code: Select all

ls -lh /var/log/{syslog,kern.log}

[JayBird707]

Here's the results I don't have a syslog or kern.log. Is there a way I can generate an event to force rsyslog to create them? I tried logger but it doesn't do it.

xxx@xxx-Studio-xxx ~ $ ls -lh /var/log/syslog
ls: cannot access /var/log/syslog: No such file or directory
xxx@xxx-Studio-xxx ~ $ ls -lh /var/log/kern.log
ls: cannot access /var/log/kern.log: No such file or directory
xxx@xxx-Studio-xxx ~ $ ls -lh /var/log/
total 44M
-rw-r--r-- 1 root root 0 Apr 9 12:04 alternatives.log
-rw-r--r-- 1 root root 928 Apr 8 00:26 alternatives.log.1
drwxr-xr-x 2 root root 4.0K Apr 9 12:04 apt
-rw-r--r-- 1 root root 0 Mar 1 06:38 aptitude
-rw-r----- 1 root root 0 Apr 5 16:48 auth.log
-rw-r----- 1 root root 35K Apr 5 21:16 auth.log.1
-rw-r--r-- 1 root root 6.0K Apr 9 17:57 boot.log
-rw-r--r-- 1 root root 64K Jan 5 07:40 bootstrap.log
-rw-rw---- 1 root utmp 0 Apr 9 13:45 btmp
-rw-rw---- 1 root utmp 0 Apr 9 12:04 btmp.1
drwxr-xr-x 2 root root 4.0K Apr 9 13:45 ConsoleKit
drwxr-xr-x 2 root root 4.0K Apr 10 07:41 cups
-rw-r----- 1 root adm 67K Apr 9 17:57 dmesg
-rw-r----- 1 root adm 69K Apr 8 00:24 dmesg.0
-rw-r----- 1 root adm 19K Apr 7 16:11 dmesg.1.gz
-rw-r----- 1 root adm 19K Apr 6 22:04 dmesg.2.gz
-rw-r----- 1 root adm 19K Apr 6 03:47 dmesg.3.gz
-rw-r----- 1 root adm 19K Apr 5 23:50 dmesg.4.gz
-rw-r--r-- 1 root root 4.7K Apr 10 13:30 dpkg.log
-rw-r--r-- 1 root root 33K Apr 8 00:26 dpkg.log.1
-rw-r--r-- 1 root root 950 Apr 5 00:07 dpkg.log.2.gz
-rw-r--r-- 1 root root 32K Mar 5 16:31 faillog
-rw-r--r-- 1 root root 3.7K Feb 8 21:51 fontconfig.log
drwxr-xr-x 2 root root 4.0K Apr 5 22:23 fsck
-rw-r--r-- 1 root root 1.5K Apr 9 17:57 gpu-manager.log
-rw-r--r-- 1 root root 169 Apr 9 16:02 gufw.log
drwxr-xr-x 3 root root 4.0K Apr 5 22:23 hp
drwxr-xr-x 2 root root 4.0K Apr 5 22:23 installer
drwxr-xr-x 2 root root 4.0K Apr 5 22:23 iptraf
-rw-rw-r-- 1 root root 287K Apr 5 23:51 lastlog
drwxr-xr-x 2 root root 4.0K Apr 9 17:57 mdm
-rw-r--r-- 1 root root 5.8K Apr 9 17:57 mintsystem.log
-rw-r--r-- 1 root root 2.4K Apr 9 18:09 pm-powersave.log
-rw-r--r-- 1 root root 12K Apr 8 00:25 pm-powersave.log.1
-rw-r--r-- 1 root root 599 Apr 5 16:06 pm-powersave.log.2.gz
-rw-r--r-- 1 root root 0 Apr 1 07:42 pm-suspend.log
-rw-r--r-- 1 root root 0 Jan 5 08:06 pycentral.log
drwxr-xr-x 3 root root 4.0K Apr 10 07:41 samba
drwxr-xr-x 2 root root 4.0K Apr 5 22:23 speech-dispatcher
-rw-r----- 1 root root 42M Apr 5 21:16 syslog.1
-rw-r--r-- 1 root root 400K Apr 9 17:57 udev
-rw-r----- 1 root root 0 Apr 5 16:48 ufw.log
drwxr-xr-x 2 root root 4.0K Apr 5 22:23 unattended-upgrades
drwxr-xr-x 2 root root 4.0K Apr 9 17:57 upstart
-rw-r--r-- 1 root root 1.3K Feb 10 12:02 vbox-install.log
drwxr-xr-x 2 root root 4.0K Apr 9 17:57 vmware
-rw-r--r-- 1 root root 22K Feb 14 07:36 vmware-installer
-rw-r--r-- 1 root root 325K Apr 9 17:57 vnetlib
-rw-rw-r-- 1 root utmp 15K Apr 10 15:49 wtmp
-rw-rw-r-- 1 root utmp 768 Apr 9 12:27 wtmp.1
-rw-r--r-- 1 root root 112K Apr 10 15:05 Xorg.0.log
-rw-r--r-- 1 root root 209K Apr 9 17:55 Xorg.0.log.old
-rw-r--r-- 1 root root 59K Mar 24 17:07 Xorg.20.log

[WharfRat]

Try writing to syslog

Code: Select all

logger "This is a test message for syslog" && tail /var/log/syslog

I didn't see that already tried logger

Did you change any permissions

[JayBird707]

I don't have a syslog file in /var/log/ and to the best of my knowledge I never changed any ownerships. I also don't have a kern.log or ufw.log in /var/log/.

I tried your command suggestion here is what happen

xxx@xxx-Studio-xxx ~ $ sudo logger "This is a test message for syslog" && tail /var/log/syslog
[sudo] password for xxx:
tail: cannot open ‘/var/log/syslog’ for reading: No such file or directory

I was under the impression that if syslog did not exist that using logger would force rsyslog to create it and then log the entry. But this is not happening.

[WharfRat]

Try creating them and see if they start filling up after a while

Code: Select all

sudo touch /var/log/syslog /var/log/kern.log

Code: Select all

sudo chown syslog:adm /var/log/syslog

Code: Select all

sudo chown syslog:adm /var/log/kern.log

Since logger didn't create syslog, perhaps the same is happening with rsyslogd.

[JayBird707]

Thank you so much. That got the logging going again. Could you be so kind as to look at this list of /var/log/ and see if the ownerships of any of the others need to be revised.

jeffrey-Studio-1737 jeffrey # ls -lash /var/log/
total 44M
4.0K drwxr-xr-x 15 root root 4.0K Apr 10 19:16 .
4.0K drwxr-xr-x 12 root root 4.0K Apr 5 22:26 ..
0 -rw-r--r-- 1 root root 0 Apr 9 12:04 alternatives.log
4.0K -rw-r--r-- 1 root root 928 Apr 8 00:26 alternatives.log.1
4.0K drwxr-xr-x 2 root root 4.0K Apr 9 12:04 apt
0 -rw-r--r-- 1 root root 0 Mar 1 06:38 aptitude
0 -rw-r----- 1 root root 0 Apr 5 16:48 auth.log
36K -rw-r----- 1 root root 35K Apr 5 21:16 auth.log.1
8.0K -rw-r--r-- 1 root root 6.0K Apr 9 17:57 boot.log
64K -rw-r--r-- 1 root root 64K Jan 5 07:40 bootstrap.log
0 -rw-rw---- 1 root utmp 0 Apr 9 13:45 btmp
0 -rw-rw---- 1 root utmp 0 Apr 9 12:04 btmp.1
4.0K drwxr-xr-x 2 root root 4.0K Apr 9 13:45 ConsoleKit
4.0K drwxr-xr-x 2 root root 4.0K Apr 10 07:41 cups
68K -rw-r----- 1 root adm 67K Apr 9 17:57 dmesg
72K -rw-r----- 1 root adm 69K Apr 8 00:24 dmesg.0
20K -rw-r----- 1 root adm 19K Apr 7 16:11 dmesg.1.gz
20K -rw-r----- 1 root adm 19K Apr 6 22:04 dmesg.2.gz
20K -rw-r----- 1 root adm 19K Apr 6 03:47 dmesg.3.gz
20K -rw-r----- 1 root adm 19K Apr 5 23:50 dmesg.4.gz
8.0K -rw-r--r-- 1 root root 4.7K Apr 10 13:30 dpkg.log
40K -rw-r--r-- 1 root root 33K Apr 8 00:26 dpkg.log.1
4.0K -rw-r--r-- 1 root root 950 Apr 5 00:07 dpkg.log.2.gz
32K -rw-r--r-- 1 root root 32K Mar 5 16:31 faillog
4.0K -rw-r--r-- 1 root root 3.7K Feb 8 21:51 fontconfig.log
4.0K drwxr-xr-x 2 root root 4.0K Apr 5 22:23 fsck
4.0K -rw-r--r-- 1 root root 1.5K Apr 9 17:57 gpu-manager.log
4.0K -rw-r--r-- 1 root root 296 Apr 10 19:18 gufw.log
4.0K drwxr-xr-x 3 root root 4.0K Apr 5 22:23 hp
4.0K drwxr-xr-x 2 root root 4.0K Apr 5 22:23 installer
4.0K drwxr-xr-x 2 root root 4.0K Apr 5 22:23 iptraf
152K -rw-r--r-- 1 syslog adm 152K Apr 10 19:19 kern.log
288K -rw-rw-r-- 1 root root 287K Apr 5 23:51 lastlog
4.0K drwxr-xr-x 2 root root 4.0K Apr 9 17:57 mdm
8.0K -rw-r--r-- 1 root root 5.8K Apr 9 17:57 mintsystem.log
4.0K -rw-r--r-- 1 root root 2.4K Apr 9 18:09 pm-powersave.log
12K -rw-r--r-- 1 root root 12K Apr 8 00:25 pm-powersave.log.1
4.0K -rw-r--r-- 1 root root 599 Apr 5 16:06 pm-powersave.log.2.gz
0 -rw-r--r-- 1 root root 0 Apr 1 07:42 pm-suspend.log
0 -rw-r--r-- 1 root root 0 Jan 5 08:06 pycentral.log
4.0K drwxr-xr-x 3 root root 4.0K Apr 10 07:41 samba
4.0K drwxr-xr-x 2 root root 4.0K Apr 5 22:23 speech-dispatcher
128K -rw-r--r-- 1 syslog adm 127K Apr 10 19:19 syslog
42M -rw-r----- 1 root root 42M Apr 5 21:16 syslog.1
400K -rw-r--r-- 1 root root 400K Apr 9 17:57 udev
92K -rw-r----- 1 syslog adm 90K Apr 10 19:19 ufw.log
4.0K drwxr-xr-x 2 root root 4.0K Apr 5 22:23 unattended-upgrades
4.0K drwxr-xr-x 2 root root 4.0K Apr 9 17:57 upstart
4.0K -rw-r--r-- 1 root root 1.3K Feb 10 12:02 vbox-install.log
4.0K drwxr-xr-x 2 root root 4.0K Apr 9 17:57 vmware
24K -rw-r--r-- 1 root root 22K Feb 14 07:36 vmware-installer
332K -rw-r--r-- 1 root root 325K Apr 9 17:57 vnetlib
16K -rw-rw-r-- 1 root utmp 15K Apr 10 15:49 wtmp
4.0K -rw-rw-r-- 1 root utmp 768 Apr 9 12:27 wtmp.1
112K -rw-r--r-- 1 root root 112K Apr 10 15:05 Xorg.0.log
212K -rw-r--r-- 1 root root 209K Apr 9 17:55 Xorg.0.log.old
60K -rw-r--r-- 1 root root 59K Mar 24 17:07 Xorg.20.log

[WharfRat]

It looks OK to me. Here's my list so you can double-check, I left out the .gz files

Code: Select all

[bill@rosa] /var/log $ sudo find /var/log/  \( ! -group root -or ! -user root \) -and  ! -group root  -ls|grep -v \.gz|sort -k11
1687560    4 drwxrwxr-x  16 root     syslog       4096 Apr 10 18:47 /var/log/
1688419    0 -rw-r-----   1 root     adm             0 Mar 30 08:08 /var/log/apport.log
1688056    4 -rw-r-----   1 root     adm           513 Mar 29 11:00 /var/log/apport.log.1
1688052   20 -rw-r-----   1 root     adm         18624 Apr  7 17:21 /var/log/apt/term.log
1691051  556 -rw-r-----   1 syslog   adm        563798 Apr 10 19:38 /var/log/auth.log
1689249  408 -rw-r-----   1 syslog   adm        412971 Apr  3 08:10 /var/log/auth.log.1
1688037    0 -rw-rw----   1 root     utmp            0 Apr  1 07:39 /var/log/btmp
1688079    0 -rw-rw----   1 root     utmp            0 Mar  1 07:31 /var/log/btmp.1
1687935    0 -rw-r-----   1 root     adm             0 Apr  9 08:15 /var/log/cups/access_log
1688047    4 -rw-r-----   1 root     adm           113 Apr 10 07:01 /var/log/cups/error_log
1688790    0 -rw-r-----   1 root     adm             0 Feb 24 08:08 /var/log/cups/page_log
1688165   64 -rw-r-----   1 root     adm         59939 Apr 10 06:56 /var/log/dmesg
1688030   64 -rw-r-----   1 root     adm         60115 Apr  9 08:10 /var/log/dmesg.0
478832    4 -rw-r-----   1 root     adm            31 May 29  2014 /var/log/fsck/checkfs
478833    4 -rw-r-----   1 root     adm            31 May 29  2014 /var/log/fsck/checkroot
1689651    4 drwxrwxr-x   2 root     lp           4096 Apr  7  2014 /var/log/hp/tmp
498181  192 -rw-------   1 syslog   adm        189297 Jan 20 12:10 /var/log/installer/syslog
1690621  696 -rw-r-----   1 syslog   adm        706483 Apr 10 19:04 /var/log/kern.log
1688865  660 -rw-r-----   1 syslog   adm        669834 Apr  3 08:06 /var/log/kern.log.1
454527  292 -rw-rw-r--   1 root     utmp       292876 Apr  6 20:21 /var/log/lastlog
1687820   40 -rw-r-----   1 syslog   adm         37288 Apr 10 19:35 /var/log/syslog
1687868  152 -rw-r-----   1 syslog   adm        150602 Apr 10 07:02 /var/log/syslog.1
1687991   96 -rw-rw-r--   1 root     utmp        93312 Apr 10 11:43 /var/log/wtmp
1687947  264 -rw-rw-r--   1 root     utmp       263424 Apr  1 07:34 /var/log/wtmp.1

[JayBird707]

I think I just realized what may have happen. when the logs blew up on me last week I moved all the files in /var onto a separate logical volume and then mounted it at /var. When I moved the files I actually used rsync which I thought kept all the ownerships in tact. Maybe this didn't work the way I thought.

Once again thank you very much for your help. I am only 2 months into Linux and rescued from the proprietary OS world. I can't believe how great the Linux community is. I'm learning a lot and thus far can still do all my work and haven't killed my computer.