[SOLVED] Huge log folder

[Stepukas]

Hello.

It kept popping, that my local/root has low disk space. I checked what causes that, and it seems my /var/log folder is eating up the space and that kern.log is one huge file at over 300 GB. Maybe any ideas why is that happening and how can i fix it?

[Moonstone Man]

Maybe any ideas why is that happening and how can i fix it?

It's caused by some error spamming the logs. The way to fix it is to discover what's causing it, then fix that.

Please post the output of dmesg --level=err | tail -n 20 from a terminal and enclose it in code tags [code]output.here[/code]. You'll see the code tags icon </> when you reply.

For now, you can delete those logs afterwards but not before. Also post the output of inxi -Fxz.

[Stepukas]

Code: Select all

[    0.769535] ACPI BIOS Error (bug): Failure creating named object [\_SB.PCI0.GPP8._DSM], AE_ALREADY_EXISTS (20190816/dswload2-326)
[    0.769542] ACPI Error: AE_ALREADY_EXISTS, During name lookup/catalog (20190816/psobject-220)

For now, you can delete those logs afterwards but not before.

What do you mean by that?

Code: Select all

System:
  Kernel: 5.4.0-66-generic x86_64 bits: 64 compiler: gcc v: 9.3.0 
  Desktop: Cinnamon 4.8.6 Distro: Linux Mint 20.1 Ulyssa 
  base: Ubuntu 20.04 focal 
Machine:
  Type: Desktop System: Gigabyte product: X570 AORUS ULTRA v: -CF 
  serial: <filter> 
  Mobo: Gigabyte model: X570 AORUS ULTRA v: x.x serial: <filter> 
  UEFI: American Megatrends v: F20a date: 06/16/2020 
CPU:
  Topology: 12-Core model: AMD Ryzen 9 3900X bits: 64 type: MT MCP arch: Zen 
  L2 cache: 6144 KiB 
  flags: avx avx2 lm nx pae sse sse2 sse3 sse4_1 sse4_2 sse4a ssse3 svm 
  bogomips: 182044 
  Speed: 2195 MHz min/max: 2200/3800 MHz Core speeds (MHz): 1: 2195 2: 2196 
  3: 2196 4: 2195 5: 2196 6: 2195 7: 2195 8: 2191 9: 2193 10: 2189 11: 2188 
  12: 2195 13: 2195 14: 2192 15: 2195 16: 2196 17: 2188 18: 2194 19: 2193 
  20: 2193 21: 2195 22: 2196 23: 2194 24: 2194 
Graphics:
  Device-1: AMD Navi 10 [Radeon RX 5600 OEM/5600 XT / 5700/5700 XT] 
  vendor: Tul driver: amdgpu v: kernel bus ID: 0c:00.0 
  Display: x11 server: X.Org 1.20.9 driver: amdgpu,ati 
  unloaded: fbdev,modesetting,radeon,vesa 
  resolution: 2560x1440~60Hz, 1680x1050~60Hz 
  OpenGL: renderer: AMD Radeon RX 5700 XT (NAVI10 DRM 3.35.0 
  5.4.0-66-generic LLVM 11.0.0) 
  v: 4.6 Mesa 20.2.6 direct render: Yes 
Audio:
  Device-1: AMD Navi 10 HDMI Audio driver: snd_hda_intel v: kernel 
  bus ID: 0c:00.1 
  Device-2: AMD Starship/Matisse HD Audio vendor: Gigabyte 
  driver: snd_hda_intel v: kernel bus ID: 0e:00.4 
  Device-3: Realtek USB2.0 Hub type: USB driver: snd-usb-audio,uvcvideo 
  bus ID: 5-4.4:6 
  Sound Server: ALSA v: k5.4.0-66-generic 
Network:
  Device-1: Intel Wi-Fi 6 AX200 driver: iwlwifi v: kernel bus ID: 05:00.0 
  IF: wlp5s0 state: up mac: <filter> 
  Device-2: Intel I211 Gigabit Network vendor: Gigabyte driver: igb 
  v: 5.6.0-k port: f000 bus ID: 06:00.0 
  IF: enp6s0 state: down mac: <filter> 
Drives:
  Local Storage: total: 3.64 TiB used: 1.51 TiB (41.4%) 
  ID-1: /dev/nvme0n1 vendor: Sabrent model: Rocket 4.0 1TB size: 931.51 GiB 
  ID-2: /dev/nvme1n1 vendor: Corsair model: Force MP510 size: 447.13 GiB 
  ID-3: /dev/sda vendor: Crucial model: CT1000MX500SSD1 size: 931.51 GiB 
  ID-4: /dev/sdb vendor: Samsung model: SSD 840 EVO 250GB size: 232.89 GiB 
  ID-5: /dev/sdc vendor: Seagate model: ST1000DM010-2EP102 size: 931.51 GiB 
  ID-6: /dev/sdd vendor: Samsung model: HD753LJ size: 698.64 GiB 
Partition:
  ID-1: / size: 913.74 GiB used: 718.99 GiB (78.7%) fs: ext4 dev: /dev/dm-1 
  ID-2: /boot size: 704.5 MiB used: 108.3 MiB (15.4%) fs: ext4 
  dev: /dev/nvme0n1p2 
  ID-3: swap-1 size: 976.0 MiB used: 0 KiB (0.0%) fs: swap dev: /dev/dm-2 
Sensors:
  System Temperatures: cpu: 51.8 C mobo: N/A gpu: amdgpu temp: 42 C 
  Fan Speeds (RPM): N/A gpu: amdgpu fan: 0 
Info:
  Processes: 507 Uptime: 2m Memory: 62.82 GiB used: 1.77 GiB (2.8%) 
  Init: systemd runlevel: 5 Compilers: gcc: 9.3.0 Shell: bash v: 5.0.17 
  inxi: 3.0.38 
  

[Pjotr]

Try this:
Update Manager - panel: View - Linux kernels
Install the latest kernel of the 5.8 series. Then reboot.

Also worth a try: a BIOS update. You might need to do that in Windows.

If solving the underlying problem (with the help of Kadaitcha man) doesn't succeed, you can tame your logs once and for all, like this:
https://easylinuxtipsproject.blogspot.c ... .html#ID10
(items 10, 11 and 12).

[Stepukas]

OK, so i updated my BIOS, changed kernel to newest 5.8 series.

Also manually deleted ker.log, syslog and syslog.1 files.

I guess now i just need to keep an eye if the problem persists.

Also, if i look to the drive in GParted, it shows that there are 0 unused space in my root partition. Dunno if it should be that way.

Thank you for the answers, guys. If you have any other insights into this, i would be grateful.

[karlchen]

Hello, Stepukas.

While a system is up and running, you cannot and should not try to delete the active logfiles like syslog and kern.log.
The system will remove the filenames from the directory /var/log, but the rsyslogd daemon will not release the corresponding inodes and keep on writing into the inodes.
As a result disk space will still be used, although the files themselves are no longer accessible through their names.

What you may delete are the backup copies: syslog.1, syslog.?.gz, kern.log.1, kern.log.?.gz
The actively used files syslog and kern.log can only be truncated at runtime:

Code: Select all

# as user root (sudo -i) do
cd /var/log
cat /dev/null > syslog
cat /dev/null > kern.log

Beware that as long as you have not resolved the underlying root cause, which makes the system blow up the files at enormous speed to enormous sizes, the files may grow more quickly than you will be able to truncate them.

Plus one more caveat:
Ever since the system is managed by systemd the system journals may grow quickly as well and become really large as well.
And you should definitely not fiddle around with system journal files using commands like "rm" or "cat /dev/null". Look up the journalctl commands instead, which permit clearing the system journals.

The most important thing however is identifying the underlying root cause and resolving it. Resolving the root cause will silence the messengers. Killing the messengers, though very popular, will resolve nothing.

Cheers,
Karl

[Stepukas]

Hello, Karl.

Thank you for your advice. I found at several places at forums that deleting these files is safe, so i did so... My idea was to delete them and watch if they fill up with GB's again and if so, when. The underlying problem might be already gone (as i don't know when did these log files grew so large, that may be in the past). But your proposition to truncate the files is obviously better. But as of now, i already deleted them and as i understand, nothing long lasting happened - as soon as i rebooted, system created these files anew. Or am i wrong? Are you saying that the logging system ate some disc space permanently?

So far so good. Or at least it seems so. Log files are of kB size.

[karlchen]

Hello, Stepukas.

It is true that deleting logfiles /var/log/syslog and /var/log/kern.log will not cause any permanent harm. In case the files do not exist when the system starts up, then it will create them. This is correct as well.

Best regards,
Karl