inotify

About writing shell scripts and making the most of your shell
Forum rules
Before you post please read how to get help
Post Reply
1000
Level 4
Level 4
Posts: 410
Joined: Wed Jul 29, 2020 2:14 am

inotify

Post by 1000 »

We have very nice tools for check internet connections.
For example

Code: Select all

lsof -i :0-65535
And we can run this with loop for get data in in continuous mode

Code: Select all

lsof -i :0-65535 -r2
A bit too much data?
Then we can clean it up a little.

Code: Select all

#!/bin/bash


echo "{NUMBER OF CONNECTIONS} , NAME , PID:"
while true ; do
	ALL_CON=$(lsof -i :0-65535 | sed '1d')
	# We want only NAME + PID of app to compare 
	ONLY_NEW_CONN=$(awk '{print $1, $2}' <<< "$ALL_CON" | sort | uniq -c | sort)

	# We want compare old output with new output of lsof command 
	OLD_1="$NEW_1"
	NEW_1="$ONLY_NEW_CONN"

	COMPARE=$(comm -13 <(echo "$OLD_1") <(echo "$NEW_1"))

	if [ ! -z "$COMPARE" ] ; then
		echo "$COMPARE"
	fi
	
	# The lsof command refreshes every 2 seconds 
	sleep 2
done
And we have something like that
# bash netstat1
{NUMBER OF CONNECTIONS} , NAME , PID:
1 NetworkMa 1111
3 netdata 1386
8 firefox 2220
7 firefox 2220
The script only shows a newline if these conditions are met
- if the number of connections changes
- if the application name changes

The script is not perfect as the system may cheat me.
the application can close one connection and open a new one at the same time.
So the script may not notice the difference.
So I should add more variables to compare. For example, a connection name.

For now, it does not matter.
I am wondering about something other.

- Using a command in a loop generates a load for the computer.
- Command lsof use files from /proc/

" The inotify API provides a mechanism for monitoring file system events. inotify can be used to monitor individual files, or to monitor directories.
When a directory is monitored, inotify will return events for the directory itself, and for files inside the directory.
... Inotify does report some but not all events in sysfs and procfs. "


It just doesn't watch the file, but reports when you try to do something,
so it doesn't overload the computer too much.

I'd like to try build a similar script, but without "lsof" command.
And compare the performance which is better.

/proc/ , /proc/*/net this is not my strong side. It will take me some time to research /proc folder.
What do you think? Is it possible?
Because I have doubts.
If the file does not exist, is it possible use inotify?
rene
Level 17
Level 17
Posts: 7558
Joined: Sun Mar 27, 2016 6:58 pm

Re: inotify

Post by rene »

1000 wrote:
Wed May 12, 2021 10:52 am
If the file does not exist, is it possible use inotify?
Not on the file itself, but you can watch the directory it pops up in. From a script, after sudo apt-get install inotify-tools, e.g.

Code: Select all

rene@hp8k:~$  inotifywait -m ~/tmp
Setting up watches.
Watches established.
/home/rene/tmp/ CREATE foo
/home/rene/tmp/ OPEN foo
/home/rene/tmp/ ATTRIB foo
/home/rene/tmp/ CLOSE_WRITE,CLOSE foo
/home/rene/tmp/ MODIFY foo
/home/rene/tmp/ OPEN foo
/home/rene/tmp/ MODIFY foo
/home/rene/tmp/ CLOSE_WRITE,CLOSE foo
/home/rene/tmp/ DELETE foo
as a result of in another terminal touch ~/tmp/foo; echo bar >~/tmp/foo; rm ~/tmp/foo. So you just watch for the file appearing...

That said, your own version I would quickly find good enough --- and there's probably tons of race-conditions with inotifywait to deal with if you use that.
1000
Level 4
Level 4
Posts: 410
Joined: Wed Jul 29, 2020 2:14 am

Re: inotify

Post by 1000 »

For now I will abandon the topic.

Conclusions:
1. I'm not sure, but I guess I should be monitoring the condition
/proc/net/tcp
/proc/net/udp
and maybe something.

I wrote a script to translate hexadecimal to decimal local_address and rem_address from there.
The number of connections and the IP and ports numbers looks identical with lsof -Pi command.

Path /proc/*/fd/ is too variable / changeable.

2. You're right, I can't monitor files inside /proc/ It is virtual filesystem or something.
https://tldp.org/LDP/Linux-Filesystem-H ... /proc.html

It's a pity because I noticed that I can monitor several files simultaneously in one script

Code: Select all

inotifywait -q --event modify --format '%w' /path/file_1  /path/file_2
3. Workaround.
I can monitor network with iptables firewall, and I can log output ( without ping to the router ) to a separate file.
Then I can use inotify to monitor file / log of iptables.

But
- For a server where the number of connections is small, this may be a good solution.
For a home user, where one website can create 30 Internet connections, it will be a very heavy load for the hard drive.
- I don't know how to save the log to a separate file.
Therefore, I will not test it for now.

Thank you very much for the suggestions.
rene
Level 17
Level 17
Posts: 7558
Joined: Sun Mar 27, 2016 6:58 pm

Re: inotify

Post by rene »

1000 wrote:
Sun May 16, 2021 4:29 am
2. You're right, I can't monitor files inside /proc/ It is virtual filesystem or something.
Certainly it's a virtual filesystem, i.e, with files/directories not backed by regular storage but "made up" on the spot by the kernel when in fact read/written by you, but that in and of itself is not a reason you can't monitor files there. It's just that a file that does not exist you can not monitor on whichever filesysten; only a directory in which it would pop up.

But yes, otherwise no comments on your conclusion.
User avatar
Termy
Level 7
Level 7
Posts: 1621
Joined: Mon Sep 04, 2017 8:49 pm
Location: UK
Contact:

Re: inotify

Post by Termy »

I'm not exactly sure how, off-hand, but I'd take an educated guess at it being possible; it'd surely add complexity, though. I agree, you would need to research procfs first, to ensure you get accurate enough results.
I use Linux Mint 18.3 with Cinnamon in a VirtualBox VM for testing & sandboxing.

I'm Terminalforlife (TFL/LL) on YouTube: https://www.youtube.com/c/terminalforlife
I'm also terminalforlife (TFL) on GitHub: https://github.com/terminalforlife
1000
Level 4
Level 4
Posts: 410
Joined: Wed Jul 29, 2020 2:14 am

Re: inotify

Post by 1000 »

With rene help, I have solved problems
1. I can log firewall events with a separate file.
2. I can use inotify on the file /var/log/iptables.log
viewtopic.php?p=2018117#p2018117

"inotify" and "sleep" have advantages and disadvantages.
The script in the first post with "sleep" will probably work on any system.
To use "inotify" I had to configure iptables firewall and add /etc/rsyslog.d/10-iptables.conf with inside

Code: Select all

:msg,contains,"[iptables] " /var/log/iptables.log 
For "inotify" script I just need to improve the logging of events. ( rules of iptables )
And obtaining data from the system.
- Because the data lifetime in the system is very short.
- Because I have problem with pstree command

Example pstree with PID of xed

Code: Select all

$ pstree -spa "3896"
systemd,1 splash
  └─lightdm,1266
      └─lightdm,1833 --session-child 13 20
          └─mate-session,1888
              └─caja,2044
                  └─firejail,3890 /usr/bin/xed /home/user/Desktop/linuxmint.blacklist
                      └─firejail,3892 /usr/bin/xed /home/user/Desktop/linuxmint.blacklist
                          └─xed,3896 /home/user/Desktop/linuxmint.blacklist
                              ├─{xed},3897
                              ├─{xed},3898
                              └─{xed},3899
But when the variable is empty there is no error.
I get the whole tree.

If the process PID now exists,
a fraction of a second later may not exist anymore.
Therefore, I can not use the condition. I just have to use the command and draw conclusions from it.
Post Reply