inotify

About writing shell scripts and making the most of your shell
Forum rules
Before you post please read how to get help
Post Reply
1000
Level 4
Level 4
Posts: 426
Joined: Wed Jul 29, 2020 2:14 am

inotify

Post by 1000 »

We have very nice tools for check internet connections.
For example

Code: Select all

lsof -i :0-65535
And we can run this with loop for get data in in continuous mode

Code: Select all

lsof -i :0-65535 -r2
A bit too much data?
Then we can clean it up a little.

Code: Select all

#!/bin/bash


echo "{NUMBER OF CONNECTIONS} , NAME , PID:"
while true ; do
	ALL_CON=$(lsof -i :0-65535 | sed '1d')
	# We want only NAME + PID of app to compare 
	ONLY_NEW_CONN=$(awk '{print $1, $2}' <<< "$ALL_CON" | sort | uniq -c | sort)

	# We want compare old output with new output of lsof command 
	OLD_1="$NEW_1"
	NEW_1="$ONLY_NEW_CONN"

	COMPARE=$(comm -13 <(echo "$OLD_1") <(echo "$NEW_1"))

	if [ ! -z "$COMPARE" ] ; then
		echo "$COMPARE"
	fi
	
	# The lsof command refreshes every 2 seconds 
	sleep 2
done
And we have something like that
# bash netstat1
{NUMBER OF CONNECTIONS} , NAME , PID:
1 NetworkMa 1111
3 netdata 1386
8 firefox 2220
7 firefox 2220
The script only shows a newline if these conditions are met
- if the number of connections changes
- if the application name changes

The script is not perfect as the system may cheat me.
the application can close one connection and open a new one at the same time.
So the script may not notice the difference.
So I should add more variables to compare. For example, a connection name.

For now, it does not matter.
I am wondering about something other.

- Using a command in a loop generates a load for the computer.
- Command lsof use files from /proc/

" The inotify API provides a mechanism for monitoring file system events. inotify can be used to monitor individual files, or to monitor directories.
When a directory is monitored, inotify will return events for the directory itself, and for files inside the directory.
... Inotify does report some but not all events in sysfs and procfs. "


It just doesn't watch the file, but reports when you try to do something,
so it doesn't overload the computer too much.

I'd like to try build a similar script, but without "lsof" command.
And compare the performance which is better.

/proc/ , /proc/*/net this is not my strong side. It will take me some time to research /proc folder.
What do you think? Is it possible?
Because I have doubts.
If the file does not exist, is it possible use inotify?
rene
Level 17
Level 17
Posts: 7766
Joined: Sun Mar 27, 2016 6:58 pm

Re: inotify

Post by rene »

1000 wrote:
Wed May 12, 2021 10:52 am
If the file does not exist, is it possible use inotify?
Not on the file itself, but you can watch the directory it pops up in. From a script, after sudo apt-get install inotify-tools, e.g.

Code: Select all

rene@hp8k:~$  inotifywait -m ~/tmp
Setting up watches.
Watches established.
/home/rene/tmp/ CREATE foo
/home/rene/tmp/ OPEN foo
/home/rene/tmp/ ATTRIB foo
/home/rene/tmp/ CLOSE_WRITE,CLOSE foo
/home/rene/tmp/ MODIFY foo
/home/rene/tmp/ OPEN foo
/home/rene/tmp/ MODIFY foo
/home/rene/tmp/ CLOSE_WRITE,CLOSE foo
/home/rene/tmp/ DELETE foo
as a result of in another terminal touch ~/tmp/foo; echo bar >~/tmp/foo; rm ~/tmp/foo. So you just watch for the file appearing...

That said, your own version I would quickly find good enough --- and there's probably tons of race-conditions with inotifywait to deal with if you use that.
1000
Level 4
Level 4
Posts: 426
Joined: Wed Jul 29, 2020 2:14 am

Re: inotify

Post by 1000 »

For now I will abandon the topic.

Conclusions:
1. I'm not sure, but I guess I should be monitoring the condition
/proc/net/tcp
/proc/net/udp
and maybe something.

I wrote a script to translate hexadecimal to decimal local_address and rem_address from there.
The number of connections and the IP and ports numbers looks identical with lsof -Pi command.

Path /proc/*/fd/ is too variable / changeable.

2. You're right, I can't monitor files inside /proc/ It is virtual filesystem or something.
https://tldp.org/LDP/Linux-Filesystem-H ... /proc.html

It's a pity because I noticed that I can monitor several files simultaneously in one script

Code: Select all

inotifywait -q --event modify --format '%w' /path/file_1  /path/file_2
3. Workaround.
I can monitor network with iptables firewall, and I can log output ( without ping to the router ) to a separate file.
Then I can use inotify to monitor file / log of iptables.

But
- For a server where the number of connections is small, this may be a good solution.
For a home user, where one website can create 30 Internet connections, it will be a very heavy load for the hard drive.
- I don't know how to save the log to a separate file.
Therefore, I will not test it for now.

Thank you very much for the suggestions.
rene
Level 17
Level 17
Posts: 7766
Joined: Sun Mar 27, 2016 6:58 pm

Re: inotify

Post by rene »

1000 wrote:
Sun May 16, 2021 4:29 am
2. You're right, I can't monitor files inside /proc/ It is virtual filesystem or something.
Certainly it's a virtual filesystem, i.e, with files/directories not backed by regular storage but "made up" on the spot by the kernel when in fact read/written by you, but that in and of itself is not a reason you can't monitor files there. It's just that a file that does not exist you can not monitor on whichever filesysten; only a directory in which it would pop up.

But yes, otherwise no comments on your conclusion.
User avatar
Termy
Level 7
Level 7
Posts: 1792
Joined: Mon Sep 04, 2017 8:49 pm
Location: UK
Contact:

Re: inotify

Post by Termy »

I'm not exactly sure how, off-hand, but I'd take an educated guess at it being possible; it'd surely add complexity, though. I agree, you would need to research procfs first, to ensure you get accurate enough results.
Confused? Try this guide to the Linux Mint Support Forums!

I'm Terminalforlife (LL) on YouTube and terminalforlife on GitHub.
1000
Level 4
Level 4
Posts: 426
Joined: Wed Jul 29, 2020 2:14 am

Re: inotify

Post by 1000 »

With rene help, I have solved problems
1. I can log firewall events with a separate file.
2. I can use inotify on the file /var/log/iptables.log
viewtopic.php?p=2018117#p2018117

"inotify" and "sleep" have advantages and disadvantages.
The script in the first post with "sleep" will probably work on any system.
To use "inotify" I had to configure iptables firewall and add /etc/rsyslog.d/10-iptables.conf with inside

Code: Select all

:msg,contains,"[iptables] " /var/log/iptables.log 
For "inotify" script I just need to improve the logging of events. ( rules of iptables )
And obtaining data from the system.
- Because the data lifetime in the system is very short.
- Because I have problem with pstree command

Example pstree with PID of xed

Code: Select all

$ pstree -spa "3896"
systemd,1 splash
  └─lightdm,1266
      └─lightdm,1833 --session-child 13 20
          └─mate-session,1888
              └─caja,2044
                  └─firejail,3890 /usr/bin/xed /home/user/Desktop/linuxmint.blacklist
                      └─firejail,3892 /usr/bin/xed /home/user/Desktop/linuxmint.blacklist
                          └─xed,3896 /home/user/Desktop/linuxmint.blacklist
                              ├─{xed},3897
                              ├─{xed},3898
                              └─{xed},3899
But when the variable is empty there is no error.
I get the whole tree.

If the process PID now exists,
a fraction of a second later may not exist anymore.
Therefore, I can not use the condition. I just have to use the command and draw conclusions from it.
Post Reply