Page 1 of 1


Posted: Wed May 12, 2021 10:52 am
by 1000
We have very nice tools for check internet connections.
For example

Code: Select all

lsof -i :0-65535
And we can run this with loop for get data in in continuous mode

Code: Select all

lsof -i :0-65535 -r2
A bit too much data?
Then we can clean it up a little.

Code: Select all


while true ; do
	ALL_CON=$(lsof -i :0-65535 | sed '1d')
	# We want only NAME + PID of app to compare 
	ONLY_NEW_CONN=$(awk '{print $1, $2}' <<< "$ALL_CON" | sort | uniq -c | sort)

	# We want compare old output with new output of lsof command 

	COMPARE=$(comm -13 <(echo "$OLD_1") <(echo "$NEW_1"))

	if [ ! -z "$COMPARE" ] ; then
		echo "$COMPARE"
	# The lsof command refreshes every 2 seconds 
	sleep 2
And we have something like that
# bash netstat1
1 NetworkMa 1111
3 netdata 1386
8 firefox 2220
7 firefox 2220
The script only shows a newline if these conditions are met
- if the number of connections changes
- if the application name changes

The script is not perfect as the system may cheat me.
the application can close one connection and open a new one at the same time.
So the script may not notice the difference.
So I should add more variables to compare. For example, a connection name.

For now, it does not matter.
I am wondering about something other.

- Using a command in a loop generates a load for the computer.
- Command lsof use files from /proc/

" The inotify API provides a mechanism for monitoring file system events. inotify can be used to monitor individual files, or to monitor directories.
When a directory is monitored, inotify will return events for the directory itself, and for files inside the directory.
... Inotify does report some but not all events in sysfs and procfs. "

It just doesn't watch the file, but reports when you try to do something,
so it doesn't overload the computer too much.

I'd like to try build a similar script, but without "lsof" command.
And compare the performance which is better.

/proc/ , /proc/*/net this is not my strong side. It will take me some time to research /proc folder.
What do you think? Is it possible?
Because I have doubts.
If the file does not exist, is it possible use inotify?

Re: inotify

Posted: Thu May 13, 2021 3:33 pm
by rene
1000 wrote:
Wed May 12, 2021 10:52 am
If the file does not exist, is it possible use inotify?
Not on the file itself, but you can watch the directory it pops up in. From a script, after sudo apt-get install inotify-tools, e.g.

Code: Select all

rene@hp8k:~$  inotifywait -m ~/tmp
Setting up watches.
Watches established.
/home/rene/tmp/ CREATE foo
/home/rene/tmp/ OPEN foo
/home/rene/tmp/ ATTRIB foo
/home/rene/tmp/ CLOSE_WRITE,CLOSE foo
/home/rene/tmp/ MODIFY foo
/home/rene/tmp/ OPEN foo
/home/rene/tmp/ MODIFY foo
/home/rene/tmp/ CLOSE_WRITE,CLOSE foo
/home/rene/tmp/ DELETE foo
as a result of in another terminal touch ~/tmp/foo; echo bar >~/tmp/foo; rm ~/tmp/foo. So you just watch for the file appearing...

That said, your own version I would quickly find good enough --- and there's probably tons of race-conditions with inotifywait to deal with if you use that.

Re: inotify

Posted: Sun May 16, 2021 4:29 am
by 1000
For now I will abandon the topic.

1. I'm not sure, but I guess I should be monitoring the condition
and maybe something.

I wrote a script to translate hexadecimal to decimal local_address and rem_address from there.
The number of connections and the IP and ports numbers looks identical with lsof -Pi command.

Path /proc/*/fd/ is too variable / changeable.

2. You're right, I can't monitor files inside /proc/ It is virtual filesystem or something. ... /proc.html

It's a pity because I noticed that I can monitor several files simultaneously in one script

Code: Select all

inotifywait -q --event modify --format '%w' /path/file_1  /path/file_2
3. Workaround.
I can monitor network with iptables firewall, and I can log output ( without ping to the router ) to a separate file.
Then I can use inotify to monitor file / log of iptables.

- For a server where the number of connections is small, this may be a good solution.
For a home user, where one website can create 30 Internet connections, it will be a very heavy load for the hard drive.
- I don't know how to save the log to a separate file.
Therefore, I will not test it for now.

Thank you very much for the suggestions.

Re: inotify

Posted: Sun May 16, 2021 2:22 pm
by rene
1000 wrote:
Sun May 16, 2021 4:29 am
2. You're right, I can't monitor files inside /proc/ It is virtual filesystem or something.
Certainly it's a virtual filesystem, i.e, with files/directories not backed by regular storage but "made up" on the spot by the kernel when in fact read/written by you, but that in and of itself is not a reason you can't monitor files there. It's just that a file that does not exist you can not monitor on whichever filesysten; only a directory in which it would pop up.

But yes, otherwise no comments on your conclusion.

Re: inotify

Posted: Mon Jun 07, 2021 10:00 pm
by Termy
I'm not exactly sure how, off-hand, but I'd take an educated guess at it being possible; it'd surely add complexity, though. I agree, you would need to research procfs first, to ensure you get accurate enough results.

Re: inotify

Posted: Wed Jun 09, 2021 6:20 am
by 1000
With rene help, I have solved problems
1. I can log firewall events with a separate file.
2. I can use inotify on the file /var/log/iptables.log

"inotify" and "sleep" have advantages and disadvantages.
The script in the first post with "sleep" will probably work on any system.
To use "inotify" I had to configure iptables firewall and add /etc/rsyslog.d/10-iptables.conf with inside

Code: Select all

:msg,contains,"[iptables] " /var/log/iptables.log 
For "inotify" script I just need to improve the logging of events. ( rules of iptables )
And obtaining data from the system.
- Because the data lifetime in the system is very short.
- Because I have problem with pstree command

Example pstree with PID of xed

Code: Select all

$ pstree -spa "3896"
systemd,1 splash
      └─lightdm,1833 --session-child 13 20
                  └─firejail,3890 /usr/bin/xed /home/user/Desktop/linuxmint.blacklist
                      └─firejail,3892 /usr/bin/xed /home/user/Desktop/linuxmint.blacklist
                          └─xed,3896 /home/user/Desktop/linuxmint.blacklist
But when the variable is empty there is no error.
I get the whole tree.

If the process PID now exists,
a fraction of a second later may not exist anymore.
Therefore, I can not use the condition. I just have to use the command and draw conclusions from it.