adblocker the other way with block tracker

[fball]

usually adblockers are installed within a browser but
- you have no control over them, no real control at least, some will accept payments for skipping certain ads to let them through
- the usually use limited lists
- websites can identify adblockers and refuse to take you to your site

There is a way around these problems. There is an adblocker which runs in the background and is not taceable by websites
You can add more lists, edit them before you do and noone will know or try to bribe you with money to let his ads through.
It can be found on this site, which is both in english and german

Pihole will work the same way but needs a fixed ip so it will be best in a network on a server. I use the blocktracker script on my notebook and I am really happy with it as it which is not possibel with pihole, at least to my knowledge

https://ajacobsen.github.io/block-tracker/

The installation is as simple as it seems on the page, but there are some points I would like to address :
You can lock yourself out from your websites if you add blocklists you have not checked before adding them
However, the script creates a backup of your orginal hosts file so you can go back to a sza where your system will work again

You must know that the lists you can add should me in hosts format (0.0.0.0 www.hideme.xyz) or the script may not work

It can be quite a task to manually modify downloaded files so they end up in hosts format

To make sure the script will run, add a cronjob " block-tracker --enable " to your system

Warning : Don't think it will 100% protect you from ads. there are so many servers you cannot block them all, I currently have 1.5 mio sites to block but I still see some ads, the same goes for the pihole in my network with 7.5 mio ads blocked

One last thing: Ads as said here is not just real ads but there are lists fpr trackers, phishing, malware, <violates forum rules> and many ther categories

As always it's a fight to reduce malware. I hope some of you will try the script and live happily ever after.

[beacon]

Hello fball. I have been trying with no luck to get pihole to work on LM20 and came across your post. It sounds great, and I'd like to use it. I have installed as per directions, but I don't know how to make use of it. How do I get it to appear so that I can add to the blocked sites? I'm sorry if this sounds naive, but I am stymied.
Regards,

[Mick-Cork]

Here's a similar solution which was mentioned by Kadaitcha Man (IIRC?) on the forum recently : https://gist.github.com/Eliastik/f4a357 ... 80491aa074

The reason I mention it is that I only installed it yesterday and it also seems to work quite well. It requires root privileges to run so I've put the script in /usr/bin for the moment and run it from there as a test.

It takes a back-up (copy) of the current hosts file and saves it as /etc/hosts.bak. It then creates a new /etc/hosts file sourcing the list of blocked sites from someonewhocares.org, yoyo.org and winhelp2002.mvps.org. These can be altered by changing the appropriate line in the script.

My next step is to try and either (or both) get it to run from the menu and/or a desktop launcher so it's easier to initiate, or possibly as a cron job at boot up so that the hosts file is automatically updated without having to think about it. I'm anticipating asking the forum for some pointers on these approaches as might have to jump some additional hoops that I'm not familiar with - but that's for another thread.

Anyway, just thought I'd add as an alternative option but with the same objective.

[beacon]

Good morning Mick-Cork. I really appreciate your suggestion, as I have had no luck with pihole or other methods. However, I think installation is probably beyond my technical abilities, much as I would like to try. I'll see if I can get help in due course, despite being locked-in at the moment.

[Mick-Cork]

Good morning Mick-Cork. I really appreciate your suggestion, as I have had no luck with pihole or other methods. However, I think installation is probably beyond my technical abilities, much as I would like to try. I'll see if I can get help in due course, despite being locked-in at the moment.

Hi Beacon,

I probably used the wrong word when I said ‘install’ and most likely made it sound more complicated than it is. It’s more a case of downloading, unzipping, copying/pasting and a bit of ‘twiddling’ along the way. Terminal not required

Anyway, thought I’d write a step-by-step on what I did here, and if you give it a go you’ve got something to refer to. I’m running LM Cinnamon 19.3 - hopefully same approach works in LM20? I’m also on a learning curve so if anyone wants to chip in with other/better ways to do this please do so.

The steps…

Download and copy (to /usr/local/bin folder):

1. Go to: https://gist.github.com/Eliastik/f4a357 ... 80491aa074

2. Click on Download ZIP (upper right) and save the file. It will have a long name ending in .zip and should go to your Downloads folder.

3. Navigate to your Downloads folder. Double click or right click the zip file and select ‘Open with Archive Manager’.

4. Click on ‘Extract’ (top bar of Archive Manager window), then click on ‘Extract’ (lower right) to confirm. This will create a new folder in Downloads with the same name as the zip file.

5. Open that folder and you’ll find update-hosts.sh

6. Right click on update-hosts.sh and select copy or cut (whichever is your preference).

7. Navigate to (File System) /usr/local/ and right click on ‘bin’. Select 'Open as root'.

8. Once in /usr/local/bin, right click and paste update-hosts.sh into this folder.


Run the script:

A. Via File Manager:

1. Navigate to /usr/local, right click on the ‘bin’ folder and select ‘Open as Root’. (Enter your password).

2. Double click on ‘update-hosts.sh’ and select ‘Run in Terminal’.

The script should now run.


B. Using a Desktop Launcher (My preference)

1. Right click on Desktop and select ‘Create a new launcher here…’
(Launcher properties window will open).

2. Enter any name you like for the launcher, e,g. Update Hosts File
- In the command box enter: sudo /usr/local/bin/update-hosts.sh
- Comment box is optional
- Put a check mark in ‘Launch in terminal’
- If you want to change the Icon, click on it and select one you prefer. (You can do this afterwards if preferred).
- Click OK

It will then ask if you would like to add this launcher to the menu. I tried ‘Yes’ put couldn’t get it to run from the menu so I stuck with ‘No’ for the moment.

You should now see the launcher on your desktop.

Double click the desktop launcher. The terminal should open and ask for your password.

Enter your password and the script should run.


Some notes:

1. Entering your password. There’s ways of bypassing this but I need to do some more reading. I think it requires changes to the sudoers setup but haven’t explored that option yet.

2. Adding the script as a menu option. As above this didn’t work when I tried adding it at the same time as creating the launcher. Not sure why yet, possibly a permissions issue or the command format?

3. Putting the script in /usr/local/bin. There seems to be various opinions about where to put user personal scripts (as opposed to system scripts). /usr/local/bin was a common suggestion after googling this. Would be good to hear from anyone on the LM forum who can confirm this or otherwise.

4. The script itself (runs as root and assumes curl already installed) :

- Downloads the latest blocking lists from the three URL resources: someonewhocares.org, pgl.yoyo.org and winhelp2002.mvps.org. (Additional URLs can be added to the script if req’d).

- Does a check for incorrect or malicious entries in the lists, and exits if any found. The check is quite CPU intensive (30%-40% across all 4 cores and runs for approx 2 minutes on an i5 with 8Mb RAM). The script can be modified if desired to exclude the integrity check by changing the variable value on line 33 to: CHECK_HOSTS=0.

- Creates a backup of the existing hosts file (/etc/hosts) and calls it hosts.bak in the same folder.

- Creates a new hosts file based on the content of the downloaded lists.

- Checks if a file called hosts.initial exists and appends it to the top of the new hosts file. (I’m presuming this allows a user to create their own list (in hosts.initial) which is then always added to the hosts file after the script is run.

- The messages in the terminal after the integrity check finishes come quick and fast, and are easily missed as the terminal closes once the script is complete. I added an extra line immediately before the ‘exit’ at the end of the script:
sleep 10
which holds the terminal open for ten seconds before the exit is executed.

- There will be duplicate entries in the hosts file as there is some commonality across the three sources. However I haven’t noticed any negative impact on browsing performance.

If you give it a go let us know how you get on, and as above, if anyone wants to add anything please chip in.

[beacon]

Hello again Mick-Cork. You are a scholar and a gentleman. It was very kind of you to go to so much trouble, and it is very much appreciated. I intend to follow your instructions as soon as I can and hope for success, which I will report in due course.
Kind regards.

[beacon]

Hello again. I have followed your instructions, which I must say are amongst the clearest I have ever read. The only problem arose at the very end: 'enter your password and the script should run'. I entered the password, the window disappeared, and nothing else happened. I have tried several times and checked that curl is installed. I wonder if you have any idea of what to try next.

Thanks and regards
Beacon

[Mick-Cork]

Hi Beacon,

First quick thing to check:

Can you navigate to (File System) /usr/local/bin and then right click on update-hosts.sh and select properties.
Then click on the 'Permissions' tab.

It should show the owner as 'root' with Read and write access.
Further down it should also have a check mark against 'Allow executing file as program'.

If either of these are different, navigate back to the local folder, right click on 'bin' and select 'Open as root' (will ask for your password).

Once back in the bin folder (now with elevated privileges), right click on update-hosts.sh again and go to properties/permissions.

Change settings as required to match the above, i.e: root with Read and write, and check to allow executing file as program.

Close the properties window, and whilst still in the bin folder with elevated privileges double click the file and select 'Run in terminal'

See if that works?

If something else is causing it not to work we'll see where we go from here.
Will be away from Laptop for a while now, so might be tomorrow before I check back in.

[beacon]

Thank you again Mick-Cork. I have done as suggested and seem to have got further, but I'm not certain I have been altogether successful. After 'installation', does the adblocker work behind the scenes, or must it be started manually? I got the following message when the integrity test had finished and am puzzled by the third line from the bottom. (I can't seem to produce a screenshot that doesn't include personal information, so I am typing the relevant parts.)
Creating temporary directory
Created tmp.directory at tmp/tmp.bpk 204Pdk5
Downloading hosts from [the sites you mentioned]
Backup old hosts file 'etc/hosts' 'etc/hosts/bak'
checking host data
No incorrect or malicious date found
installing host list
The initial hosts file doesn't exist: /etc/hosts.initial
Removing cache
Done
Now when I use the launcher I am asked for a password, and the whole procedure repeats itself.

[Mick-Cork]

Let's start with the good news - it's working, and as expected

Now the detail:

a. Yes the 'blocker' does work behind the scenes. The Hosts file exists on Linux, Windows and IOS operating systems and blocks unwanted connections. My understanding is that the contents are loaded into cache when the system is booted - hence the tiny (if any) impact on performance. It doesn't matter what browser or app you are using, the system will be using the content of the hosts file regardless to block those connections. It will work on anything, eg, a dodgy link in an email, browser connections, any apps that try to make a background connection to any of the listed sites.

I'm not sure how often the three URL resources update their respective lists (maybe monthly?), but basically you just run the script periodically and keep your hosts file up to date. There's a bit more info here: https://winhelp2002.mvps.org/hosts.htm. It's a bit wordy, but should give you the gist. Here's the actual file at someonewhocares, and it gives a short description at the top as to what it does: https://someonewhocares.org/hosts/

b. The hosts.initial file.
The three URL resources will be combined to create a hosts file with approx 26000 lines. These are all the sites they have identified as beneficial to block. However, you might also find sites of your own that you'd like to block, and the script uses the hosts.initial file to add these to the hosts file. This file does not initially exist, it will only be present if you choose to create it and add sites of your own, hence the output message that you're seeing. It's a good method in that your list (if you create one) will always be preserved after running the script.

I was thinking of making the script a cron job that would run every time I boot up, therefore removing the need to remember to run it periodically. Have decided against this though as the integrity check in the script is quite CPU intensive on my machine for a couple of minutes - so decided to just run it manually now and again.

I hope that mostly covers it, but to summarise all the script does is copy the lists from the three URLs and combines them into one hosts file on your system. It is then always used 'behind the scenes'. The hosts file only changes when you run the script and the source files have been updated since the last time you ran it. Either run it say, once a month, or optionally have a look at the three sites and see if they've been updated since you last looked. Personally I think it's easier to just run it maybe once a month?

I might also look at the option of a cron job such that it runs once automatically on the 1st day of each month. Not sure if that's possible, but if I find a way I'll post it back here.

[beacon]

Hello Mick-Cork,
Another 'thank you' seems quite inadequate to express my appreciation for your help in this thread. The advice has been very helpful and detailed enough for someone who is not a technological genius. I have learned a lot as well and expect to know even more after following the links you provided. In the interval I am enjoying a largely ads-free experience.
Kind regards,
Beacon

[Mick-Cork]

My pleasure Beacon, happy it's working for you.

There's quite a few web resources that talk about using the hosts file to block unwanted connections if you want to read more.
Here's a selection (Google search) : https://bit.ly/34RZBsr

[Kseanfitz_1]

Just wanted to jump in here to say that the step-by-step instructions provided by "Mick-Cork" seemed to work well. The script ran and sourced the entries for the hosts file, backed-up the 9 line original file to hosts.bak and generated a 29,193 line hosts file. The "Create new launcher here" entry to the menu also worked and do not forget to click on the ugly oil well looking generic icon and replace with something/anything else, and using the menu-editor I cut it from "other" and pasted it to the Internet list.
I have yet to run a browser with all my ad blockers disabled to see how it works and see if i get anti-ad blocker whining spams from websites or not.
One major benefit of blocking at a root level before it even gets to a web browser is a savings on the internet bandwidth that a person pays for just to get wasted as mush as 80% on spam in some use situations.

Edit: Just a little feedback for anyone that may be interested...
Just visited sites that are not worth visiting without ad blocker add-ons installed, with all add-ons disabled, and they loaded up fast without dragging on and on trying to contact spam, doubleclick, analytics etc etc etc and no anti-ad blocker whining popups. YouTube will need something to block maybe multi purpose YouTube Helper will be all that is needed there.
Looks like I can disable about 5 different things that deal with the spam you to death problem.

[Mick-Cork]

Just wanted to jump in here to say that the step-by-step instructions provided by "Mick-Cork" seemed to work well. The script ran and sourced the entries for the hosts file, backed-up the 9 line original file to hosts.bak and generated a 29,193 line hosts file. The "Create new launcher here" entry to the menu also worked and do not forget to click on the ugly oil well looking generic icon and replace with something/anything else, and using the menu-editor I cut it from "other" and pasted it to the Internet list.
I have yet to run a browser with all my ad blockers disabled to see how it works and see if i get anti-ad blocker whining spams from websites or not.
One major benefit of blocking at a root level before it even gets to a web browser is a savings on the internet bandwidth that a person pays for just to get wasted as mush as 80% on spam in some use situations.

Edit: Just a little feedback for anyone that may be interested...
Just visited sites that are not worth visiting without ad blocker add-ons installed, with all add-ons disabled, and they loaded up fast without dragging on and on trying to contact spam, doubleclick, analytics etc etc etc and no anti-ad blocker whining popups. YouTube will need something to block maybe multi purpose YouTube Helper will be all that is needed there.
Looks like I can disable about 5 different things that deal with the spam you to death problem.

Hi Kseanfitz,

Good to hear it's working for you. I've since been getting to grips with bash scripts, and my first one is a little routine for adding and removing domains on the fly. It uses Zenity to display a GUI with two input fields: Block a domain and/or Allow a domain.

Any domain entered in the Block field gets written to the hosts and hosts.initial files.
Any domain entered in the Allow field gets deleted from the hosts and hosts.initial files.

I called the script file manage-hosts.sh, saved it in the same /usr/local/bin folder, made it executable, and created a desktop launcher with the name 'Manage Hosts Files' and the command: sudo /usr/local/bin/manage-hosts.sh

I haven't added any input validation to check the domain format, so it's a case of making sure you cut and paste or type the domain correctly. Might get round to that in the next version.

If you want to give it a try here's the script:

Code: Select all

#!/bin/bash

HOSTS_PATH="/etc/hosts"
HOSTS_INITIAL_PATH="/etc/hosts.initial"

DOMAINS=$(zenity --forms --title="Manage Hosts Files" --width=400 \
	--text="Enter Domain Information" \
	--separator="," \
	--add-entry="Block a domain" \
	--add-entry="Allow a domain" )
	
	[[ "$?" != "0" ]] && exit 1
	
	block=$(awk -F, '{print $1}' <<<$DOMAINS)
	allow=$(awk -F, '{print $2}' <<<$DOMAINS)

if [ $block != '' ] || [ $allow != '' ]; then
# Check if domain block entered
	if [ $block != "" ]; then
	
		if ! grep -q "0.0.0.0 $block" $HOSTS_PATH; then
			# domain not found in hosts file, so add it.
			sed  -i '1i 0.0.0.0 '"$block" $HOSTS_PATH
		fi
		
		if ! grep -q "0.0.0.0 $block" $HOSTS_INITIAL_PATH; then
			# domain not found in hosts.initial, so add it.
			sed  -i '1i 0.0.0.0 '"$block" $HOSTS_INITIAL_PATH
		fi
		
		bm="$block - blocked"
		
	fi

# Check if domain allow entered
	if [ $allow != "" ]; then
	
		if grep -q "0.0.0.0 $allow" $HOSTS_PATH; then
			# domain found in hosts file, so delete it.
			grep -v "0.0.0.0 $allow" $HOSTS_PATH > tmpfile && mv tmpfile $HOSTS_PATH
		fi
		
		if grep -q "0.0.0.0 $allow" $HOSTS_INITIAL_PATH; then
			# domain found in hosts.initial, so delete it.
			grep -v "0.0.0.0 $allow" $HOSTS_INITIAL_PATH > tmpfile && mv tmpfile $HOSTS_INITIAL_PATH
		fi
		
		am="$allow - allowed"
		
	fi
	
	#Display "Domain(s) entered"
	zenity --title="Manage Hosts Files" --info --text "$bm\n$am" --width=400

else
	#Display "No domain entered"
	zenity --title="Manage Hosts Files" --info --text "No domain entered" --width=200
	
fi
exit 0

I might extend the original script to cater for a third file with a list of allowed domains, so that these can be preserved after an update from the three URL sources. Basically it would be a case of comparing the domains in that file with the update, and deleting them from the hosts file after it had been merged.

If I do I'll add to this thread with the details.

[beacon]

Hello Mick-Cork. This is a message out-of the blue. As I hope you realise, your help on this issue was one of the best things I have got from this forum, and I hope you won't mind another related query on the matter. For some reason, after a re-install I have had to follow your instructions once again. They remain clear and helpful, and all seems to go well until I tick at the final stage on 'update hosts-sh' and select 'run in terminal'. Nothing happens' the script doesn't run. I wonder if something has changed in the years when you provided the information. Or perhaps because of my update to LM 21.3?
Many thanks,