Diversion Diversion - the Router Ad-Blocker

[thelonelycoder]

Had a little install loop when fresh installing Diversion 4.0.6 today. I have modified my router's https port to listen on 443, but Diversion Standard would not install while that port was listening for httpd, even though Pixelserv would end up using a dedicated IP. I worked around it by temporarily stopping the httpd service, but I don't think the installer should block itself if 443 is active on the main router LAN IP.

Happy New Year!

Probably is listening on all interfaces on that port, that's why it triggers it. Will look into it when time allows.

I keep losing my diversion install on router ugprade or reboot. What could be causing this? I've noticed I lose my ssh terminal command history when this happens as well

Post the content of /jffs/scripts/post-mount, use sf to show the file.

 

[thelonelycoder]

I have been using diversion for a while and it is really helpful. However, sometimes when I add a domain to my whitelist, it does not apply after processing it. To fix that, I have to disable the blocking and re-enable it to actually make the change effective. Any ideas why this happens?

I thought I tried that already, but followed your advice, and tried again. It is still blocking it. I have tried to add other domains to the whitelist (el, 1, 1), then did b and then 4. They are still being blocked by secondary blocking file. I see the entries I added in the whitlelist file (el, 1).

Am I doing something wrong here? Thanks and Happy New Year!

I'm looking into that next.

 

[thelonelycoder]

Has there ever been talk of keeping a list of the commonly whitelisted sites and why?

Yes, and it died for privacy reasons.

 

[adampk17]

Yes, and it died for privacy reasons.

[emoji53] Hrmmm

 

[thelonelycoder]

@Eric22 & @Melvin McKracken & all
Looks like the whitelisting needs some serious overhaul and simplification. I'm thinking of going the Skynet/Pi-hole way.
When whitelisting, the exact domain gets added to the whitelist if not already found, while it is removed in the blacklist if found and vice versa.
No checks are done if the domain is found anywhere in the blocking file(s) or whitelist or blacklist, it just gets added.
The wildcard-blacklist is exempted from these auto removals, entries there remain once added.

The possibility to set entries to active/inactive will also be removed in the white, black and wildcard-blacklist.
The option to auto-add the www or non-www domain remains for the whitelist.
And, the list of near matches if no exact match is found will also be gone.

Any thoughts on that? Let me know. The code for these functions is a nightmare to work on and it needs simplification, especially with the changes that come in Diversion 4.1.

 

[thelonelycoder]

@Eric22 & @Melvin McKracken
Running a manual update of the blocking file(s) in b will make sure all whitelisted domains are removed. That is if you are running the latest Diversion 4.0.6.

 

[^Tripper^]

I personally find the near match check especially useful. My $0.02.

 

[Dabombber]

It's a trap.

  ✖  Your router's firmware (384.8) is too old to
     run Diversion Standard Edition,
     only the Lite Edition can be installed.

     Upgrade your routers firmware first before
     attempting to install Standard Edition.

 Continue? [1=Yes e=Exit] e

  i  Running function upgrade
  i  Getting latest installer file
  ✔  install.div         integrated
  i  Checking router
  ✔  Asuswrt-Merlin
  ✔  dos2unix

  ✖  Your router's firmware (384.8) is too old to
     run Diversion Standard Edition,
     only the Lite Edition can be installed.

     Upgrade your routers firmware first before
     attempting to install Standard Edition.

 Continue? [1=Yes e=Exit]

Looking through the code this is from line 107 of https://diversion.ch/diversion/4.0/file/install.div. https://diversion.ch/diversion/4.0/file/install.div

Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name   
tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      4034/nginx: master

Maybe a different warning than outdated firmware could be used here.

 

[Asad Ali]

I personally find the near match check especially useful. My $0.02.

Yep, if possible keep that one feature intact.

 

[thelonelycoder]

It's a trap.

  ✖  Your router's firmware (384.8) is too old to
     run Diversion Standard Edition,
     only the Lite Edition can be installed.

     Upgrade your routers firmware first before
     attempting to install Standard Edition.

 Continue? [1=Yes e=Exit] e

  i  Running function upgrade
  i  Getting latest installer file
  ✔  install.div         integrated
  i  Checking router
  ✔  Asuswrt-Merlin
  ✔  dos2unix

  ✖  Your router's firmware (384.8) is too old to
     run Diversion Standard Edition,
     only the Lite Edition can be installed.

     Upgrade your routers firmware first before
     attempting to install Standard Edition.

 Continue? [1=Yes e=Exit]

Looking through the code this is from line 107 of https://diversion.ch/diversion/4.0/file/install.div.

Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name  
tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      4034/nginx: master

Maybe a different warning than outdated firmware could be used here.

What's that nginx server do? Is that from the Asus download master?

 

[dave14305]

Probably is listening on all interfaces on that port, that's why it triggers it. Will look into it when time allows.

This is the netstat output from port_check (after pixelserv installation):

# netstat -lnt | grep [0-9]:${1:-443}
tcp        0      0 192.168.1.2:443         0.0.0.0:*               LISTEN     
tcp        0      0 127.0.0.1:443           0.0.0.0:*               LISTEN     
tcp        0      0 192.168.1.1:443         0.0.0.0:*               LISTEN

And nvram:

https_lanport=443

Got caught by the “Port 443 is in use by unknown application\n (AiCloud port is OK, check other applications).” error.

 

[Dabombber]

What's that nginx server do? Is that from the Asus download master?

It's just there to redirect the webui, manually installed so I guess it's a somewhat uncommon use case.

/opt/etc/nginx/nginx.conf

user nobody;
pid /opt/var/run/nginx.pid;

events {
        worker_connections 64;
        # multi_accept on;
}

http {
    include       mime.types;
    default_type  application/octet-stream;
    client_max_body_size 100m;
    
    ssl_certificate           /jffs/.cert/cert.pem;
    ssl_certificate_key       /jffs/.cert/key.pem;

    server {
        listen 80;
        server_name router.example.com router.lan router.asus.com 192.168.1.1;
        return 301 https://router.example.com$request_uri;
    }

    server {
        listen 443 ssl;
        server_name router.example.com;

        location / {
            proxy_buffering off;
            proxy_pass https://localhost:8443;
            proxy_set_header Host $host;
            proxy_redirect https://localhost:8443 https://router.example.com;
        }
    }
}

 

[thelonelycoder]

It's just there to redirect the webui, manually installed so I guess it's a somewhat uncommon use case.

/opt/etc/nginx/nginx.conf

user nobody;
pid /opt/var/run/nginx.pid;

events {
        worker_connections 64;
        # multi_accept on;
}

http {
    include       mime.types;
    default_type  application/octet-stream;
    client_max_body_size 100m;
   
    ssl_certificate           /jffs/.cert/cert.pem;
    ssl_certificate_key       /jffs/.cert/key.pem;

    server {
        listen 80;
        server_name router.example.com router.lan router.asus.com 192.168.1.1;
        return 301 https://router.example.com$request_uri;
    }

    server {
        listen 443 ssl;
        server_name router.example.com;

        location / {
            proxy_buffering off;
            proxy_pass https://localhost:8443;
            proxy_set_header Host $host;
            proxy_redirect https://localhost:8443 https://router.example.com;
        }
    }
}

Stop nginx for the install.

 

[Dabombber]

Stop nginx for the install.

Thanks, I'm fine with the lite version for now though . My concern was the 'e' option didn't exit the install.

 

[Melvin McKracken]

@Eric22 & @Melvin McKracken
Running a manual update of the blocking file(s) in b will make sure all whitelisted domains are removed. That is if you are running the latest Diversion 4.0.6.

You're a genius. I updated to 4.0.6 (didn't realize there was an update), and whitelisting for both works great! Thanks for the easy update option. I also read the changelog on the diversion site, and sure enough.

I'm fine with the way you can enable and disable certain domains added to the whitelist. And like the entire system overall, now that I got it working. Thanks again.

 

[faria]

Hi guys,
Does anyone knows how i can stop this entry " blocked by blockinglist wpad.RT-AC86U" showing in my logs?
I know its related to proxies but i got none of that setup in my network.
thanks.

 

[Protik]

Haven't played with the WebUI code for a while. So it's still open. What are your preferences for the web server type?

I am not an expert in this arena. But I remember lighttpd being pretty good. It's also available from Entware if I remember correctly.

Happy new year, by the way!

 

[adampk17]

Can I ask for more comments from folks about issues they have run in to with Diversion and family members? So far I've had one comment and he had to exclude his wife's devices.

Thank you in advance

 

[thelonelycoder]

@Dabombber , @dave14305 , @Asad Ali
I have uploaded a fix with better logic that will allow to continue install in case of:
- Port 443 is in use by other process than AiCloud
- Allow to install Diversion Standard even if port 80 is listening on 0.0.0.0 (the "too old firmware" message)

A warning is given, but you may continue the install after.

No version change.
Use 1233 to re-download all files or 12 for just the affected addon files.

 

[Xentrk]

Can I ask for more comments from folks about issues they have run in to with Diversion and family members? So far I've had one comment and he had to exclude his wife's devices.

Thank you in advance

Always best to start with the small block list rather than the large one. I recall iCloud being a site that was blocked in one of the host files. There was also a yahoo domain or two that mucked up access to yahoo mail. Dropbox was another one.

I suggest you enable stats reporting to see the top sites being blocked in addition to getting familiar with the follow the log file feature.

The ad blocker on pfSense now comes with a default whitelist that may be a good place to start. You can view the whitelist on the site
https://www.linuxincluded.com/block-ads-malvertising-on-pfsense-using-pfblockerng-dnsbl/