AB-Solution 2.0 is out! (v2.0.1)

[L&LD]

With the next AB-Solution version comes an email function to send yourself the stats. This is for the weekly stats (ws) as well as the current log stats (ls).

To make this work reliably, a gmail account is needed to send the mail to, or through it to another mail address.
This, of course, requires the gmail account credentials to be stored on the device.
ATM the file is at <your device>/adblocking/.config/email.cfg.

I was just thinking this may not be a good idea. If one rogue fellow unplugs the stick they may find the file.
Would /jffs/ be a better place to store it?
I know, the router itself is not much heavier but access to /jffs/ requires more effort than plugging a USB device into a PC with the capability to read ext* file system.
Your thoughts? Thanks.

Isn't there any other way to enable email other than gmail?

I'd rather I just saw the stats rather than whoever works at google.

Just my thoughts on this.

Thanks for continuing with this project!

 

[thelonelycoder]

Isn't there any other way to enable email other than gmail?

I'd rather I just saw the stats rather than whoever works at google.

Just my thoughts on this.

Thanks for continuing with this project!

Which means you are using AB-Solution? I'm pleased you took the plunge and hit the command line. Very pleased indeed!

I have spent way more time on that email function then I anticipated.
It's a job I'm really proud of. Because not only will you be able to send the stats as part of the body message but as attachment too.
That may sound trivial.
I'll quote from this thread:

You would need tools to convert the attachment into a MIME encoded file, and properly generate the mail file before sending it. Definitely not something trivial.

And @RMerlin was right about it. But I wanted the attachment option so I dug into it.

I'd rather rest the email function for the moment and concentrate on the other goodies that will come with the next version.
Not that I'm tired of the email code. But I really like what I'm working on right now. It's always very pleasing when a large new part of code runs for the first time completely without errors.

 

[redvers]

From dim & distant memory, is it uuencode you need?

 

[thelonelycoder]

From dim & distant memory, is it uuencode you need?

Maybe that would have helped, but is not built in. A rather stripped down makemime is built in. And that is what I used.
I sed-strip and replace some parts of the generated code to make it a usable mail format.
That was the tricky part.

 

[calzor suzay]

Is AB-Solution a stand alone product or do I need pixelserv-tls to run it or enhance it?

 

[redvers]

Maybe that would have helped, but is not built in. A rather stripped down makemime is built in. And that is what I used.
I sed-strip and replace some parts of the generated code to make it a usable mail format.
That was the tricky part.

Fun times!

I forget what is and isn't included with the router, and don't do that much scripting in my current role so no need to keep the brain cells that remember how to do all this stuff

 

[thelonelycoder]

Is AB-Solution a stand alone product or do I need pixelserv-tls to run it or enhance it?

No, you do not need pixelserv-tls for AB-Solution to run and perform it's job properly.
In the next version of AB-Solution, pixelserv-tls will be an optional add-on that will be fully configured by it when installing.
It still will be optional and not required if https ads don't bother you.

 

[thelonelycoder]

Fun times!

I forget what is and isn't included with the router, and don't do that much scripting in my current role so no need to keep the brain cells that remember how to do all this stuff

Well, I picked the proper MIME parts that I needed from one of my CentOS server mails. And moved them around until my server and google's were satisfied when sending mail from the router.
Fun times indeed!

 

[calzor suzay]

It still will be optional and not required if https ads don't bother you.

Ah so pixelserv-tls would only be needed for HTTPS adds.

Still waiting for my router to turn up but look forward to using your software, I never knew this sort of thing was possible

 

[thelonelycoder]

update-hosts.sh version 1.1 is released.
This time, the (cu) option can do it's job

As part of the new code in the next version of AB-Solution, I have done some work on the hosts update script.
Everything is backwards compatible for the 2.x versions. So, please, update away.

Changelog update-hosts.sh 1.1
Improvement: Completely rewritten hosts download procedure.
Better error tolerance when downloading files. Individual check of downloaded file.
Fallback to last successful download if one of the files fail the test.
More details are shown when files are downloading and when they are being processed.

 

[thelonelycoder]

Still waiting for my router to turn up but look forward to using your software, I never knew this sort of thing was possible

I knew it had to be, that's how and why this project was born.

 

[RMerlin]

From dim & distant memory, is it uuencode you need?

MIME is the proper, compliant way to do so with an Email. uuencode was born out of necessity, for Usenet binary groups.

 

[redvers]

MIME is the proper, compliant way to do so with an Email. uuencode was born out of necessity, for Usenet binary groups.

As I said, all from memory from something i did around 10-15 years ago, but am pretty sure it worked if you including the correct MIME string as part of the email body...

And while the binary groups used uuencode to their advantage, am sure it was developed long before people realised they could fileshare using it

 

[HardCat]

update-hosts.sh version 1.1 is released.
This time, the (cu) option can do it's job

As part of the new code in the next version of AB-Solution, I have done some work on the hosts update script.
Everything is backwards compatible for the 2.x versions. So, please, update away.

Changelog update-hosts.sh 1.1
Improvement: Completely rewritten hosts download procedure.
Better error tolerance when downloading files. Individual check of downloaded file.
Fallback to last successful download if one of the files fail the test.
More details are shown when files are downloading and when they are being processed.

TLC, i ran cu option to update and did receive the following message.

Update update-hosts.sh now? [1=Yes 2=No] 1
getting latest update-hosts.sh

download of update-hosts.sh successful

rm: can't remove 'version.info': No such file or directory

Is this anything to worry about?

TKS...

 

[Santiago C]

It's indeed an extremely easy fix.
I ran the stats file on 20MB of log files with the 192.168.2. IP and then sed'ed the four noisiest IP's to 10.0.0.3, 31, 1 and 11 and ran the stats again.

Not a single digit difference between the two.

For some weird reason I know what you said should make sense, but I cannot understand it, lol

My LAN assigns addresses in 10.0.0.1/255.255.255.0. Let's say 10.0.0.2 is my mobile and 10.0.0.22 my SmartTV, this is what I get:

Top 10 domains for top 10 clients:
--------------------------------------------------
10.0.0.2:
   1668 api-global.netflix.com
   1095 ichnaea.netflix.com
   1020 secure.netflix.com
   914 e6858.dscc.akamaiedge.net
   870 ssl.google-analytics.com
   853 www.google.com
   814 mqtt.c10r.facebook.com
   671 10.0.0.2
   645 api.twitter.com
   604 gs-loc.ls-apple.com.akadns.net
10.0.0.22:
   1668 api-global.netflix.com
   1095 ichnaea.netflix.com
   1020 secure.netflix.com
   569 lcprd1.samsungcloudsolution.net
   558 appboot.netflix.com
   470 nrdp.nccp.netflix.com
   338 upu.samsungelectronics.com
   327 www.samsungotn.net
   220 dpu.samsungelectronics.com
   208 osb.samsungqbe.com

I don't even have Netflix installed in my mobile, so it is impossible to have that huge number of request (which is also exactly the same as in my SmartTV). Another strange thing is seeing a lot of request for the IP of my mobile itself from my mobile...

#offtopic: On a side note, I hate having my SmartTV trying to phone home to Sammy's servers that frequently... and don't even get into the fact that I don't have Netflix at all...

 

[thelonelycoder]

TLC, i ran cu option to update and did receive the following message.

Update update-hosts.sh now? [1=Yes 2=No] 1
getting latest update-hosts.sh

download of update-hosts.sh successful

rm: can't remove 'version.info': No such file or directory

Is this anything to worry about?

TKS...

No, it's just the version.info file that gets downloaded to the /home/root/ dir, read in and deleted after the check is done.
If you run (cu) again, it will show that you are up to date with the versions.

 

[RMerlin]

And while the binary groups used uuencode to their advantage, am sure it was developed long before people realised they could fileshare using it

You're correct, it actually originated from UUCP Mail (hence its name).

 

[thelonelycoder]

For some weird reason I know what you said should make sense, but I cannot understand it, lol

My LAN assigns addresses in 10.0.0.1/255.255.255.0. Let's say 10.0.0.2 is my mobile and 10.0.0.22 my SmartTV, this is what I get:

Top 10 domains for top 10 clients:
--------------------------------------------------
10.0.0.2:
   1668 api-global.netflix.com
   1095 ichnaea.netflix.com
   1020 secure.netflix.com
   914 e6858.dscc.akamaiedge.net
   870 ssl.google-analytics.com
   853 www.google.com
   814 mqtt.c10r.facebook.com
   671 10.0.0.2
   645 api.twitter.com
   604 gs-loc.ls-apple.com.akadns.net
10.0.0.22:
   1668 api-global.netflix.com
   1095 ichnaea.netflix.com
   1020 secure.netflix.com
   569 lcprd1.samsungcloudsolution.net
   558 appboot.netflix.com
   470 nrdp.nccp.netflix.com
   338 upu.samsungelectronics.com
   327 www.samsungotn.net
   220 dpu.samsungelectronics.com
   208 osb.samsungqbe.com

I don't even have Netflix installed in my mobile, so it is impossible to have that huge number of request (which is also exactly the same as in my SmartTV). Another strange thing is seeing a lot of request for the IP of my mobile itself from my mobile...

#offtopic: On a side note, I hate having my SmartTV trying to phone home to Sammy's servers that frequently... and don't even get into the fact that I don't have Netflix at all...

Do you have manually assigned the IP's? Either the devices switch between the two or the Samsung is extremlely noisy with it's broadcasting. I'll have to fire up mine sometime to check it. The TV is more dust collector than entertainment center here. We all have other things to do.

 

[rotorbudd]

reposting this to the correct thread. I mistakenly posted it to the 1.07 thread
EDIT Something in the pasted text is triggering Cloudflare's blocking software.

thelonelycoder
I'm having trouble completing a fresh install.
Everything goes OK until it tries to download the hosts files and all files fail with the same error:

getting hosts file 1
http://winhelp2002.mvps.org/hosts.txt
wget: invalid option -- 'o'
BusyBox v1.24.2 () multi-call binary.

Usage: wget [-c|--continue] [-s|--spider] [-q|--quiet] [-O|--output-document FILE]
[--header 'header: value'] [-Y|--proxy on/off] [-P DIR]
[-U|--user-agent AGENT] URL...

 

[thelonelycoder]

reposting this to the correct thread. I mistakenly posted it to the 1.07 thread
EDIT Something in the pasted text is triggering Cloudflare's blocking software.

thelonelycoder
I'm having trouble completing a fresh install.
Everything goes OK until it tries to download the hosts files and all files fail with the same error:

getting hosts file 1
http://winhelp2002.mvps.org/hosts.txt
wget: invalid option -- 'o'
BusyBox v1.24.2 () multi-call binary.

Usage: wget [-c|--continue] [-s|--spider] [-q|--quiet] [-O|--output-document FILE]
[--header 'header: value'] [-Y|--proxy on/off] [-P DIR]
[-U|--user-agent AGENT] URL...

Quick question: You don't have IPv6 enabled, do you?