pixelserv pixelserv - A Better One-pixel Webserver for Adblock

[swetoast]

@mstombs might be good to review the code then so far i havent noticed any bugs in pixelserv tls so keep up the good work on pixelserv cause i need it for my minimalist adblockr called ublockr

ls /opt/var/cache/pixelserv/ | wc -l
107

 

[mstombs]

@mstombs might be good to review the code then so far i havent noticed any bugs in pixelserv tls so keep up the good work on pixelserv cause i need it for my minimalist adblockr called ublockr

ls /opt/var/cache/pixelserv/ | wc -l
107

The ad handling forked process does correctly ignore the USR1 with this line

      signal(SIGUSR1, SIG_DFL); // default is ignore?

https://github.com/kvic-z/pixelserv-tls/blob/master/pixelserv.c#L586

But that is my comment, so I never got round to testing it fully. The cert generation thread is started much earlier, and might use pthreads, I can't see any signal handling, so I might be able to fix the fork version by adding this line in here
https://github.com/kvic-z/pixelserv-tls/blob/master/pixelserv.c#L267
if I can remember how to compile such things again....

 

[kvic]

@mstombs

I cannot reproduce the issue on my build using pthread with "killall -USR1 pixelserv-tls".

That was a bit puzzling... Then I downloaded the Entware build and I think I know what issue you encountered.

The problem is partially because the Entware build is not compiled with using pthread. So the main and cert generation are two separate processes.

After forking, the cert generation process shall indeed ignore signals. This commit will fix it: https://github.com/kvic-z/pixelserv...083#diff-aeb9b74d64ceb5d0ab76b9aff00727c7R271

To use pthread, need to add -DUSE_PTHREAD to CFLAGS in the Makefile. I didn't add it by default because Asuswrt/Merlin comes with the silly linuxthread. Good old processes are better than that.

I'll switch default to pthread. No one is compiling for Asuswrt/Merlin anyway I think..

For now, the above commit shall resolve the issue in a new build.

 

[mstombs]

Great, thanks. For my education - I know nothing about pthreads - why do we still use a fork for the reply handler?

https://github.com/kvic-z/pixelserv-tls/blob/master/pixelserv.c#L584

 

[kvic]

Great, thanks. For my education - I know nothing about pthreads - why do we still use a fork for the reply handler?

https://github.com/kvic-z/pixelserv-tls/blob/master/pixelserv.c#L584

Mainly because the codes were there and are working as expected. I could have changed that handler into pthread but not much to gain from such an overhaul...

Also if there is any memory leak in the handler, the leak will die with a process but not a thread.

 

[bayern1975]

i upgrade yesterday merlin firmware from 380.59 alpha 1 to 380.59 beta1.....then i do factory reset and lost https certificate to my router.....is there an option to save certificate and use it for future upgrade of firmware?

 

[swetoast]

did you store your cert is /jffs ?

 

[bayern1975]

did you store your cert is /jffs ?

i do just backup jffs....and there are stored just openvpn keys and certs....

sent from Kodi 17 Krypton

 

[swetoast]

https://github.com/RMerl/asuswrt-merlin/wiki/JFFS

should use a usb to store it on

 

[bayern1975]

AiCloud use port 443 for default....but i would like to use that port for https....any solution for this?

sent from Kodi 17 Krypton

 

[bayern1975]

errors for installing? just me or?

ASUSWRT-Merlin RT-AC68U 380.59-beta1-g3b6a743 Sun Apr 24 17:20:10 UTC 2016
admin@RT-AC68U:/tmp/home/root# opkg install pixelserv-tls
Installing pixelserv-tls (V35.HZ12.Kh-20160313-1) to root...
Downloading http://pkg.entware.net/binaries/armv7/pixelserv-tls_V35.HZ12.Kh-20160313-1_armv7soft.ipk.
Collected errors:
* opkg_download: Failed to download http://pkg.entware.net/binaries/armv7/pixelserv-tls_V35.HZ12.Kh-20160313-1_armv7soft.ipk, wget returned 4.
* opkg_install_pkg: Failed to download pixelserv-tls. Perhaps you need to run 'opkg update'?
* opkg_install_cmd: Cannot install package pixelserv-tls.

admin@RT-AC68U:/tmp/home/root# opkg update
Downloading http://pkg.entware.net/binaries/armv7/Packages.gz.
Collected errors:
* opkg_download: Failed to download http://pkg.entware.net/binaries/armv7/Packages.gz, wget returned 4.
admin@RT-AC68U:/tmp/home/root#

 

[ryzhov_al]

Some IPs/subnets was blocked during last DDoS attack. See how to use proxy for opkg.

 

[bayern1975]

Some IPs/subnets was blocked during last DDoS attack. See how to use proxy for opkg.

it is ok now.....

 

[RMerlin]

AiCloud use port 443 for default....but i would like to use that port for https....any solution for this?

Reconfigure AiCloud to use a different port.

 

[XIII]

Thank you for creating this!

I would like to use this (manually, without Entware) in combination with an automatic HTTP-Proxy (proxy.pac file) on my iOS devices.

1) So far all my attempts failed until I used my external (ISP) IP address (or the *.asuscomm.com DDNS address) instead of my internal (192.168.x.y) IP address as the first pixelserv parameter. Is it supposed to work like this? (Did I not understand correctly by trying a local address?)

2) Even though I do not use the "-l" parameter I see many logs like listed below in /jffs/syslog.log. Won't this wear out the flash memory?

Apr 27 22:22:34 pixelserv[5405]: Sending HTTP 501 response for unknown HTTP method or non-SSL, non-HTTP request: CONNECT

 

[mstombs]

Never considered using the WAN IP, but this is an old Linux feature, the router owns all the IP addresses they just get routed via specific interfaces, and accessing the router via WAN IP from the LAN is allowed for "WAN IP local nat loopback". I assign a secondary IP on the LAN interface via services-start script which is used to start Entware, and use that for redirection and pixelserv. Note you must move the router gui off ports 80,443 as it hogs those ports on all IPs.

PIXIP=192.168.66.254
ifconfig br0:0 $PIXIP up

 

[XIII]

I'm not using Entware... I do use port 8080 instead of 80.

Most server stats are zero or a low number, except for this one:

bad: # of unknown HTTP methods 568

1) Does this mean I'm doing it wrong? (It does seem to block ads on iOS though)

2) Is there a way to prevent (any pixelserv) logging to syslog.log?

 

[mstombs]

If you do not use ports 80 and 443 you need iptables diverts, your browser will be asking for data on these ports! You need to define a second port for https. I currently have no 'bad' counts

pixelserv-tls version: V35.HZ12.Kh compiled: Mar 26 2016 12:16:28 options: 192.168.66.254 -p 80 -p 81 -p 8080 -p 8081 -k 443 -o 2
uts: Uptime 4 days 12:09
req: Total # of requests 8393
avg: Avg size of reqs 939 bytes
rmx: Max size of reqs 5707 bytes
tav: Avg process time 437 ms
tmx: Max process time 3970 ms
err: # of error reqs 0
tmo: # of client timeout 507
cls: # of client shutdown 4135
nou: # of reqs w/o URL 0
pth: # of invalid URL 0
nfe: # of missing file ext 1264
ufe: # of unknown file ext 51
gif: # of GIF reqs 63
bad: # of unknown HTTP methods 0
txt: # of TXT reqs 1192
jpg: # of JPG reqs 0
png: # of PNG reqs 0
swf: # of SWF reqs 0
ico: # of ICO reqs 2
slh: # of HTTPS /w a good cert 3243
slm: # of HTTPS w/o a cert 11
sle: # of HTTPS /w a bad cert 0
slu: # of unrecognized HTTPS 1044
sta: # of HTML stats 2
stt: # of text stats 1
204: # of HTTP/204 (no content) 0
rdr: # of redirects 1169
pst: # of POST method 6
hed: # of HEAD method 0
log: access log enabled (0=no 1=yes) 0

On an N66 the logs are only in ram disk, I do not know how not to get anything in the log, I think a code change would be required to support another 'quiet' option!

 

[XIII]

I use port 8080 for HTTP and 8081 for HTTPS. I can (already) open <pixelserv.ip>:8080/servstats in my browser to see the stats. Does this mean iptable changes are not needed?

I have temporarily disabled copying /tmp/syslog.log to /jffs by creating two empty directories (/jffs/syslog.log and /jffs/syslog.log-1), as posted by RMerlin in the past.

I also enabled "-l" logging and now see lines like these (where 192.168.1.104 is my iPad trying to load ads):

Apr 28 22:30:26 pixelserv[6419]: Sending HTTP 501 response for unknown HTTP method or non-SSL, non-HTTP request: CONNECT
Apr 28 22:30:26 pixelserv[6419]: ( 9) 192.168.1.104: ads.mopub.com CONNECT ads.mopub.com:443 HTTP/1.1

Maybe I misunderstood and pixelserv is not meant to be a black hole proxy server?

It does seem to work as that as well, since it does block the ads on the iPad by sending status code 501, which fits the "requirements"?

I'd rather have a transparent pixel than the 501, but I guess I'm combining two things (pixelserv, proxy.pac) that are not supposed to cooperate that way?

Perhaps I need privoxy instead? (I'm still learning about all this...)

 

[mstombs]

Pixelserv doesn't know about the "http connect" tunnel method hence the response. I guess that is a method for a real proxy server? If you ask for <pixelserv.ip>:8080/anything.gif you will get the null pixel, same for png, swf etc