Aegis Aegis (simple yet effective protection)

[HELLO_wORLD]

First, great job @mith_y2k !

Aegis already have a way to check the post-mount, but I am not sure it is the way to go here, as some users might not have a usb drive and aegis can be installed internally as well (and post mount is no use here...)

I am thinking more of using rc.local to check the configuration and add the line if needed.
But I have a question: can lighttpd have several config files?
For example, would it accept a 31-aegis.conf that adds only the entry we want, or overwrite just the cgi-bin setting?
In that case, I would create the file in my bolemo/etc folder and just soft link it in /etc/lighttpd/conf.d/.

I would be interested to have to whole default 30-cgi.conf file for sure, and know what is in the conf.d folder. If the Orbi firmware is semi-opensourced like the R7k and R9k ones, I can find that.

For the other point, it is either problematic or very simple...
uhttpd on R7k and R9k models also has a very short length for cgi output, and throw an error 500; however, it still sends the data preceded by an error line. I just ignore the error line and all is ok.
On the Orbi, when you have the 500 error, do you get the data? Seems like it from your screenshot.
What I need is the source of the error 500 http answer. From that, I will be able to ignore the error, remove all garbage (Internal Server blabla) before the data, like I do for uhttpd.

Idea Dream:
Wouldn’t it be great to have some docker containers with the exact environment of our routers?
A R7800 container, a R9000 one and an Orbi one?
Would be perfect to develop for them without having the devices.
Not sure if it would be doable because anything hardware related would not work, and creating dummies would be involved...

 

[mith_y2k]

Aegis already have a way to check the post-mount, but I am not sure it is the way to go here, as some users might not have a usb drive and aegis can be installed internally as well (and post mount is no use here...)

I am thinking more of using rc.local to check the configuration and add the line if needed.
But I have a question: can lighttpd have several config files?
For example, would it accept a 31-aegis.conf that adds only the entry we want, or overwrite just the cgi-bin setting?
In that case, I would create the file in my bolemo/etc folder and just soft link it in /etc/lighttpd/conf.d/.

I would be interested to have to whole default 30-cgi.conf file for sure, and know what is in the conf.d folder. If the Orbi firmware is semi-opensourced like the R7k and R9k ones, I can find that.

I'm no expert of lighttpd, really just playing around here. It looks like I cannot attach files, but here is the information you asked for and also how I did it.

This is what's in lighttpd conf directory:

root@RBR50:/etc/lighttpd/conf.d# ls -l
-rw-r--r--    1 root     root           789 Jan 19 08:39 30-cgi.conf
-rw-r--r--    1 root     root           173 Dec 23 02:58 30-rewrite.conf

As you suggested I reverted the 30-cgi.conf file to the default version and then created 31-aegis.conf, this is how it looks like:

#######################################################################
##
##  CGI modules
## --------------- 
##
## See http://redmine.lighttpd.net/projects/lighttpd/wiki/docs_modcgi
##

##
## Plain old CGI handling
##
## For PHP don't forget to set cgi.fix_pathinfo = 1 in the php.ini.
##
# cgi.assign                 += (
#                "aegis_web.cgi" => "/opt/bolemo/www/cgi-bin/aegis_web.cgi"
#                )
$HTTP["url"] =~ "/bolemo/" {
      cgi.assign = ( "aegis_web.cgi" => "/opt/bolemo/www/cgi-bin/aegis_web.cgi" )
}

##

With the above I can now reload lighttpd and it seems to work as expected (I played around with the basic Web UI and Aegis.

For the other point, it is either problematic or very simple...
uhttpd on R7k and R9k models also has a very short length for cgi output, and throw an error 500; however, it still sends the data preceded by an error line. I just ignore the error line and all is ok.
On the Orbi, when you have the 500 error, do you get the data? Seems like it from your screenshot.
What I need is the source of the error 500 http answer. From that, I will be able to ignore the error, remove all garbage (Internal Server blabla) before the data, like I do for uhttpd.

About the Error 502 this is what I gathered and what I can't figure out:

1. Any time I get an error (4xx, 5xx, etc), I see a popup in my browser and there is NO data displayed; it either displays nothing (for 4xx errors) or the "Internal server error" message you can see in the screenshot above (for 5xx errors)
2. it seems like it works 50% of the times or a little less (see in my screenshot above in the Web inspector window more or less every other time the call to "aegis_web.cgi" is red, those are the times it fails
3. when it fails it logs just 1 line in the lighttpd error.log file, but if the same error happens again it doesn't even log it; if I restart the Web server it logs it once more then silence

Other than the Internal server error message I don't see any output from aegis_web.cgi, so I cannot tell what the error is. According to some quick research online it sounds like the headers you are sending are too long. If there was a way for me to see the output from aegis_web.cgi I could try debugging more.

 

[HELLO_wORLD]

@mith_y2k that is nice work!
The conf fix will be easy, this part is good news.

About the aegis_web.cgi, you can get the output by calling it from terminal.

You can for example try:
/www/bolemo/cgi-bin/aegis_web.cgi log
Or
/www/bolemo/cgi-bin/aegis_web.cgi status

aegis_web.cgi is strictly called via ajax through url (GET) for various html outputs or json ones, except the black/white lists that are sent through POST when saved from web interface.
No specific headers are generated except what is automatically sent by the browser on one side, and the http cgi server on the other.

This 50% failure rate makes no sense, because the content is about the same except the first time when it loads about 300 lines of history.
I think it might be related with frequency. Log is refreshed every 10 seconds... Maybe the server it does not like to be called so often?

 

[mith_y2k]

An idea related to the Web companion: for blocked requests it could be handy to have a link to a site like https://exchange.xforce.ibmcloud.com/ip, i.e. I can click on the blocked IP and a new browser window opens to https://exchange.xforce.ibmcloud.com/ip/23.227.38.65 and shows me if/why the IP is suspicious.

In my brief research it looks like most of the IPs in fireHOL are also shown as malicious here with a few exceptions. I have found it useful as I was debugging and testing

 

[mith_y2k]

I now realize that actually you already have this feature. It looks like my browser settings and adblockers are preventing it from working. All good, thank you

For anyone using Safari you need to click View -> Reload without content blockers and then it works.

 

[HELLO_wORLD]

1.6.9

Full Orbi support (including Web Companion working on lighttpd). Big thank to @mith_y2k for his interest and help in this!

Silent bug fixes.

Changed the installer code a bit. Apparently, one line was not working properly on latest firmware.

Now the upgrade command offers to reapply the current version if for some reason there is a need for it.

 

[h1ms3lf]

just P E R F E C T ! Thank you!

 

[R. Gerrits]

Idea Dream:
Wouldn’t it be great to have some docker containers with the exact environment of our routers?
A R7800 container, a R9000 one and an Orbi one?
Would be perfect to develop for them without having the devices.
Not sure if it would be doable because anything hardware related would not work, and creating dummies would be involved...

Perhaps this can be used

GitHub - attify/firmware-analysis-toolkit: Toolkit to emulate firmware and analyse it for security vulnerabilities

Toolkit to emulate firmware and analyse it for security vulnerabilities - GitHub - attify/firmware-analysis-toolkit: Toolkit to emulate firmware and analyse it for security vulnerabilities

github.com

quickly tried to boot the voxel r7800 with that toolkit, without much success. I see it booting, but then it starts spitting errors and never can ping the router.

In the past I did succeed in using QEMU to run an emulated version of my netgear nas.

 

[ducdeptrai1234]

I got softirq around 3% constantly after enabling aegis. Is this normal behavior?

 

[HELLO_wORLD]

I got softirq around 3% constantly after enabling aegis. Is this normal behavior?

Yes if you have logging enabled. Turning it off should reduce it to around 1%

 

[D3FenD3r]

in the r9000 the usb ports no longer work. then i'm back to voxel firmware ... how do i install kamoj the 2 power and internet lights turn off, even if it works ................ nvram as far as i did the reset and full of various installations ....... nvram erase.nvram commit,
reboot did not produce results. ideas?

 

[kamoj]

Wrong thread...
But have you read kamoj FAQ?

   nvram default
   nvram commit
   ngmtd="$(awk -F: '/"netgear"$/ {print $1}' /proc/mtd | grep mtd)"
   [ -n "$ngmtd" ] && flash_erase /dev/"$ngmtd" 0 0
   reboot

in the r9000 the usb ports no longer work. then i'm back to voxel firmware ... how do i install kamoj the 2 power and internet lights turn off, even if it works ................ nvram as far as i did the reset and full of various installations ....... nvram erase.nvram commit,
reboot did not produce results. ideas?

 

[ducdeptrai1234]

Yes if you have logging enabled. Turning it off should reduce it to around 1%

I have the logging already turned off. I don't understand why it happened. Anyway, does 3% softirq makes the anything bad for the router?

 

[HELLO_wORLD]

I have the logging already turned off. I don't understand why it happened. Anyway, does 3% softirq makes the anything bad for the router?

No, no worries about 3%.

Do you have a lot of blocked traffic by aegis?

I have reached moments with about 5 hits per second, and softirq is between 0.5 and 1.5 (5% with logging on).

 

[HELLO_wORLD]

News for Orbi users: @Voxel included iprange into the latest firmware, so Entware is not mandatory anymore on Orbi to have it.
Aegis was working on Orbi without iprange, but this allow better optimization.

Thank you @Voxel for this nice touch.

 

[ducdeptrai1234]

No, no worries about 3%.

Do you have a lot of blocked traffic by aegis?

I have reached moments with about 5 hits per second, and softirq is between 0.5 and 1.5 (5% with logging on).

I have my softirq about 3% without log, and 6% with log. Here is the screen shots of my softirq when logging off

 

[R. Gerrits]

I have my softirq about 3% without log, and 6% with log. Here is the screen shots of my softirq when logging off

I guess it also all depends on how many devices you have in your network and how much network traffic they are generating.
My R7800 usually has softirq below 1% and most is consumed by NET_RX. (with aegis + logging)
But if I start a speedtest (.net) test from my laptop, then softirq goes to 11%, while it saturates the downlink (250 Mbps)

 

[ducdeptrai1234]

I guess it also all depends on how many devices you have in your network and how much network traffic they are generating.
My R7800 usually has softirq below 1% and most is consumed by NET_RX. (with aegis + logging)
But if I start a speedtest (.net) test from my laptop, then softirq goes to 11%, while it saturates the downlink (250 Mbps)
View attachment 30000

I agreed; you definitely has more NET_RX than me. I believe the high softirq happened with my specific network. I will invest it further. Thanks guys!

 

[HELLO_wORLD]

Version 1.6.10

Minor update:
- Changes versioning algorithm.
- Updated ReadMe about iprange (now installed in all @Voxel firmwares, or in next releases).

 

[HELLO_wORLD]

WARNING

The next major release 1.7.0 (probably the next one unless a bug appears and needs to be fixed before that) will require iprange.
Since iprange is now installed on all @Voxel firmwares, it simply means that Aegis will require latest firmwares (or if not: having iprange manually installed)

I can simplify my code to not have a part dealing when iprange is not installed, and allowing me to focus more on new features.
The next major release will allow to have custom lists (blacklists and whitelists) specific to WAN or VPN.

So all current lists will be working the same, and for most of you, it won’t change anything in how your custom lists are; it just means that it will be possible to create custom lists that will be applied only to WAN or VPN (and of course the regular lists will continue to apply to both).