Aegis Aegis 1.7.x

[HELLO_wORLD]

I love your Aegis and it's very impressive to me.

I just missed the statistics, so I added the list of worst abusers (Blocked by Aegis i.e.) to the add-on.
It would make sense to have this as part of Aegis, but I know you want to keep Aegis minimal.
However I wish you consider to implement something like that with possibility to sort by clicking headings etc.
But until then I'll continue to support Aegis as much as I can.
I mean, who want to run these routers without Aegis?
And I'm so happy to bring up the abuse list, because just then I can get an understanding of how good Aegis is!
Aegis rocks!
Example:

 POS   NUM   FIRST LAST       BPM IP              ORG                                 TIMEZONE             COUNTRY REGION               CITY                 LOC                  HOSTNAME                
____ _____ _______ _______ ______ _______________ ___________________________________ ____________________ _______ ____________________ ____________________ ____________________ _________________________
   1   122   28485-43830     0.48 45.143.200.102  AS212283 ROZA HOLIDAYS EOOD         Europe/Sofia         BG      Sofia-Capital        Sofia                42.697,23.3241                                
   2   102   28523-41026     0.49 125.64.94.134   AS38283 CHINANET SiChuan Telecom In Asia/Shanghai        CN      Sichuan              Deyang               31.130,104.3820                              
   3    96   29014-43407     0.40 146.88.240.4    AS20052 Arbor Networks              America/Detroit      US      Michigan             Southfield           42.473,-83.2219      "www.arbor-observatory.com"
   4    81   28600-37351     0.56 89.248.165.98   AS202425 IP Volume inc              Europe/Amsterdam     NL      North Holland        Amsterdam            52.374,4.8897        "recyber.net"            
   5    74   33079-44000     0.41 167.99.243.100  AS14061 DigitalOcean                Europe/Berlin        DE      Hesse                Frankfurt am Main    50.115,8.6842                                
   6    71   26561-39751     0.32 89.248.165.69   AS202425 IP Volume inc              Europe/Amsterdam     NL      North Holland        Amsterdam            52.374,4.8897        "recyber.net"            
   7    66   27303-44229     0.23 183.136.225.16  AS58461 CT-HangZhou-IDC             Asia/Shanghai        CN      Shanghai             Shanghai             31.222,121.4581                              
   8    65   29758-40300     0.37 89.248.165.63   AS202425 IP Volume inc              Europe/Amsterdam     NL      North Holland        Amsterdam            52.374,4.8897        "recyber.net"            
  ..
  32    27   32843-32846   405.00 81.161.63.100   AS202984 Chernyshov Aleksandr Aleks Europe/Moscow        RU      Moscow               Moscow               55.752,37.6156                        
  ..
  56    18   32765-32768   270.00 193.123.70.211  AS31898 Oracle Corporation          Asia/Dubai           AE      Dubai                Dubai                25.077,55.3093
  ..
  86    11   28250-28250   660.00 172.105.77.209  AS63949 Linode                      Europe/Berlin        DE      Hesse                Frankfurt am Main    50.115,8.6842        "li2038-209.members.linode.com"
  87    11   27946-33767     0.11 167.248.133.25  AS398722 Censys                     America/Chicago      US      Illinois             Chicago              41.850,-87.6500      "scanner-03.ch1.censys-scanner.com"
  88    10   33339-33339   600.00 74.82.47.59     AS6939 Hurricane Electric LLC       America/Los_Angeles  US      California           San Jose             37.339,-121.8950     "scan-10n.shadowserver.org"
  89    10   29263-43667     0.04 5.188.206.157   AS59900 Balkan Internet Exchange Lt Europe/Sofia         BG      Sofia-Capital        Sofia                42.697,23.3241                                
  90    10   42167-43877     0.35 193.27.229.47   AS49505 OOO Network of data-centers Europe/Moscow        RU      St.-Petersburg       Saint Petersburg     59.938,30.3141                                
  91    10   33248-33248   600.00 150.109.182.140 AS132203 Tencent Building           Asia/Bangkok         TH      Bangkok              Bangkok              13.754,100.5014                              
  92     9   35917-35917   540.00 89.40.70.51     AS3280 LayerBridge SRL              Europe/Bucharest     RO      Bucure##ti           Bucharest            44.432,26.1063       "hecadigi.co.uk"        
  93     9   30149-30149   540.00 89.248.169.12   AS202425 IP Volume inc              Europe/Amsterdam     NL      North Holland        Amsterdam            52.374,4.8897                                
  94     9   30616-30616   540.00 89.248.168.220  AS202425 IP Volume inc              Europe/Amsterdam     NL      North Holland        Amsterdam            52.374,4.8897        "security.criminalip.com"
  95     9   28161-39566     0.05 89.248.165.73   AS202425 IP Volume inc              Europe/Amsterdam     NL      North Holland        Amsterdam            52.374,4.8897        "recyber.net"

That is a very good idea!

I will think about that, but that would be really cool in the web companion to have some stats...
The log daemon could build a metrics file, that could be used by the web companion, or any geek wanting to feed custom grafana graphs from the metrics.
Only thing is I would store them in RAM only, or USB, to avoid writing to the flash memory...

Not sure when I will find the time, but definitely in the todo list

Thank you @kamoj

 

[n1llam1]

Could it be that the Aegis "nightly update" is run by a cron job at the same time as the Bandwidth monitoring?
Both "programs" use the net-wall to update the iptables.
You can on the net-wall log by:

touch /var/log/net-wall.log

If you use the add-on default, try to change the Aegis cron job to not run simultaneously with Bandwidth Monitor:

[ -x /opt/bolemo/scripts/aegis ] && sleep 29 && /bin/sh /opt/bolemo/scripts/aegis refresh -html

(The Bandwidth Monitoring code is almost unchanged since implemented.
It scans for new devices every minute,
updates the counters every 2:nd minute (30:th minute at night)
)

Thanks for the additional information. I have updated the Aegis cron job as above and will circle back again after I have 5.4b27 up and running for several days.

 

[mith_y2k]

That is a very good idea!

I will think about that, but that would be really cool in the web companion to have some stats...
The log daemon could build a metrics file, that could be used by the web companion, or any geek wanting to feed custom grafana graphs from the metrics.
Only thing is I would store them in RAM only, or USB, to avoid writing to the flash memory...

Not sure when I will find the time, but definitely in the todo list

Thank you @kamoj

I love the idea of a top 10 or top 20 blocked IPs. I think it should also have a time frame, I can imagine some being blocked because added to a list or because of a malware. Knowing an IP is being blocked a lot this week, but not last week seems useful.

 

[Luizk]

Hi guys i have to unset aegis because i have problems with xbox Network. . Unset and voala xbox live works flawless. Any advice to solve this. My Problems: party chat disconnet and kick from online games.

 

[HELLO_wORLD]

Hi guys i have to unset aegis because i have problems with xbox Network. . Unset and voala xbox live works flawless. Any advice to solve this. My Problems: party chat disconnet and kick from online games.

Hello @Luizk
That’s when the logging becomes handy

When that happens, check the log and you will see what is being blocked (it should even show your xbox device being the destination and/or the source on the LAN).

Once you have established which IP(s) or range of IPs is/are being blocked, just add them to the whitelist and restart aegis

 

[HELLO_wORLD]

Just a quick feedback: I am working on the metrics.

I first had to deal with several requirements:
- keeping it simple.
- keeping it low resources (CPU and memory).
- limiting as much as possible write operations (particularly for the ones without external drive).

After several tries, I decided to go this way:
- the aegis log file will not be limited by number of records (like now), but by a TTL. Only records less than 1 sliding day are kept, meaning the aegis log file will contain at any moment the last 24 hours of records (up to 10 minutes more).
All of that is happening in /var/log, that is in RAM, so there will be no persistance at reboots, and it will then take 24 hours to build a full sliding day log, but it means no write to flash memory.
The TTL will be preset to 86400 seconds (24 hours), but it can be changed using uci or an aegis command that will replace the get/set log history command.

From that log file, at any time (except the first 24 hours after a reboot), it will be possible to get the stats for the last 24 hours.


There are a lot of exciting metrics that could be made, sorting, analyzing (with ports, with devices on. LAN, etc...) but it is too complicated for the firmware environment for my taste. It would require to run a database engine, and none is available without Entware, on top of that, storing it outside of RAM would create a lot of write operations, and I don’t like this idea.

This is still in progress and subject to changes. I will now be implementing that and the next release will include the metrics.

 

[Gar]

I setup Aegis but it always changes my R7800 to my computer's MAC address. It will not work on default or a cloned address, is this normal?

 

[HELLO_wORLD]

I setup Aegis but it always changes my R7800 to my computer's MAC address. It will not work on default or a cloned address, is this normal?

Aegis does not uses or changes the MAC address of any interface on the router, and works independently of them.

Something else is altering your setup. Can you share about your setup, what you tried (commands, settings), and how you came to this conclusion?

 

[Gar]

Aegis does not uses or changes the MAC address of any interface on the router, and works independently of them.

Something else is altering your setup. Can you share about your setup, what you tried (commands, settings), and how you came to this conclusion?

Aegis does not uses or changes the MAC address of any interface on the router, and works independently of them.

Something else is altering your setup. Can you share about your setup, what you tried (commands, settings), and how you came to this conclusion?

I wish I had a clue, probably something I just don't understand yet. So far all I know is if I clear nvram and restore everything is fine, but installing aegis and rebooting changes the mac address. I use stubby but that doesn't seem to matter. Don't confuse this with a complaint, I'm not complaining about aegis as it still works fine! Another computer mystery, #234,810 for me...

 

[HELLO_wORLD]

I wish I had a clue, probably something I just don't understand yet. So far all I know is if I clear nvram and restore everything is fine, but installing aegis and rebooting changes the mac address. I use stubby but that doesn't seem to matter. Don't confuse this with a complaint, I'm not complaining about aegis as it still works fine! Another computer mystery, #234,810 for me...

There is definitely a mystery here, as aegis does not write on the nvram, and does not care about the MAC, but since it happens to you, something on your setup is definitely doing so. If it clones your computer MAC, it means that whatever is doing it knows this MAC.
Have you been changing the MAC in the past?

What is your setup? USB?
What steps are you doing to do a clean install, to install aegis, etc?
The debug output could be useful too.

What is the output of nvram show | grep mac
What happens if you set the MAC address back after installation?

 

[Gar]

There is definitely a mystery here, as aegis does not write on the nvram, and does not care about the MAC, but since it happens to you, something on your setup is definitely doing so. If it clones your computer MAC, it means that whatever is doing it knows this MAC.
Have you been changing the MAC in the past?

What is your setup? USB?
What steps are you doing to do a clean install, to install aegis, etc?
The debug output could be useful too.

What is the output of nvram show | grep mac
What happens if you set the MAC address back after installation?

No USB drive. Aegis installed onto R7800. Clean install = factory reset, then your github default install link. Everything works as it should until I reboot (not required I know) then no internet until Netgear connection wizard automatically settles on comp's MAC. Changing MAC address back doesn't work, no internet connection. A reset seems to be only cure (for my skill level) at that point.

 

[HELLO_wORLD]

No USB drive. Aegis installed onto R7800. Clean install = factory reset, then your github default install link. Everything works as it should until I reboot (not required I know) then no internet until Netgear connection wizard automatically settles on comp's MAC. Changing MAC address back doesn't work, no internet connection. A reset seems to be only cure (for my skill level) at that point.

Thank you.
This is quite a weird issue.
And I supposed you checked that this problem does not appear after a clean install, then a reboot (but not installing aegis).

Generally, when only a cloned MAC is required (and working like in your case), it means that the ISP is only allowing a registered MAC. So here, I am quite lost on aegis (iptables rules) could be linked to that.

When you have aegis running, and the original MAC is in place (and internet is not working), do you know if it is the WAN that has no internet (the router itself has no internet access), or only the LAN that has no access to internet (meaning the router is somehow blocking all traffic between LAN and WAN)?

When that happens, what does the aegis log shows? Any blocked traffic showing? That would indicate if aegis is blocking something, and what.
What does a ping 8.8.8.8 from router tells? and from aegis, test ip 8.8.8.8?
Also, the output of the aegis debug either from web interface or ssh/telnet would help to see clearer here... as would knowing what kind of connection you have, static or dynamic ip? Do you have IPv6 enabled, and if so, is it functional or not when the MAC problem occurs?

 

[Gar]

Thank you.
This is quite a weird issue.
And I supposed you checked that this problem does not appear after a clean install, then a reboot (but not installing aegis).

Generally, when only a cloned MAC is required (and working like in your case), it means that the ISP is only allowing a registered MAC. So here, I am quite lost on aegis (iptables rules) could be linked to that.

When you have aegis running, and the original MAC is in place (and internet is not working), do you know if it is the WAN that has no internet (the router itself has no internet access), or only the LAN that has no access to internet (meaning the router is somehow blocking all traffic between LAN and WAN)?

When that happens, what does the aegis log shows? Any blocked traffic showing? That would indicate if aegis is blocking something, and what.
What does a ping 8.8.8.8 from router tells? and from aegis, test ip 8.8.8.8?
Also, the output of the aegis debug either from web interface or ssh/telnet would help to see clearer here... as would knowing what kind of connection you have, static or dynamic ip? Do you have IPv6 enabled, and if so, is it functional or not when the MAC problem occurs?

I'll start the full investigation tomorrow, thanks. Aegis works fine when first installed and started, it's only after the first reboot the issue starts. I also had stubby running which I will not do for testing.

 

[Gar]

Thank you.
This is quite a weird issue.
And I supposed you checked that this problem does not appear after a clean install, then a reboot (but not installing aegis).

Generally, when only a cloned MAC is required (and working like in your case), it means that the ISP is only allowing a registered MAC. So here, I am quite lost on aegis (iptables rules) could be linked to that.

When you have aegis running, and the original MAC is in place (and internet is not working), do you know if it is the WAN that has no internet (the router itself has no internet access), or only the LAN that has no access to internet (meaning the router is somehow blocking all traffic between LAN and WAN)?

When that happens, what does the aegis log shows? Any blocked traffic showing? That would indicate if aegis is blocking something, and what.
What does a ping 8.8.8.8 from router tells? and from aegis, test ip 8.8.8.8?
Also, the output of the aegis debug either from web interface or ssh/telnet would help to see clearer here... as would knowing what kind of connection you have, static or dynamic ip? Do you have IPv6 enabled, and if so, is it functional or not when the MAC problem occurs?

I accidentally discovered the issue only occurs when using my main computer at home. I had to connect and do the setup with a different computer (still at home) and the issue didn't repeat. I don't know why the other computers MAC address was used after reboot...and I'd really like to know! So thanks for following this "issue", but it seems the software is fine and I have an unrelated, unsolved problem here.

 

[mith_y2k]

I think we all want to know what it is when you figure it out

 

[HELLO_wORLD]

I think we all want to know what it is when you figure it out

I second that.

@Gar , I am glad that you have a solution to have your setup working, and I hope you will be able to get to the bottom of this bizarre issue. We all are very puzzled as you must be about this.

Don’t hesitate to create a thread in this forum to give news on your progress, and/or ask for help.

 

[Gar]

Plugged my main comp into the 7800 with a fully functional Aegis install from other comp and rebooted...MAC address changes to computer MAC. Factory reset the 7800 and did minimal reconfig and installed Aegis from the other comp and reboot...remains on default MAC. So far narrowed down to one comp, might be something I added or modified but can no longer recall? A full Win reinstall is probably overdue on this box anyway.

 

[HELLO_wORLD]

Plugged my main comp into the 7800 with a fully functional Aegis install from other comp and rebooted...MAC address changes to computer MAC. Factory reset the 7800 and did minimal reconfig and installed Aegis from the other comp and reboot...remains on default MAC. So far narrowed down to one comp, might be something I added or modified but can no longer recall? A full Win reinstall is probably overdue on this box anyway.

I don’t think the R7800 has anything that automatically changes its MAC.
There must be something on the one computer that access the router and changes it’s configuration using UPnP (although I am not sure it can alter MAC) or some Windows Netgear utility that decides to autoconfig your router.

 

[sppmaster]

Plugged my main comp into the 7800 with a fully functional Aegis install from other comp and rebooted...MAC address changes to computer MAC. Factory reset the 7800 and did minimal reconfig and installed Aegis from the other comp and reboot...remains on default MAC. So far narrowed down to one comp, might be something I added or modified but can no longer recall? A full Win reinstall is probably overdue on this box anyway.

Why don't you check this first?

 

[Gar]

A fresh Win10 install eliminated the issue. As much as I wanted to know the source of the issue, I needed the comp to work well without a possible virus, etc. Thanks again for the help.