Suricata Suricata - IDS on AsusWRT Merlin

[glehel]

One of the biggest developers! Welcome here!

 

[rgnldo]

From what I observed, a big jump, a request with the Entware dev team.
Advances for Suricata at FW Merlin
Package Request: libpfring and suricata compiled with libpfring

 

[glehel]

if I understand correctly it is not necessary to add pf_ring. I now have an ax88u. in the evening i install suricata with this config and test it.

 

[XIII]

# Linux high speed capture support
   tpacket-v3: yes

When using that I get this on my AC86U:

Aug 14 13:03:59 ac86u suricata[6993]: 14/8/2020 -- 13:03:59 - <Warning> - [ERRCODE: SC_ERR_RUNMODE(187)] - Using tpacket_v3 in IPS mode will result in high latency

Do you get that too?

 

[rgnldo]

if I understand correctly it is not necessary to add pf_ring.

pcap function

 

[rgnldo]

faux123 Changlog-fork: updated documentation

Additional Notes:
With pktrunner (proprietary kernel module) and pktflow acceleration
(another proprietary kernel module) disabled when entering promiscuous mode as
required by Suricata + CAKEQoS, every bit of performance optimization from
base Linux kernel will help to make up for lack of proprietary HW accelertaion.

TrendMicro has several proprietary kernel modules such as tdts_udbfw (TrendMicro engine), tdts_udb (TrendMicro database interface) and the infamous
DCD (Data Collection Daemon, which constantly has level 3 translation errors),
these and the other proprietary kernel modules makes the firmware difficult to
be fully optimized without breaking something.

Not having full source to those modules makes this enhancement projects harder and more difficult to keep bug free (whenever the kernel panicked in those proprietary modules, it is nearly impossible to troubleshoot or fix).

 

[faux123]

# Linux high speed capture support
   tpacket-v3: yes

When using that I get this on my AC86U:

Aug 14 13:03:59 ac86u suricata[6993]: 14/8/2020 -- 13:03:59 - <Warning> - [ERRCODE: SC_ERR_RUNMODE(187)] - Using tpacket_v3 in IPS mode will result in high latency

Do you get that too?

Good catch, I overlooked it, yes, change it to tpacket-v2, I've updated my post as well!

 

[faux123]

pcap function

Yes, for pcap having an optimized libpcap using PF_RING will improve its performance. af-packet with mmap is the same as PF_RING, both have highly optimized ring buffers and very low overhead userspace access, so PF_RING will be redundant for suricata itself, but libpcap functions within suricata may get performance boost from it.

 

[rgnldo]

Yes, for pcap having an optimized libpcap using PF_RING will improve its performance. af-packet with mmap is the same as PF_RING, both have highly optimized ring buffers and very low overhead userspace access, so PF_RING will be redundant for suricata itself, but libpcap functions within suricata may get performance boost from it.

Is there a way to host your compiled fw fork?

 

[faux123]

Is there a way to host your compiled fw fork?

Here you go https://github.com/faux123/asuswrt-merlin.ng/releases/tag/384.19-beta2 as some of you know my work from my Android past, your warranty is now void . Download and use at your own risk. WARNING: this is a GPL only release, meaning if you use this release with AdaptiveQoS or AiProtect (AiMesh is okay which I'm using myself), you will get RANDOM kernel panics (system crash and you will lose internet connectivity for 2 minutes until it rebooted)!

Because Zswap is enabled by default on my fork, I recommend you add the following line to your post-mount script located in /jffs/scripts/

echo 100 > /proc/sys/vm/swappiness

just before the swapon line. Zswap will use 25% of your RAM and using z3fold memory compression (about 2.6x compression ratio) to expand your RAM further. so 25% of 512MBytes, then 128Mbytes * 2.6 compression ratio = 332 MByte total compressed memory then add back to your remaining real RAM of 384 MBytes, you will have approximately 716 Mbytes of RAM to play with which helps the poor AC86U with just 512 MBytes of real RAM. This will help suricata a bit as suricata is a memory hog.

Oh and if you use wireguard, please follow the wireguard post in the VPN forum, the only thing you need to change is to change the line in the wireguard shell scripts (located /opt/etc/wireguard/wg-up wg-policy wg-server) where you see:

#insmod /opt/lib/modules/wireguard
insmod /lib/modules/4.1.27/kernel/net/wireguard/wireguard

so it uses the built-in kernel module rather than kernel module from Entware.

oh and for CakeQos make the following change to /jffs/addons/cake-qos/cake-qos script where

        #insmod /opt/lib/modules/sch_cake.ko 2>/dev/null
        insmod /lib/modules/4.1.27/kernel/net/sch_cake/sch_cake.ko 2>/dev/null

again to use the built-in kernel module rather than the module from Entware

for full change log see here: https://github.com/faux123/asuswrt-merlin.ng/blob/mainline_enh/Changelog-fork.txt

I don't want to overshadow Merlin's work here, so this is just a test release with NO support from me and I may not publish anything further. This was mainly for myself as I like to run my own forks on many open source projects I use myself.

 

[Mutzli]

EDIT: changed tpacket-v3 to tpacket-v2 for better latency

I have an AX88U and tried your configuration, but after a few minutes Suricata quits with the following error in the syslog:

Aug 14 10:46:40 kernel: device eth0 left promiscuous mode
Aug 14 10:46:40 kernel: device br0 left promiscuous mode

I did make a change to tpacket-v3: yes, other than that I have used the same values. I'll try to see if tpacket-v3 is the cause of this crash. I'm running it now with tpacket-v2.

 

[rgnldo]

I don't want to overshadow Merlin's work here, so this is just a test release with NO support from me and I may not publish anything further. This was mainly for myself as I like to run my own forks on many open source projects I use myself.

Smoothly. I'm used to it.

 

[rgnldo]

@faux123 I believe that its compilation will help a lot to members who want an FW adapted to Suricata, Cake-QOS, Wireguard. Perhaps providing the link, with the appropriate unstable warnings.

 

[rgnldo]

Aug 14 10:46:40 kernel: device eth0 left promiscuous mode Aug 14 10:46:40 kernel: device br0 left promiscuous mode

As far as I know, this is not a mistake. Working properly for Suricata.

 

[Mutzli]

As far as I know, this is not a mistake. Working properly for Suricata.

It seems to be stable with tpacket-v2 instead of tpacket-v3.

 

[ugandy]

i'm wondering what's the likelihood of merlin's FW including the cakeqos module at some point... sounds like for proper support it would be a requirement?

 

[heysoundude]

i'm wondering what's the likelihood of merlin's FW including the cakeqos module at some point... sounds like for proper support it would be a requirement?

I think @RMerlin supports it as best he can for the time being: until Asus includes it in their work/version, this is the best option for those of us who wish to use it.
As @rgnldo alluded to, it would require a fork for it to be built-in.

 

[heysoundude]

I finally got Suricata to operate in IDS/IPS on af-packet mode (which is the ONLY mode available for our Entware build, I was trying to get a PF_RING build but was having some issues with libraries and other Entware dependencies). Below is the relevant yaml code from my own config. mmap (memory mapped ring buffers) must be enabled for IPS to work, without it, it's only running in IDS mode. I'm running this on my AC86U (with suricata, I kinda wish I have the newer AX88U not for its wifi but for its quad core CPU and 1 GB of real RAM). The configuration is a good compromise for my own build, the average load on my router is around 3.50 so your router will be hotter with it. Download speed is close to my native ISP speed (which is around 200 Mbps), upload speed is somewhat limited around 160 Mbps which is far below my normal ISP speed of 225 Mbps (this is due to AC86U's CPU capability). I'm running CakeQoS as well (I don't like the proprietary closed sourced solutions from TrendMicro as it causes some random kernel panics on my own fork with their modules which I can't fix, I much prefer the GPL alternative which is Suricata+CakeQoS). You will experience around 2.1% packet drops (again this is due to dual core and limited real RAM of AC86U). I can live with this small amount of packet drops as I didn't experience any real usage issues with such packet loss.

Thanks to all those who contributed so far, it took me a while to read through tons of source code from Suricata and their piss poor documentations/examples to finally crack this.

# Linux high speed capture support
af-packet:
- interface: eth0 ## set your wan interface
   copy-mode: ips
   copy-iface: br0
   buffer-size: 3072
   cluster-id: 99
   cluster-type: cluster_flow
   use-mmap: yes
#   mmap-locked: yes
   tpacket-v2: yes
   ring-size: 3072
- interface: br0
   copy-mode: ips
   copy-iface: eth0
   buffer-size: 3072
   cluster-id: 98
   cluster-type: cluster_flow
   use-mmap: yes
#   mmap-locked: yes
   tpacket-v2: yes
   ring-size: 3072

EDIT: changed tpacket-v3 to tpacket-v2 for better latency

This is off-topic for this thread, but as you have a higher connection speed than I, I'm curious about your cake-qos settings.
Have you, for instance, adjusted your rtt to more closely reflect avg pings recorded by @Jack Yaz 's connmon? I've found that by stepping back from the default (100ms) to regional (30ms) when my avg ping is 7-8ms has smoothed something out on my 50/10 connection and my network feels much more responsive. my "quality" graph is flatlined at 100% indicating zero packet loss, and my speeds are averaging within 1-2% of what I pay for from my ISP. (I don't want to take this down to metro rtt 10ms, because when my ISP rotates my WAN IP, I can end up with 28ms pings, and I don't know what that would do to my connection/network)

 

[faux123]

This is off-topic for this thread, but as you have a higher connection speed than I, I'm curious about your cake-qos settings.
Have you, for instance, adjusted your rtt to more closely reflect avg pings recorded by @Jack Yaz 's connmon? I've found that by stepping back from the default (100ms) to regional (30ms) when my avg ping is 7-8ms has smoothed something out on my 50/10 connection and my network feels much more responsive. my "quality" graph is flatlined at 100% indicating zero packet loss, and my speeds are averaging within 1-2% of what I pay for from my ISP. (I don't want to take this down to metro rtt 10ms, because when my ISP rotates my WAN IP, I can end up with 28ms pings, and I don't know what that would do to my connection/network)

Actually I just used "best effort" and adjusted the initial bandwidth of my up/down speeds based on my average speedtest speeds. So far default setting is serving me well as my family were doing tons of streaming (youtube, disney+) while I was using video conferencing apps for work. So far I haven't experienced any issues where I needed to tweak cake setting at all.. I'm super happy with the default setting of "best effort". I love cake, just set and forget.. now with Suricata, this is match made in heaven as I now have a complete GPL (open source) software solutions to serve all my needs (I'm willing to give up some bandwidth for IPS/IDS as this will protect my entire family's internet experience).

 

[rgnldo]

I love cake, just set and forget.. now with Suricata, this is match made in heaven as I now have a complete GPL (open source) software solutions to serve all my needs (I'm willing to give up some bandwidth for IPS/IDS as this will protect my entire family's internet experience).

complete GPL (open source) software solutions. Just set and forget. That's what I always mention.

it would require a fork for it to be built-in.

This fork is what I recommend. But use if you have intermediate knowledge in linux. I'm using it on my other router.

I will wait for @juched to check the status of the Suricata manager and stats.