Suricata Suricata - IDS on AsusWRT Merlin

[Mutzli]

If I relaunch /jffs/addons/suricata/suricata_manager.sh install, I have :

26/8/2020 -- 18:17:51 - <Info> - Running suricata under test mode
Warning: Output_interface not supplied by user.  Falling back on default_output_interface "Console"
26/8/2020 -- 18:17:51 - <Notice> - This is Suricata version 4.1.8 RELEASE
26/8/2020 -- 18:17:51 - <Info> - CPUs/cores online: 4
26/8/2020 -- 18:17:51 - <Info> - fast output device (regular) initialized: fast.log
26/8/2020 -- 18:17:57 - <Info> - stats output device (regular) initialized: stats.log
26/8/2020 -- 18:17:57 - <Info> - 20 rule files processed. 3119 rules successfully loaded, 0 rules failed
26/8/2020 -- 18:17:57 - <Info> - Threshold config parsed: 0 rule(s) found
26/8/2020 -- 18:17:57 - <Info> - 3119 signatures processed. 216 are IP-only rules, 567 are inspecting packet payload, 2475 inspect application layer, 0 are decoder event only
26/8/2020 -- 18:18:01 - <Notice> - Configuration provided was successfully loaded. Exiting.
26/8/2020 -- 18:18:01 - <Info> - cleaning up signature grouping structure... complete
Starting suricata...              failed.

What is the output of:

suricata -T

 

[Olivier L]

all is fine with suricata -T, no errors.

 

[Mutzli]

all is fine with suricata -T, no errors.

Could be limited memory, do you have a swap file configured?

 

[Olivier L]

yes. 2GB.
I am running skynet too.

 

[Torson]

all is fine with suricata -T, no errors.

You probably have a stale /opt/var/run/suricata.pid file. Just delete it and the run:

suricata_manager start

 

[Mutzli]

I would do a reinstall of Suricata and see if that fixes the problem:
uninstall:

/jffs/addons/suricata/suricata_manager.sh uninstall

install:

mkdir /jffs/addons 2>/dev/null;mkdir /jffs/addons/suricata 2>/dev/null; curl --retry 3 "https://raw.githubusercontent.com/juched78/suricata-merlin/master/suricata_manager.sh" -o "/jffs/addons/suricata/suricata_manager.sh" && chmod 755 "/jffs/addons/suricata/suricata_manager.sh" && /jffs/addons/suricata/suricata_manager.sh install

 

[Olivier L]

You probably have a stale /opt/var/run/suricata.pid file. Just delete it and the run:

suricata_manager start

nothing in /opt/var/run.

 

[crkpot]

If I relaunch /jffs/addons/suricata/suricata_manager.sh install, I have :

26/8/2020 -- 18:17:51 - <Info> - Running suricata under test mode
Warning: Output_interface not supplied by user.  Falling back on default_output_interface "Console"
26/8/2020 -- 18:17:51 - <Notice> - This is Suricata version 4.1.8 RELEASE
26/8/2020 -- 18:17:51 - <Info> - CPUs/cores online: 4
26/8/2020 -- 18:17:51 - <Info> - fast output device (regular) initialized: fast.log
26/8/2020 -- 18:17:57 - <Info> - stats output device (regular) initialized: stats.log
26/8/2020 -- 18:17:57 - <Info> - 20 rule files processed. 3119 rules successfully loaded, 0 rules failed
26/8/2020 -- 18:17:57 - <Info> - Threshold config parsed: 0 rule(s) found
26/8/2020 -- 18:17:57 - <Info> - 3119 signatures processed. 216 are IP-only rules, 567 are inspecting packet payload, 2475 inspect application layer, 0 are decoder event only
26/8/2020 -- 18:18:01 - <Notice> - Configuration provided was successfully loaded. Exiting.
26/8/2020 -- 18:18:01 - <Info> - cleaning up signature grouping structure... complete
Starting suricata...              failed.

Based on this, Suricata is failing to start. Have you edited your yaml to suit your system? Can you post the top half of that file here?

 

[rgnldo]

pcap/IPS

without af_packet
Try

# IPS Mode Configuration
# PCAP
pcap:
  - interface: br0
    checksum-checks: auto
    promisc: yes

Insert Interface based by adding the desired interface to the cmdline:

suricata -c /opt/etc/suricata/suricata.yaml -i br0 -D

 

[rgnldo]

af_packet

Leave TSO enabled for Linux AF_PACKET runmode

suricata -c /opt/etc/suricata/suricata.yaml -i eth0 --set capture.disable-offloading=false

Could you update the release version with your changes?

 

[Torson]

Shouldn't the default policy be linux as opposed to windows in this section of the yaml file?

# Host specific policies for defragmentation and TCP stream reassembly.
host-os-policy:
  # Make the default policy windows.
  windows: [0.0.0.0/0]
  bsd: []
  bsd-right: []
  old-linux: []
  linux: []
  old-solaris: []
  solaris: []
  hpux10: []
  hpux11: []
  irix: []
  macos: []
  vista: []
  windows2k3: []

 

[Olivier L]

Based on this, Suricata is failing to start. Have you edited your yaml to suit your system? Can you post the top half of that file here?

my yaml is default as per installation. nothing edited yet. I have to learn how to fine tune. I have deinstall/reinstall, for now it work but I havent rebooted the router yet.

# Holds variables that would be used by the engine.
vars:

  # Holds the address group vars that would be passed in a Signature.
  address-groups:
    HOME_NET: "[192.168.1.0/24]"
    EXTERNAL_NET: "any"
    DNS_SERVERS: "[1.1.1.1]"
    SMTP_SERVERS: "$HOME_NET"
    HTTP_SERVERS: "$HOME_NET"
    SQL_SERVERS: "$HOME_NET"
    TELNET_SERVERS: "$HOME_NET"
    DNP3_SERVER: "$HOME_NET"
    DNP3_CLIENT: "$HOME_NET"
    MODBUS_SERVER: "$HOME_NET"
    MODBUS_CLIENT: "$HOME_NET"
    ENIP_SERVER: "$HOME_NET"
    ENIP_CLIENT: "$HOME_NET"
    FTP_SERVERS: "$HOME_NET"
    SSH_SERVERS: "$HOME_NET"

 # Holds the port group vars that would be passed in a Signature.
  port-groups:
    FTP_PORTS: "21"
    HTTP_PORTS: "80"
    ORACLE_PORTS: "1521"
    SSH_PORTS: "29100"
    SHELLCODE_PORTS: "!80"
    DNP3_PORTS: "20000"
    FILE_DATA_PORTS: "$HTTP_PORTS,110,143"   

# Runmode the engine should use.
runmode: workers

# If set to auto, the variable is internally switched to 'router' in IPS
# mode and 'sniffer-only' in IDS mode.
host-mode: auto

# Linux high speed capture support
af-packet:
  - interface: eth0
#    threads: 1
#    defrag: no
#    cluster-type: cluster_flow
#    cluster-id: 99
#    copy-mode: ips
#    copy-iface: br0
#    buffer-size: 3072
#    use-mmap: yes
#    tpacket-v2: yes
#    tpacket-v3: no
#    ring-size: 3072
  - interface: br0
#    threads: 1
#    cluster-id: 98
#    defrag: no
#    cluster-type: cluster_flow
#    copy-mode: ips
#    copy-iface: eth0
#    buffer-size: 3072
#    use-mmap: yes
#    tpacket-v2: yes
#    tpacket-v3: no
#    ring-size: 3072
#  - interface: wl0.2

# IPS Mode Configuration
# PCAP
pcap:
  - interface: eth0
    checksum-checks: auto
    promisc: yes
  - interface: br0
    checksum-checks: auto
    promisc: yes

pid-file: /opt/var/run/suricata.pid

legacy:
  uricontent: enabled

max-pending-packets: 1024

# The default logging directory.
default-log-dir: /opt/var/log/suricata

stats:
  enabled: no

 

[Mutzli]

my yaml is default as per installation. nothing edited yet. I have to learn how to fine tune. I have deinstall/reinstall, for now it work but I havent rebooted the router yet.

# Holds variables that would be used by the engine.
vars:

  # Holds the address group vars that would be passed in a Signature.
  address-groups:
    HOME_NET: "[192.168.1.0/24]"
    EXTERNAL_NET: "any"
    DNS_SERVERS: "[1.1.1.1]"
    SMTP_SERVERS: "$HOME_NET"
    HTTP_SERVERS: "$HOME_NET"
    SQL_SERVERS: "$HOME_NET"
    TELNET_SERVERS: "$HOME_NET"
    DNP3_SERVER: "$HOME_NET"
    DNP3_CLIENT: "$HOME_NET"
    MODBUS_SERVER: "$HOME_NET"
    MODBUS_CLIENT: "$HOME_NET"
    ENIP_SERVER: "$HOME_NET"
    ENIP_CLIENT: "$HOME_NET"
    FTP_SERVERS: "$HOME_NET"
    SSH_SERVERS: "$HOME_NET"

# Holds the port group vars that would be passed in a Signature.
  port-groups:
    FTP_PORTS: "21"
    HTTP_PORTS: "80"
    ORACLE_PORTS: "1521"
    SSH_PORTS: "29100"
    SHELLCODE_PORTS: "!80"
    DNP3_PORTS: "20000"
    FILE_DATA_PORTS: "$HTTP_PORTS,110,143" 

# Runmode the engine should use.
runmode: workers

# If set to auto, the variable is internally switched to 'router' in IPS
# mode and 'sniffer-only' in IDS mode.
host-mode: auto

# Linux high speed capture support
af-packet:
  - interface: eth0
#    threads: 1
#    defrag: no
#    cluster-type: cluster_flow
#    cluster-id: 99
#    copy-mode: ips
#    copy-iface: br0
#    buffer-size: 3072
#    use-mmap: yes
#    tpacket-v2: yes
#    tpacket-v3: no
#    ring-size: 3072
  - interface: br0
#    threads: 1
#    cluster-id: 98
#    defrag: no
#    cluster-type: cluster_flow
#    copy-mode: ips
#    copy-iface: eth0
#    buffer-size: 3072
#    use-mmap: yes
#    tpacket-v2: yes
#    tpacket-v3: no
#    ring-size: 3072
#  - interface: wl0.2

# IPS Mode Configuration
# PCAP
pcap:
  - interface: eth0
    checksum-checks: auto
    promisc: yes
  - interface: br0
    checksum-checks: auto
    promisc: yes

pid-file: /opt/var/run/suricata.pid

legacy:
  uricontent: enabled

max-pending-packets: 1024

# The default logging directory.
default-log-dir: /opt/var/log/suricata

stats:
  enabled: no

Check your DNS_SERVERS and SSH_PORTS if they are correct for your configuration. If you have unbound running your DNS should be your routers IP.

 

[Olivier L]

I run dnscrypt with NextDNS DOH service. I guess DNS_SERVERS should be 192.168.1.1 (LAN IP of my router).
Which SSH_PORTS I should enter ? I have one accessible from WAN located on a seedbox and the private one to access my router from LAN only.

 

[Mutzli]

I run dnscrypt with NextDNS DOH service. I guess DNS_SERVERS should be 192.168.1.1 (LAN IP of my router).
Which SSH_PORTS I should enter ? I have one accessible from WAN located on a seedbox and the private one to access my router from LAN only.

The routers SSH port or both separated by a coma.

SSH_PORTS: "22,29200"

 

[heysoundude]

The problem is lack of enough real RAM on AC86U. Whenever you swap, even with SSD, it's NOT fast enough to do "inline" IPS where the packet is intercepted as it come in and inspected against various rule then "copy" back out. If the system needs to rely on "swapping", the effects of this slowness will manifest as packet loss or "lag" in the connection. You want to do all these copy in ->inspect->copy out all in RAM where it's faster than you can notice and not drop packets.

So with 512MB on AC86U (and more than 1/2 of that is reserved for other critical system services), the constant swapping in/out to disk (SSD or HDD) will lead to packet loss and lag. So virtual memory via SWAP isn't ideal or option for Suricata (fun fact, if you disabled swap, Suricata won't even start and will tell you that you lack memory run it).

This has been filtering in the back of my brain for a while - it's still early days in this 3rd decade of the 21st century, and even if the world doesn't seem to have much work happening, progress grinds forward incessantly albeit incrementally:
OK, so there's not enough RAM in an AC86 - it might come closer to being less of a problem with the AX86 (1GB, like the AX88) or disappear substantially if whatever follows that model with wifi6e has 2GB+

Processors invariably get faster too, and now with Apple committed to ARM processors, that will change how a lot of tech (and code) around those processors gets made to work

What if Suricata gets streamlined and rolled into unbound rather than its Firewall? They're almost made for each other...why not start work now to marry them off? Call me Captain Obvious if I missed this until now

 

[faux123]

Another totally off-topic update:

I've run Snort classic (2.x series) which is single threaded in IPS using af_packet no issues at all with random drops (TCP connection issues), then I managed to run Snort3 (multi-threaded) alpha version again in IPS mode using af_packet, no issues again with random drops, so it means the Linux 4.1.27 kernel (and my hybrid kernel) have no issues at the driver level.

So based on the 2 limited evidence, it seems random packet drops with af_packet is with Suricata 4.1.8 (I haven't tested the newer 5.xx series since it needs RUST and at this moment there are no RUST ports on Entware). I don't know if series 4.1.x is still supported by Suricata but unless they fix their af_packet implementations, af_packet inline IPS mode is currently broken...

 

[XIII]

Maybe it’s then “better” to start using snort instead of Suricata?

(in particular on the AC86U with less memory?)

 

[faux123]

Maybe it’s then “better” to start using snort instead of Suricata?

(in particular on the AC86U with less memory?)

Maybe... Snort3 is a memory hog as well by default (pun intended), serious tweaks had to be made to get it to run on AC86U.

With it running in multi-threaded mode (currently 2 threads total 1, thread per core), the system load is averaging around ~4.60 and sometimes as high as in the 9s (without anything running, my avg system load is around 2.96). I also have an external USB fan blasting at back of my AC86U all the time to keep it cool due to the high loads introduced by IPS.

From a speed perspective, it cuts my download speed by -17% and upload speed by -30%... (original 180/230 to 150/160) A steep price to pay for some people.

So it sorta works as a router based IPS if you have fan to keep it cool (Suricata had issues with load as well, for me it was around 4.50ish avg and spiked to 9s as well when it ran out of memory and started swapping like mad) and willing to trade some loss in speed for an additional (marginal?) IPS protection...

IMHO, AC86U with only dual core, 512 MB of real RAM is really borderline to have a full fledged IPS running on it (AiProtection is a super light weight IPS? compare to Snort3 or Suricata, and it's designed like that for a reason due to low processing power and low RAM of routers but AiProtection's effectiveness is questionable at best). Maybe the newer AX88U with quadcore and 1GB of real RAM can handle this better???

 

[XIII]

I have an AC86U. Maybe I should go back to AiProtect...