[Release] FreshJR Adaptive QOS (Improvements / Custom Rules / and Inner workings)

[Miiidnight]

I've just set up the fast version of the script on my AC68U running merlin 384.4_2. Seems to work very well but I'm having trouble using the custom iptables upload rules.

I've added this to the bottom of the ##UPLOAD (OUTGOING TRAFFIC) CUSTOM RULES START HERE section of the script, .

iptables -D POSTROUTING -t mangle -o eth0 -m mac --mac-source AA:BB:CC:DD:EE:FF -j MARK --set-mark ${Downloads_mark} &> /dev/null                    #MINE
iptables -A POSTROUTING -t mangle -o eth0 -m mac --mac-source AA:BB:CC:DD:EE:FF -j MARK --set-mark ${Downloads_mark}                                 #MINE

If I look at the system log in the router admin page, I can see this line multiple times:

Jul  6 10:20:34 adaptive QOS: iptables v1.4.15: unknown option "--set-mark"
Jul  6 10:20:34 adaptive QOS: Try `iptables -h' or 'iptables --help' for more information.

Am I doing something silly here? The custom download rules set by realtc seem to be working fine and I get the same results on 385.4.

 

[FreshJR]

Am I doing something silly here? The custom download rules set by realtc seem to be working fine and I get the same results on 385.4.

Put your iptable rules in the iptables section.

Your rule is currently not working since you currently have it in upload section. That upload section is only meant for TC rules but I can see how the current naming is ambiguous.

The correct location will look something like this in the fast version of the script.

    iptables_up_rules() {
        set_iptables_variables
        echo "Applying  Iptables Rules"
        ##UPLOAD (OUTGOING TRAFFIC) CONTINUED HERE
            #iptables -D POSTROUTING -t mangle -o eth0 -s 192.168.2.100/30 -m mark --mark 0x40000000/0x4000ffff -j MARK --set-mark ${Gaming_mark} &> /dev/null    #Gaming (3/4) - Routes Unidentified Traffic into "Gaming", instead of "Others", from specified LAN devices in rule (line 1/2)
            #iptables -A POSTROUTING -t mangle -o eth0 -s 192.168.2.100/30 -m mark --mark 0x40000000/0x4000ffff -j MARK --set-mark ${Gaming_mark}                #Gaming (4/4) - Routes Unidentified Traffic into "Gaming", instead of "Others", from specified LAN devices in rule (line 2/2)
        ##UPLOAD (OUTGOING TRAFFIC) CUSTOM RULES END HERE
 
    }

Perhaps in future revisions I will rename the catagories to

-custom TC down rules
-custom TC up rules
-iptables up rules

This should make the correct location will be slightly clearer.

 

[FadgewackeR]

Good day @FreshJR ... I promise I'll disappear into the woodwork soon.

I noticed in your tutorial that there are special rules for router as a VPN client here: https://www.snbforums.com/threads/r...-and-inner-workings.36836/page-78#post-412034

In my situation, I have a NAS that handles the VPN well, and really, the NAS activity is the only thing that I am particularly bothered about running through VPN, so as things stand, I would not be adding a requirement for VPN at the gateway when I set up the 86U later. It works just fine on my current DSL-N55U with no requirement for modification at the router.

Will the VPN traffic in my scenario be whitelisted and therefore subject to the rule requiring Cleint mods to the VPN'd intenral IP, or is that only if you want to use the router's capability to VPN traffic (through a paid connection) to certain devices on the LAN?

My knowledge base on the routers capability (or lack of) to tunnel to designated LAN devices is zero, as it's not something that I've felt the need to investigate so far.

 

[FreshJR]

Your question is not clear.

1) If the NAS is directly connecting to a paid VPN server via its own means, then the rules are NOT necessary.

2) If your NAS is connecting to your router as a regular/typical device but you then have the router pushing all your NAS traffic through a paid VPN server (eg. the router is acting as a VPN client on behalf of the NAS), then the VPN rules are necessary.

3) If your NAS is acting as a VPN server to make your local network accessible via the internet then no rules are necessary.

4) If your router is acting as a VPN server to make your local network accessible via the internet, then a QOS fix is necessary but does not exist.

—

The difference between a VPN server and VPN client was already explained in a link at the beginning of the thread, 3rd post.

 

[FadgewackeR]

Your question is not clear.

I'm quite good at that...

1) If the NAS is directly connecting to a paid VPN server via its own means, then the rules are NOT necessary.

This is my situation, as I understand it. My router provides a connection to the NAS. The NAS has it's own VPN functionality, and the paid for VPN connection is in effect on the QNAP NAS only, at this point. I can connect my other divices to the PIA VPN service, at the device (phones, laptop) yet I haven't felt the need to do so.
All inbound and outbound network activity from the NAS is masked as a different Ip to all other network activity from every other device that my router handles. The router does nothing intelligent in this arrangement, it's all done on the NAS.

The difference between a VPN server and VPN client was already explained in linking from the beginning of the thread.

I read that all before asking my question, I feel that I understand the difference, yet to my mind, I was still uncertain as to exactly how the router level code (which I have yet to install or investigate in detail), would handle the traffic that my NAS introduces via its VPN application. So I asked the question rather than breaking something and asking later under duress.

Thanks again for the swift response.

 

[FreshJR]

I was still uncertain as to exactly how the router level code would handle the traffic that my NAS introduces via its VPN application.

The VPN client on the NAS will be encrypting traffic in both directions.

Your router will only see encrypted traffic which will not be indentifiable by it or any other stop between the NAS and the VPN Server itself.

I would create a custom upload/download rule for the NAS itself via its local ip and stick all it’s traffic into file downloads.

Encrypted traffic with my script either goes to “Others” or “Web Surfing”.
**Encrypted traffic without my script either goes to "Net Control" or "Defaults"**

I do not recommend either of those categories for sortable bulk traffic.

 

[FadgewackeR]

The VPN client on the NAS will be encrypting traffic in both directions.

Your router will only see encrypted traffic which will not be indentifiable by it or any other stop between the NAS and the VPN Server itself.

I would create a custom upload/download rule for the NAS itself via its local ip and stick all it’s traffic into file downloads.

Encrypted traffic out of the box either goes to “Others” or “Web Surfing”.

I do not recommend either of those categories for sortable bulk traffic.

Perfect, that's exactly what I needed to know. Cheers

 

[Vexira]

Im getting the feeling that these rules don't work any more i used to see an message when before when you first gave them to me as in it would say the bandwidth values were getting modified again, im using Default message log level = Debug & Log only messages more urgent than = all, i vaguely remember it used to say something about changing to user defined percentages that seems to be gone now, giving me the impression it no longer works.

Rules:
DownCeil="$(expr ${DownCeil} \* 95 / 100) UpCeil="$(expr ${UpCeil} \* 95 / 100)

From my log:
8 19:34:40 rc_service: httpd 897:notify_rc start_sig_check
Jul 8 19:34:42 adaptive QOS: Clearing qos environment
Jul 8 19:34:44 adaptive QOS: Clearing qos environment
Jul 8 19:34:58 adaptive QOS: Clearing qos environment
Jul 8 19:34:58 adaptive QOS: Clearing qos environment
Jul 8 19:35:00 adaptive QOS: Changing 1:10 class rate
Jul 8 19:35:02 adaptive QOS: Changing 1:11 class rate
Jul 8 19:35:02 kernel: HTB: quantum of class 10011 is big. Consider r2q change.
Jul 8 19:35:03 adaptive QOS: Changing 1:12 class rate
Jul 8 19:35:04 adaptive QOS: Changing 1:13 class rate
Jul 8 19:35:06 adaptive QOS: Changing 1:14 class rate
Jul 8 19:35:08 adaptive QOS: Changing 1:15 class rate
Jul 8 19:35:09 kernel: HTB: quantum of class 10015 is big. Consider r2q change.
Jul 8 19:35:09 rc_service: httpd 897:notify_rc restart_qos;restart_firewall
Jul 8 19:35:09 rc_service: waitting "start_sig_check" via httpd ...
Jul 8 19:35:10 adaptive QOS: Changing 1:16 class rate
Jul 8 19:35:10 adaptive QOS: Applying ------ Up Rules
Jul 8 19:35:11 adaptive QOS: Changing 1:17 class rate
Jul 8 19:35:11 adaptive QOS: Applying ---- Down Rules
Jul 8 19:35:24 rc_service: skip the event: restart_qos;restart_firewall.
Jul 8 19:35:29 adaptive QOS: *- Clearing Iptables -*
Jul 8 19:42:48 syslogd started: BusyBox v1.25.1

 

[Sinner]

Its working. You have alot pf log action. Near the bottom there the lines starting with applying ----- arw the new msgs now. Looks to wme like the acript is running at least. As for working well i just hope so

 

[Vexira]

Its working. You have alot pf log action. Near the bottom there the lines starting with applying ----- arw the new msgs now. Looks to wme like the acript is running at least. As for working well i just hope so

The script is working its just the two rules aren't.

 

[Sinner]

Well it might be a good time to comment out those rules and see what trend is doing with it now. Ive had trends database updated todo do what i was doing manually twice now and was able to remove my rules

 

[Vexira]

Well it might be a good time to comment out those rules and see what trend is doing with it now. Ive had trends database updated todo do what i was doing manually twice now and was able to remove my rules

They were given to me by fresh I'm just wondering if there is an updated version of them it sets the bandwith to 95% in the script saves me setting that manually since I have non whole number results form speedtest.net.

 

[Sinner]

They were given to me by fresh I'm just wondering if there is an updated version of them it sets the bandwith to 95% in the script saves me setting that manually since I have non whole number results form speedtest.net.

you mean the upceil/downceil line?

 

[Vexira]

you mean the upceil/downceil line?

Yes

 

[Sinner]

Yes

well that appears to just reduce the figures your entering for ul/dl on the qos setup page. Ill be honest with you... It's summertime and kids are home now. Ive notice massive fluctuations in my own at home here.. I have 10/2 wireless cellular internet here.. normally9/2 all day long. it peacks at 11/4 actually with bloat. latly everyone 5min its been dropping to like 1.5/0.5 for a sec or two causing lag spikes and hovering at the 3-4 range so ive had to reduce my setting to 3/1 in qos to be be stable most of the time. you may need todo the same because the second the internet in your area gets burdened and the performance drops anything below your setting qos blows up and cant work at all.

 

[Sinner]

Yes

or modify it to 90 / 100 see if that help instead of 95 / 100

 

[skeal]

[Resolved] I'm testing the alternative interface version. I have almost no problems now I have it configured. The problem is that my ovpn traffic specifically video upload traffic is missing. My video traffic shows the download but no video upload overhead. Nothing I do seems to get the upload overhead working. The stats page below is while a 4 Mb/s video stream is running as you can see no upload video traffic while the stream is running. My added rules are these:

iptables -D POSTROUTING -t mangle -o br0 -p tcp --dport 1912 -j MARK --set-mark ${Streaming_mark_down} &> /dev/null
iptables -A POSTROUTING -t mangle -o br0 -p tcp --dport 1912 -j MARK --set-mark ${Streaming_mark_down}

iptables -D POSTROUTING -t mangle -o br0 -d 192.168.14.66/31 -j MARK --set-mark ${Streaming_mark_down} &> /dev/null
iptables -A POSTROUTING -t mangle -o br0 -d 192.168.14.66/31 -j MARK --set-mark ${Streaming_mark_down}

iptables -D POSTROUTING -t mangle -o br0 -d 192.168.14.68/32 -j MARK --set-mark ${Streaming_mark_down} &> /dev/null
iptables -A POSTROUTING -t mangle -o br0 -d 192.168.14.68/32 -j MARK --set-mark ${Streaming_mark_down}

iptables -D POSTROUTING -t mangle -o $wan -s 192.168.14.66/31 -j MARK --set-mark ${Streaming_mark_up} &> /dev/null
iptables -A POSTROUTING -t mangle -o $wan -s 192.168.14.66/31 -j MARK --set-mark ${Streaming_mark_up}

iptables -D POSTROUTING -t mangle -o $wan -s 192.168.14.68/32 -j MARK --set-mark ${Streaming_mark_up} &> /dev/null
iptables -A POSTROUTING -t mangle -o $wan -s 192.168.14.68/32 -j MARK --set-mark ${Streaming_mark_up}

The stats page looks like this while stream is running, any help would be appreciated your script is awesome. Thank you in advance.

RESOLVED fixed it myself by rereading the instructions over and over again.

 

[FadgewackeR]

I really didn't want to ask, but, CIDR... My mechanical mind is confused slightly. Think that bar this, my script mod is good to go.

My subnet (as set on my router) is 255,255,255,0
I want to assign rules around 192.168.1.120
This gives me /24.

Do I just append my IP addresses with the /24, or is there more to it?

 

[skeal]

I really didn't want to ask, but, CIDR... My mechanical mind is confused slightly. Think that bar this, my script mod is good to go.

My subnet (as set on my router) is 255,255,255,0
I want to assign rules around 192.168.1.120
This gives me /24.

Do I just append my IP addresses with the /24, or is there more to it?

If you want to create rules that are meant to effect 192.168.1.120 I would show it as 192.168.1.120/32 this singles out that ip by itself. If you were to use 192.168.1.120/31 you would have the use of 2 ips 192.168.1.120 and 192.168.1.121 you can google "ip to cidr" and find convert tools.

 

[FadgewackeR]

If you want to create rules that are meant to effect 192.168.1.120 I would show it as 192.168.1.120/32 this singles out that ip by itself. If you were to use 192.168.1.120/31 you would have the use of 2 ips 192.168.1.120 and 192.168.1.121 you can google "ip to cidr" and find convert tools.

Got you. So If already have 3 ip addresses within my router, and they are 192.168.1.120, 60 and 211, I just append them all with /32 to specify the single addresses?

Alternatively, I create a range that is /30, and re-point my devices and router to those 3 sequential IPs?