Beta VPN Director testing

[RMerlin]

386.3 will introduce a complete overhaul of the OpenVPN routing handling, serving as the platform on top of which is implemented the new VPN Director.

Backend changes:
Starting with 386.3, all OpenVPN routing will now be directly configured by the firmware itself, rather than having the OpenVPN client create routes, and having the firmware modify them afterward. The biggest benefit of this method is that routing handling will be cleaner, no more need to remove unwanted routes that were pushed by the server. The vpnrouting.sh script is gone, the routing handlers are now implemented within libovpn, making it more robest and more flexible.

The new implementation allows the "Redirect Internet Traffic" option to behave more accurately, as it will no longer be silently overriden by the remote server. If you set it to "No" and that remote server tries to push itself as the new default gateway, that route will not be configured.

With the new implementation there is no longer a separate Policy Rule and Policy Rule (Strict) method - just one single Policy Rule.

Killswitch behaviour has also changed a bit, now when you manually stop a client by turning it off on the webui (or by manually running a "service stop_vpnclientX" command over SSH), the killswitch will not be activated. The killswitch is now activated in these scenarios:

- If a VPN client is set to connect at boot time AND the killswitch is enabled, then the killswitch is applied at boot time until that client connects
- If a VPN client is abnormally terminated (for example if you kill the openvpn process), or another connectivity issue that causes the down-handler to be executed, but NOT the regular "stop_client" code in the firmware.

Another benefit of the new implementation is that the killswitch can also be used in "Redirect All" mode now, not just in policy mode.


This allows me to introduce the new feature that sits on top of this: VPN Director.

VPN Director replaces the original policy-based routing implementation. It's a bit closer in design to Asus' VPN Fusion, meaning that all rules are now configured in a central location. However it's still only limited to OpenVPN - PPTP and L2TP clients are not supported.

This feature is accessed through a new VPN Director page, in the VPN section. There, you can create rules (up to 199 rules total), and assign which VPN tunnel should be accessed by it (or WAN if you want to override it). That page will also show you a summary of your current OpenVPN setup, and allow you to start/stop clients directly from it.

Notes about VPN Director:

• For VPN rules related to OpenVPN instances to be applied, that instance must be set to Policy mode. (I might eventually rename that to "VPN Director" to make it clearer?)
• It more clearly shows how each rules interact with one another. Highest priority will be any client set to redirect all traffic, followed by WAN rules, and then followed by OpenVPN rules, with OpenVPN 1 having the highest priority, and OpenVPN 5 the lowest (last) priority. Also note that Dual WAN routes have a higher priority than all of this.
• Rules are now stored in JFFS rather than nvram, allowing us to store more rules, and also save a good bit of nvram even when rules weren't used. There should now be enough space to store 199 rules total (unless you use insanely long descriptions for each rules), while the previous implementation would often run out of space before hitting the established limit of 100 per client.
• Rules can now be disabled/enabled through the web interface, without the need of removing/adding them. You can just click on the ckeckmark to disable it (but don't forget to Apply your changes afterward!)
• Just like before, local/remote IPs can be entered in CIDR format if you want to specify a range.
• Rules can be updated without needing to restart the VPN client. Once you click Apply, rules are applied to the RPDB tables.
• Existing rules will be imported into VPN Director on first boot. However if you revert back to a previous firmware release, your rules will be gone.

Changes in custom settings storage
Another noteworthy change in 386.3 is that the custom config entries for OpenVPN clients and servers are now stored in JFFS instead of base64-encoded nvram. This allows to greatly increase the maximum amount of config lines, from ~380 characters to close to 3999 characters now.


IMPORTANT:
Make a backup of both your settings AND JFFS partition before flashing 386.3! First, because first boot will convert existing rules into VPN Director rules, as well as migrating the custom configuration entries. If you go back, you will need to either re-enter your rules and custom VPN configs, or restore a backup of your settings. Also, in case something goes wrong during the first boot import process, which may require a factory default reset (it happened to me during development, and while I fixed the bug that caused this, there's always a chance that there is another bug still in there - that's what testing is for).

Test builds are available in the VPN Director folder, at https://www.asuswrt-merlin.net/test-builds/

Things to test:

• Are existing VPN rules properly imported in?
• Same with any custom config you may have with either OpenVPN server or client instances, are they still showing up properly in the Custom Settings area?
• Pretty much related to OpenVPN client routing, either set to All or using rules
• Killswitch behaviour, both on unexpected disconnects or on reboots
• Rule management on the VPN Director page
• Is everything properly applied if you change then apply rules?
• Client status report, stopping/starting clients from the VPN Director page
• Each client page will show rules that are related to itself (plus WAN rules since they also apply). Is that list shown correctly?

Thanks everyone!

 

[RMerlin]

Known issues:

• DNS Exclusive mode doesn't work (Code wasn't updated to VPN Director, fixed)
• Routing fails to work with providers that push a hostname as gateway address instead of an IP (fixed)
• VPN routing failing if DNS Mode is set to "Disable" (up-handler was skipping route configuration, fixed)
• After making changes to rules, Exclusive DNS rules aren't refreshed (fixed)

 

[SomeWhereOverTheRainBow]

386.3 will introduce a complete overhaul of the OpenVPN routing handling, serving as the platform on top of which is implemented the new VPN Director.

Backend changes:
Starting with 386.3, all OpenVPN routing will now be directly configured by the firmware itself, rather than having the OpenVPN client create routes, and having the firmware modify them afterward. The biggest benefit of this method is that routing handling will be cleaner, no more need to remove unwanted routes that were pushed by the server. The vpnrouting.sh script is gone, the routing handlers are now implemented within libovpn, making it more robest and more flexible.

The new implementation allows the "Redirect Internet Traffic" option to behave more accurately, as it will no longer be silently overriden by the remote server. If you set it to "No" and that remote server tries to push itself as the new default gateway, that route will not be configured.

With the new implementation there is no longer a separate Policy Rule and Policy Rule (Strict) method - just one single Policy Rule.

Killswitch behaviour has also changed a bit, now when you manually stop a client by turning it off on the webui (or by manually running a "service stop_vpnclientX" command over SSH), the killswitch will not be activated. The killswitch is now activated in these scenarios:

- If a VPN client is set to connect at boot time AND the killswitch is enabled, then the killswitch is applied at boot time until that client connects
- If a VPN client is abnormally terminated (for example if you kill the openvpn process), or another connectivity issue that causes the down-handler to be executed, but NOT the regular "stop_client" code in the firmware.

Another benefit of the new implementation is that the killswitch can also be used in "Redirect All" mode now, not just in policy mode.


This allows me to introduce the new feature that sits on top of this: VPN Director.

View attachment 34438

VPN Director replaces the original policy-based routing implementation. It's a bit closer in design to Asus' VPN Fusion, meaning that all rules are now configured in a central location. However it's still only limited to OpenVPN - PPTP and L2TP clients are not supported.

This feature is accessed through a new VPN Director page, in the VPN section. There, you can create rules (up to 199 rules total), and assign which VPN tunnel should be accessed by it (or WAN if you want to override it). That page will also show you a summary of your current OpenVPN setup, and allow you to start/stop clients directly from it.

Notes about VPN Director:

• For VPN rules related to OpenVPN instances to be applied, that instance must be set to Policy mode. (I might eventually rename that to "VPN Director" to make it clearer?)
• It more clearly shows how each rules interact with one another. Highest priority will be any client set to redirect all traffic, followed by WAN rules, and then followed by OpenVPN rules, with OpenVPN 1 having the highest priority, and OpenVPN 5 the lowest (last) priority. Also note that Dual WAN routes have a higher priority than all of this.
• Rules are now stored in JFFS rather than nvram, allowing us to store more rules, and also save a good bit of nvram even when rules weren't used. There should now be enough space to store 199 rules total (unless you use insanely long descriptions for each rules), while the previous implementation would often run out of space before hitting the established limit of 100 per client.
• Rules can now be disabled/enabled through the web interface, without the need of removing/adding them. You can just click on the ckeckmark to disable it (but don't forget to Apply your changes afterward!)
• Just like before, local/remote IPs can be entered in CIDR format if you want to specify a range.
• Rules can be updated without needing to restart the VPN client. Once you click Apply, rules are applied to the RPDB tables.
• Existing rules will be imported into VPN Director on first boot. However if you revert back to a previous firmware release, your rules will be gone.

Changes in custom settings storage
Another noteworthy change in 386.3 is that the custom config entries for OpenVPN clients and servers are now stored in JFFS instead of base64-encoded nvram. This allows to greatly increase the maximum amount of config lines, from ~380 characters to close to 3999 characters now.


IMPORTANT:
Make a backup of both your settings AND JFFS partition before flashing 386.3! First, because first boot will convert existing rules into VPN Director rules, as well as migrating the custom configuration entries. If you go back, you will need to either re-enter your rules and custom VPN configs, or restore a backup of your settings. Also, in case something goes wrong during the first boot import process, which may require a factory default reset (it happened to me during development, and while I fixed the bug that caused this, there's always a chance that there is another bug still in there - that's what testing is for).

Test builds are available in the VPN Director folder, at https://www.asuswrt-merlin.net/test-builds/

Things to test:

• Are existing VPN rules properly imported in?
• Same with any custom config you may have with either OpenVPN server or client instances, are they still showing up properly in the Custom Settings area?
• Pretty much related to OpenVPN client routing, either set to All or using rules
• Killswitch behaviour, both on unexpected disconnects or on reboots
• Rule management on the VPN Director page
• Is everything properly applied if you change then apply rules?
• Client status report, stopping/starting clients from the VPN Director page
• Each client page will show rules that are related to itself (plus WAN rules since they also apply). Is that list shown correctly?

Thanks everyone!

Where did you find the space in the memory to come up with such a beast!?!.

 

[nzwayne]

All's good on this new drop tonight... 386.3_alpha2-g420d8061c5 ... On my RT-AX88U, AX86U, AC86U, AX56U. Now to check out the VPN aspects.

 

[Gar]

If I don't use VPN is there any benefit to using this over the current release? Other than to test?

 

[eibgrad]

Interesting. Many issues I've been concerned about in the past will now be corrected.

Does the lack of the Strict option for Routing Policy mean there is no strict mode? Or does it mean strict mode is always assumed?

 

[SomeWhereOverTheRainBow]

If I don't use VPN is there any benefit to using this over the current release? Other than to test?

I will be sticking with the current one for a little bit, only because I am not ready to migrate my current vpn setup from the policy based routing.

 

[SomeWhereOverTheRainBow]

386.3 will introduce a complete overhaul of the OpenVPN routing handling, serving as the platform on top of which is implemented the new VPN Director.

Backend changes:
Starting with 386.3, all OpenVPN routing will now be directly configured by the firmware itself, rather than having the OpenVPN client create routes, and having the firmware modify them afterward. The biggest benefit of this method is that routing handling will be cleaner, no more need to remove unwanted routes that were pushed by the server. The vpnrouting.sh script is gone, the routing handlers are now implemented within libovpn, making it more robest and more flexible.

The new implementation allows the "Redirect Internet Traffic" option to behave more accurately, as it will no longer be silently overriden by the remote server. If you set it to "No" and that remote server tries to push itself as the new default gateway, that route will not be configured.

With the new implementation there is no longer a separate Policy Rule and Policy Rule (Strict) method - just one single Policy Rule.

Killswitch behaviour has also changed a bit, now when you manually stop a client by turning it off on the webui (or by manually running a "service stop_vpnclientX" command over SSH), the killswitch will not be activated. The killswitch is now activated in these scenarios:

- If a VPN client is set to connect at boot time AND the killswitch is enabled, then the killswitch is applied at boot time until that client connects
- If a VPN client is abnormally terminated (for example if you kill the openvpn process), or another connectivity issue that causes the down-handler to be executed, but NOT the regular "stop_client" code in the firmware.

Another benefit of the new implementation is that the killswitch can also be used in "Redirect All" mode now, not just in policy mode.


This allows me to introduce the new feature that sits on top of this: VPN Director.

View attachment 34438

VPN Director replaces the original policy-based routing implementation. It's a bit closer in design to Asus' VPN Fusion, meaning that all rules are now configured in a central location. However it's still only limited to OpenVPN - PPTP and L2TP clients are not supported.

This feature is accessed through a new VPN Director page, in the VPN section. There, you can create rules (up to 199 rules total), and assign which VPN tunnel should be accessed by it (or WAN if you want to override it). That page will also show you a summary of your current OpenVPN setup, and allow you to start/stop clients directly from it.

Notes about VPN Director:

• For VPN rules related to OpenVPN instances to be applied, that instance must be set to Policy mode. (I might eventually rename that to "VPN Director" to make it clearer?)
• It more clearly shows how each rules interact with one another. Highest priority will be any client set to redirect all traffic, followed by WAN rules, and then followed by OpenVPN rules, with OpenVPN 1 having the highest priority, and OpenVPN 5 the lowest (last) priority. Also note that Dual WAN routes have a higher priority than all of this.
• Rules are now stored in JFFS rather than nvram, allowing us to store more rules, and also save a good bit of nvram even when rules weren't used. There should now be enough space to store 199 rules total (unless you use insanely long descriptions for each rules), while the previous implementation would often run out of space before hitting the established limit of 100 per client.
• Rules can now be disabled/enabled through the web interface, without the need of removing/adding them. You can just click on the ckeckmark to disable it (but don't forget to Apply your changes afterward!)
• Just like before, local/remote IPs can be entered in CIDR format if you want to specify a range.
• Rules can be updated without needing to restart the VPN client. Once you click Apply, rules are applied to the RPDB tables.
• Existing rules will be imported into VPN Director on first boot. However if you revert back to a previous firmware release, your rules will be gone.

Changes in custom settings storage
Another noteworthy change in 386.3 is that the custom config entries for OpenVPN clients and servers are now stored in JFFS instead of base64-encoded nvram. This allows to greatly increase the maximum amount of config lines, from ~380 characters to close to 3999 characters now.


IMPORTANT:
Make a backup of both your settings AND JFFS partition before flashing 386.3! First, because first boot will convert existing rules into VPN Director rules, as well as migrating the custom configuration entries. If you go back, you will need to either re-enter your rules and custom VPN configs, or restore a backup of your settings. Also, in case something goes wrong during the first boot import process, which may require a factory default reset (it happened to me during development, and while I fixed the bug that caused this, there's always a chance that there is another bug still in there - that's what testing is for).

Test builds are available in the VPN Director folder, at https://www.asuswrt-merlin.net/test-builds/

Things to test:

• Are existing VPN rules properly imported in?
• Same with any custom config you may have with either OpenVPN server or client instances, are they still showing up properly in the Custom Settings area?
• Pretty much related to OpenVPN client routing, either set to All or using rules
• Killswitch behaviour, both on unexpected disconnects or on reboots
• Rule management on the VPN Director page
• Is everything properly applied if you change then apply rules?
• Client status report, stopping/starting clients from the VPN Director page
• Each client page will show rules that are related to itself (plus WAN rules since they also apply). Is that list shown correctly?

Thanks everyone!

The one toss-up I can see is that on each vpn client 1-5, you are able to define up to 100 rules per client. With the central approach you are limited to 199 and vpn 1 rules get preference over the next one. The central management is a nice thought, but who would sacrifice individual management for the sake of this one screen approach where vpns fight for priority on rules and the amount of rules allowed for a single vpn client have become limited. I can see this being a good approach for running more that 2 or 3 vpns, but when do all five vpns ever get used at once. I would rather lose 2 vpn clients for the sake of 3, than run a central management screen for 5 that I never run all at once.

 

[RMerlin]

If I don't use VPN is there any benefit to using this over the current release? Other than to test?

I am only gathering feedback on the VPN-related changes at this time.

Does the lack of the Strict option for Routing Policy mean there is no strict mode? Or does it mean strict mode is always assumed?

Strict mode was related to the route copy - it determined whether only routes with the tunnel as target was getting copied. Now that I no longer copy any routes to the VPN tables, there is no need for two separate mode. The VPN table already contains only routes related to the VPN itself.

With the central approach you are limited to 199

If you need more than 199 rules, then you need to rethink your setup. I highly doubt anyone would be willing to manually configure more than 199 rules, one by one. And the previous limit of 100 was mostly theorical, as you would run out of nvram space before reaching that limit.

nd vpn 1 rules get preference over the next one.

That was always the case, as VPN1 used priorities of 10000 through 10099, VPN2 used 10100 through 10199, etc... Just that now it's more clearly defined, and more intuitive as you can see all rules at the same time. No more guessing as to which other VPN client may potentially override the client you are currently configuring.

 

[SomeWhereOverTheRainBow]

I am only gathering feedback on the VPN-related changes at this time.


Strict mode was related to the route copy - it determined whether only routes with the tunnel as target was getting copied. Now that I no longer copy any routes to the VPN tables, there is no need for two separate mode. The VPN table already contains only routes related to the VPN itself.


If you need more than 199 rules, then you need to rethink your setup. I highly doubt anyone would be willing to manually configure more than 199 rules, one by one. And the previous limit of 100 was mostly theorical, as you would run out of nvram space before reaching that limit.


That was always the case, as VPN1 used priorities of 10000 through 10099, VPN2 used 10100 through 10199, etc... Just that now it's more clearly defined, and more intuitive as you can see all rules at the same time. No more guessing as to which other VPN client may potentially override the client you are currently configuring.

Thank you for taking the time to early on divulge of any concerns through discourse. That was my goal was to weed out some of the potential early at first glance "thoughts". As always, you have gracefully answered all concerns. Thank you @RMerlin

 

[eibgrad]

Strict mode was related to the route copy - it determined whether only routes with the tunnel as target was getting copied. Now that I no longer copy any routes to the VPN tables, there is no need for two separate mode. The VPN table already contains only routes related to the VPN itself.

Hmm, not sure I follow this. Sounds to me as if the effect is the same as if the user was specifying Strict. That is, the VPN table will NOT contain any routes that point to anything other than the VPN (e.g., static routes bound to the WAN, which is what Strict strips out). IOW, it's effectively Strict mode all the time, thus no need to make the distinction anymore in the GUI.

 

[RMerlin]

Hmm, not sure I follow this. Sounds to me as if the effect is the same as if the user was specifying Strict. That is, the VPN table will NOT contain any routes that point to anything other than the VPN (e.g., static routes bound to the WAN, which is what Strict strips out). IOW, it's effectively Strict mode all the time, thus no need to make the distinction anymore in the GUI.

The final result should be mostly the same, yes.

 

[Jack Yaz]

How are policy routes stored in the backend please? are they still nvram or jffs now too?

 

[Martineau]

Are Dual-WAN interfaces supported?

 

[SomeWhereOverTheRainBow]

Are Dual-WAN interfaces supported?

doubt it. dual wan has always been in a league of its own, unless @RMerlin has found the time to figure out that.

 

[SomeWhereOverTheRainBow]

How are policy routes stored in the backend please? are they still nvram or jffs now too?

i assume the 199 rules are still store in nvram. before it was 100 rules per client... was that nvram or jffs?

 

[kernol]

As a noob on VPN Clients [no prior settings used on my AX86U] I played with 2 VPN clients from scratch.

This may be by design - but until I selected "Redirect Internet traffic through tunnel" to "Policy Rules" I could not get either client VPN to work. In other words if I selected "Yes" to that setting [before creating any VPN Director Rules] - the VPN client did not work ... and some other weird issues like taking ages to go from Tab to Tab on the webui took place.

Once I created a Rule [or three] and selected "Policy Rules" everything worked as I expected.

 

[Martineau]

doubt it. dual wan has always been in a league of its own, unless @RMerlin has found the time to figure out that.

[AC87U] Firmware 384.4, VPN rules not working

Hello, Since I updated my AC87Uto 384.4, my VPN rules not working. On my device when I check on myip.org, I have the FAI IP and not VPN IP. What I can do? Thanks,

www.snbforums.com

Basically we are using the RPDB rules, and in the legacy vpn_routing.sh script, @RMerlin's Selective VPN routing rules were unfortunately implemented at a lower RPDB priority than ASUS' Dual-WAN rules, and my vpn_routing.sh patch simply changed the RPDB PRIO to ensure that the VPN Selective routing took priority.

 

[SomeWhereOverTheRainBow]

Basically we are using the RPDB rules, and in the legacy vpn_routing.sh script, @RMerlins Selective VPN routing rules were unfortunately implemented at a lower RPDB priority than ASUS' Dual-WAN rules, and my vpn_routing.sh patch simply changed the RPDB PRIO to ensure that the VPN Selective routing took priority.

I hope he implemented your methods as a hotfix, if he hasn't already resolved that issue. After poking around in his most recent commits, at first glance I don't see anything that solely talks about Dual-WAN, but maybe @RMerlin likes for users to guess....

 

[SomeWhereOverTheRainBow]

As a noob on VPN Clients [no prior settings used on my AX86U] I played with 2 VPN clients from scratch.

This may be by design - but until I selected "Redirect Internet traffic through tunnel" to "Policy Rules" I could not get either client VPN to work. In other words if I selected "Yes" to that setting [before creating any VPN Director Rules] - the VPN client did not work ... and some other weird issues like taking ages to go from Tab to Tab on the webui took place.

Once I created a Rule [or three] and selected "Policy Rules" everything worked as I expected.

the rules must not be created unless you specify an initial rule; thus preventing launch of the client. Maybe @RMerlin can elaborate if this is intended or a technical blip.