VPNMON VPNMON-R3 Custom Server List Generation Tutorials and Examples

[Viktor Jaep]

I have not tried Unbound via NordVPN yet so what exactly is not working when it comes to Unbound via NordVPN ? Top level DNS servers are not accesible via NordVPN ?

Like something is getting blocked when using Unbound over NordVPN. They have acknowledged the issue. No resolution in sight. It still works great on AirVPN.

 

[Rajjco]

Here is the code for ProtonVPN servers If anyone wants it. Haven't tested it yet.

curl --silent --retry 3 --connect-timeout 3 --max-time 6 --retry-delay 1 --retry-all-errors https://api.protonvpn.ch/vpn/logicals | jq --raw-output '.LogicalServers[] | select(.Name | contains("US")) | .Servers[].ExitIP'

replace the contains("US") with the country prefix of your choice.

 

[Viktor Jaep]

Here is the code for ProtonVPN servers If anyone wants it. Haven't tested it yet.

curl --silent --retry 3 --connect-timeout 3 --max-time 6 --retry-delay 1 --retry-all-errors https://api.protonvpn.ch/vpn/logicals | jq --raw-output '.LogicalServers[] | select(.Name | contains("US")) | .Servers[].ExitIP'

replace the contains("US") with the country prefix of your choice.

Nice job! Looks like it does the trick! I'll add this to the OP instructions!

 

[Rajjco]

Nice job! Looks like it does the trick! I'll add this to the OP instructions!

I just tested the previous command and it didn't work I had to replace the .ExitIP with .EntryIP to make it connect.

However I ran into an issue where it returns multiple entries for each ip please run this curl and tell me what you think of the result. One thing I think we should take into account is the port number since the api doesn't list any port numbers for the servers.

curl --silent --retry 3 --connect-timeout 3 --max-time 6 --retry-delay 1 --retry-all-errors https://api.protonvpn.ch/vpn/logicals | jq --raw-output '.LogicalServers[] | select(.Name | contains("DE")) | .Servers[].EntryIP'

 

[Viktor Jaep]

I just tested the previous command and it didn't work I had to replace the .ExitIP with .EntryIP to make it connect.

However I ran into an issue where it returns multiple entries for each ip please run this curl and tell me what you think of the result. One thing I think we should take into account is the port number since the api doesn't list any port numbers for the servers.

curl --silent --retry 3 --connect-timeout 3 --max-time 6 --retry-delay 1 --retry-all-errors https://api.protonvpn.ch/vpn/logicals | jq --raw-output '.LogicalServers[] | select(.Name | contains("DE")) | .Servers[].EntryIP'

Good call. Typically the port would be the same across the board... you would receive that information when you download your server cert and import that into the VPN Client Slot section.

By a complete stroke of luck, I've had success dumping out the entire list of cities and countries in a sorted list as well!

All possible City Names:

curl --silent --retry 3 --connect-timeout 3 --max-time 6 --retry-delay 1 --retry-all-errors https://api.protonvpn.ch/vpn/logicals | jq --raw-output '.LogicalServers | group_by(.City) | map(.[0].City)'

All possible Country codes:

curl --silent --retry 3 --connect-timeout 3 --max-time 6 --retry-delay 1 --retry-all-errors https://api.protonvpn.ch/vpn/logicals | jq --raw-output '.LogicalServers | group_by(.EntryCountry) | map(.[0].EntryCountry)'

 

[Rajjco]

Awesome Job!!

here is the command to which filters duplicate entries by adding | sort | uniq at the end.

curl --silent --retry 3 --connect-timeout 3 --max-time 6 --retry-delay 1 --retry-all-errors https://api.protonvpn.ch/vpn/logicals | jq --raw-output '.LogicalServers[] | select(.EntryCountry | contains("DE")) | .Servers[].EntryIP' | sort | uniq

 

[Viktor Jaep]

Awesome Job!!

Same to you, @Rajjco! I've added your instructions here under the main OP! Thanks again!

VPNMON - VPNMON-R3 Custom Server List Generation Tutorials and Examples

FULL DISCLOSURE: I absolutely SUCK at JQuery... 😋 It's not like SQL at all, which I'm pretty decent with. I think the reason I am no good at this is because the sources this data comes from is always structured very differently in JSON, so there's always a completely different way at getting at...

www.snbforums.com

 

[JTnola]

Here’s so info I got about Private Internet Access (PIA)…. In case anyone who’s better at this than me wants to rewrite this as a useable CURL + JQ statement for PIA servers.

curl https://serverlist.piaservers.net/vpninfo/servers/v6

Do note that the result is 1-line JSON with some sort of digital signature appended, so jq or similar will pop a warning but otherwise should work fine - or you can just feed it through head -n1 if you don't like the warning.

My pia-wg script [https://github.com/triffid/pia-wg/blob/master/pia-wg.sh] may offer some hints on parsing this file in case it helps.

 

[Viktor Jaep]

Here’s so info I got about Private Internet Access (PIA)…. In case anyone who’s better at this than me wants to rewrite this as a useable CURL + JQ statement for PIA servers.

I think I may have something here!!

This provides a list of country codes... I'm going to be using "US":

curl --silent --retry 3 --connect-timeout 3 --max-time 6 --retry-delay 1 --retry-all-errors https://serverlist.piaservers.net/vpninfo/servers/v6 | jq --raw-output '.regions[] | .country' | sort | uniq 2>/dev/null

This query extracts all OVPN UDP servers for the US:

curl --silent --retry 3 --connect-timeout 3 --max-time 6 --retry-delay 1 --retry-all-errors https://serverlist.piaservers.net/vpninfo/servers/v6 | jq --raw-output '.regions[] | select(.country=="US") | .servers.ovpnudp[].ip' | sort | uniq 2>/dev/null

But even though it comes back with what seems like a legit list of IPs, I can't seem to match any of the output back to the original https://serverlist.piaservers.net/vpninfo/servers/v6 list... where am going wrong?

EDIT: OK... when you refresh the URL, all IPs keep changing. WTH? So that's why. I would be interesting to see if an IP that was queried from an hour ago would still work.

This provides a list of all US-based cities and states mixed (if you want to narrow down where you connect to - again, using your country code):

curl --silent --retry 3 --connect-timeout 3 --max-time 6 --retry-delay 1 --retry-all-errors https://serverlist.piaservers.net/vpninfo/servers/v6 | jq --raw-output '.regions[] | select(.country=="US") | .name' | sort | uniq 2>/dev/null

So if you just want to connect to all servers in the "US New York" area, you would use:

curl --silent --retry 3 --connect-timeout 3 --max-time 6 --retry-delay 1 --retry-all-errors https://serverlist.piaservers.net/vpninfo/servers/v6 | jq --raw-output '.regions[] | select(.name=="US New York") | .servers.ovpnudp[].ip' 2>/dev/null

And for some reason, this API extract contains some sort of key on the bottom that messes with the JQ output, and produces this error message after it extracts all the IP addresses... need to figure out how to eliminate this.

jq: parse error: Invalid numeric literal at line 4, column 0

EDIT: and that's fixed by putting a "2>/dev/null" on the end of the curl statement! Whoo! Looks like PIA is getting added to the list!

 

[Viktor Jaep]

@JTnola ... please confirm this all works OK for you once integrated into VPNMON-R3? Once I have your blessing, I'll get it added to the OP! Thanks for sharing... you got me down a rabbit hole, but I think this is going to work!

 

[JTnola]

@JTnola ... please confirm this all works OK for you once integrated into VPNMON-R3? Once I have your blessing, I'll get it added to the OP! Thanks for sharing... you got me down a rabbit hole, but I think this is going to work!

Thank you for all your work on this… the one issue (at least I think it’s an issue?) is: when I execute the command listed, it generates a single IP address. Rather than adding that new ip to my existing list of server ip addresses, it replaces the list with the single entry.

Is that supposed to happen?

 

[Viktor Jaep]

Thank you for all your work on this… the one issue (at least I think it’s an issue?) is: when I execute the command listed, it generates a single IP address. Rather than adding that new ip to my existing list of server ip addresses, it replaces the list with the single entry.

Is that supposed to happen?

Could very well be if the narrowed query you're running is for a certain city that only has 1 server. PIA doesn't have a huge footprint like Nord does. If you can post the query you ran, I can validate. To get a bigger list, you would need to expand to a state region or even the country.

 

[JTnola]

Could very well be if the narrowed query you're running is for a certain city that only has 1 server. PIA doesn't have a huge footprint like Nord does. If you can post the query you ran, I can validate. To get a bigger list, you would need to expand to a state region or even the country.

what happens is that every time I run the command, it returns a single IP address — not the same IP address. Run it 5 times, get 5 server IP addresses

curl --silent --retry 3 --connect-timeout 3 --max-time 6 --retry-delay 1 --retry-all-errors https://serverlist.piaservers.net/vpninfo/servers/v6 | jq --raw-output '.regions[] | select(.name=="US Alabama") | .servers.ovpnudp[].ip' 2>/dev/null

Running it 5 separate times, it returns one of these addresses—among others by for each time I run it.
84.239.6.3
84.239.6.9
84.239.6.15
84.239.6.8
84.239.6.20

And when run from inside VPNMON, let’s say I had input those 5 IP addresses manually. Then later go to execute the script using X1… it removes my existing list and replaces it with the single returned up address, rather than adding the new address to my existing list. (Which is prob the right course of action—at least when the script returns a long list

 

[Viktor Jaep]

what happens is that every time I run the command, it returns a single IP address — not the same IP address. Run it 5 times, get 5 server IP addresses

curl --silent --retry 3 --connect-timeout 3 --max-time 6 --retry-delay 1 --retry-all-errors https://serverlist.piaservers.net/vpninfo/servers/v6 | jq --raw-output '.regions[] | select(.name=="US Alabama") | .servers.ovpnudp[].ip' 2>/dev/null

Running it 5 separate times, it returns one of these addresses—among others by for each time I run it.
84.239.6.3
84.239.6.9
84.239.6.15
84.239.6.8
84.239.6.20

And when run from inside VPNMON, let’s say I had input those 5 IP addresses manually. Then later go to execute the script using X1… it removes my existing list and replaces it with the single returned up address, rather than adding the new address to my existing list. (Which is prob the right course of action—at least when the script returns a long list

I had found the same thing... PIA seems to randomly rotate their servers IP addresses very frequently. Not sure if that's by design, or if they have more servers than what they indicate... but I guess whatever works for them!

But what VPNMON is doing is by design. It's meant to pull the latest list of available servers based on what the API returns, and replaces your list. You can set it on a scheduled reset and updates the list at the same time. VPN providers are notorious for shutting servers down, creating new ones, etc. in the middle of the day, so it's important to keep that list fresh.

 

[JTnola]

I had found the same thing... PIA seems to randomly rotate their servers IP addresses very frequently. Not sure if that's by design, or if they have more servers than what they indicate... but I guess whatever works for them!

But what VPNMON is doing is by design. It's meant to pull the latest list of available servers based on what the API returns, and replaces your list. You can set it on a scheduled reset and updates the list at the same time. VPN providers are notorious for shutting servers down, creating new ones, etc. in the middle of the day, so it's important to keep that list fresh.

So basically, even if it only populates my “list” with one server IP address, it could still be valuable, in so far as it’ll be using one of the most current IP addresses..?

 

[Viktor Jaep]

So basically, even if it only populates my “list” with one server IP address, it could still be valuable, in so far as it’ll be using one of the most current IP addresses..?

Correct... it would be the most current IP available. And I'm sure it will change again tomorrow knowing how PIA operates now.

 

[Rajjco]

Since PIA rotates their servers and only returns a couple of ip's per request you can use this command to grab more ip's if you wish. It will grab more ip's using a for loop and show only the unique ones.

for i in {1..5}; do   curl --silent --retry 3 --connect-timeout 3 --max-time 6 --retry-delay 1 --retry-all-errors https://serverlist.piaservers.net/vpninfo/servers/v6 | jq --raw-output '.regions[] | select(.name=="US New York") | .servers.ovpnudp[].ip' 2>/dev/null; done | sort | uniq

If you want a larger list just increase the number of iterations e.g: change for i in {1..5} to for i in {1..10}

-----------------------------------------------------------------------------------------------

Another tip which can be used to grab and filter ip's is to ping every single ip that is returned before displaying the list of ips with this method you will only get the ip's that are close to your location.

Just add this command at the end of the curl statment.
To set the target ping just change the value here e.g: 'if (a[2] < 200)' 200 = 200ms
This will grab only ip's with less than 200ms ping.

| while read -r ip; do ping -c 1 -W 1 "$ip" | awk -v ip="$ip" '/time=/{split($7,a,"="); if (a[2] < 200) print ip}'; done

Full Example:

curl --silent --retry 3 --connect-timeout 3 --max-time 6 --retry-delay 1 --retry-all-errors https://api.nordvpn.com/v1/servers?limit=16354 | jq --raw-output '.[] | select(.locations[0].country.city.name == "New York") | .station' | while read -r ip; do ping -c 1 -W 1 "$ip" | awk -v ip="$ip" '/time=/{split($7,a,"="); if (a[2] < 200) print ip}'; done

Tested on NordVPN and ProtonVPN.

 

[Viktor Jaep]

Since PIA rotates their servers and only returns a couple of ip's per request you can use this command to grab more ip's if you wish. It will grab more ip's using a for loop and show only the unique ones.

for i in {1..5}; do   curl --silent --retry 3 --connect-timeout 3 --max-time 6 --retry-delay 1 --retry-all-errors https://serverlist.piaservers.net/vpninfo/servers/v6 | jq --raw-output '.regions[] | select(.name=="US New York") | .servers.ovpnudp[].ip' 2>/dev/null; done | sort | uniq

If you want a larger list just increase the number of iterations e.g: change for i in {1..5} to for i in {1..10}

-----------------------------------------------------------------------------------------------

Another tip which can be used to grab and filter ip's is to ping every single ip that is returned before displaying the list of ips with this method you will only get the ip's that are close to your location.

Just add this command at the end of the curl statment.
To set the target ping just change the value here e.g: 'if (a[2] < 200)' 200 = 200ms
This will grab only ip's with less than 200ms ping.

| while read -r ip; do ping -c 1 -W 1 "$ip" | awk -v ip="$ip" '/time=/{split($7,a,"="); if (a[2] < 200) print ip}'; done

Full Example:

curl --silent --retry 3 --connect-timeout 3 --max-time 6 --retry-delay 1 --retry-all-errors https://api.nordvpn.com/v1/servers?limit=16354 | jq --raw-output '.[] | select(.locations[0].country.city.name == "New York") | .station' | while read -r ip; do ping -c 1 -W 1 "$ip" | awk -v ip="$ip" '/time=/{split($7,a,"="); if (a[2] < 200) print ip}'; done

Tested on NordVPN and ProtonVPN.

Wow, that's ingenious, @Rajjco!

 

[Rajjco]

Wow, that's ingenious, @Rajjco!

Glad you liked it! I'm a self taught Python developer I try to use flexible Python methods in other languages using some prompt engineering

 

[Viktor Jaep]

Glad you liked it! I'm a self taught Python developer I try to use flexible Python methods in other languages using some prompt engineering

You definitely have a gift, and what seems a much easier time developing these statements than I do!

I gave your "for i" loop sample a try, and while New York resulted in 3 returns, Alabama (as in @JTnola's case) only comes back with 1 result for me... I would have expected 5 for each... I wasn't sure if posix could handle a loop like that, so I changed it up... also added a sleep 1 in there, to see if that would cause the API to give me more IPs as well... so this statement consistently gives me 4 IPs:

i=0; while [ $i -ne 5 ]; do i=$((i + 1)); curl --silent --retry 3 --connect-timeout 3 --max-time 6 --retry-delay 1 --retry-all-errors https://serverlist.piaservers.net/vpninfo/servers/v6 | jq --raw-output '.regions[] | select(.name=="US Alabama") | .servers.ovpnudp[].ip' 2>/dev/null; sleep 1; done | sort | uniq