VPN Policy-Based Routing + Web UI - ARCHIVE #1

I tried this with a PIA UK VPN only and it did go to a UK DNS Server from my Win10 PC but when I set my PC back to the WAN with OPR it still went to the UK DNS Server when it should have went through the local DNS server, not UK. Added another route entry in the NLD PIA VPN with another DNS Server IP and it also went through the UK DNS Server when it should have gone through a NLD DNS Server,

How do I get this to react correctly?

What I want to do is the following:

When my PC is not set in OPR to any VPN but to WAN I want the DNS server to be a local DNS Server.

When my PC is set for a UK VPN in OPR I want the DNS request to go through the DNS Server in the UK.

When my PC is set for a NLD VPN in OPR I want the DNS request to go through the DNS Server in the NLD.

Anyone have any ideas how to do this with OPR or OpenVPN settings?

Updated both OPR and OPR WebUI to build 11. Some basic code to support wireguard in both packages (wireguard tunnel routing is not working yet tho). Also I think I've identified why OPR stopped reloading automatically and fixed that.

Bad news @stangri ... Both VPNbypass and OPR are not working at all for me, even after clean install :pensive:

(though for the record, when vpnbypass was working, and OPR not, I could sinply opkg remove OPR and opkg install vpnbypass and vpnbypass would work properly, so fresh isntall not required, in the past)

EDIT: vpnbypass is working sorry, just took several save&applies in luci, or, I guess, jsut wait enough time for services to restart? But vpnbypass is working as intended.

EDIT2: i assume you've stuck with the format "/whatismyipaddress.com/vpnbypass" incase people have multiple wan routes and want to choose? (i have only one, so I'm not sure what else you'd actually put there). May I just suggest, since I believe most people would just have two routes, the vpn tunnel and the wan route, that you change the format to just one domain per line and just "whatismyipaddress.com" you could then programmatically add the leading and trailing /, and default 'vpnbypass/wanroute' as appropriate. It's more unecessary work for you, but a convenience thing for us :wink:

Updated to latest OPR and I think it solved my startup issue.

I had set OPR to 99 (vice 94) startup sequence and added "sleep 30
and /etc/init.d/openvpn-policy-routing reload" in the "rc.local" via GUI to get OPR to correctly stasrtup.

I have reverted to normal OPR startup and it works fine.

@stangri Thanks for the effort to find and fix the reloading issue.

Edit: I spoke too soon. The system log and the routing table all appeared good but when I tested a static IP that should have been set to tun1 it went through the WAN. Did some adjustments and rebooted a few time but it did not change. I reverted back to OPR 4.1.5-9 and all is working OK again.

I have no idea what is wrong but will try the next OPR Revision to see if it is better.

I have a test build of OPR/OPR-luci which:

  1. does not use ipset/dnsmasq-full and resolves domains at the time service starts and adds their IPs as iptables rules instead
  2. allows you to mix and match IPv4, IPv6 and domain names in remote address
  3. also allows you to mix and match IPv4, IPv6 and local device names in local address
  4. is config-incompatible with the earlier versions, read below for upgrade instructions
  5. as each domain name can result in multiple new iptables rules, it's a bit slower to start than previous versions

Some of the example policies:

config policy
option comment 'LogmeIn Hamachi'
option gateway 'wan'
option remote_addresses '25.0.0.0/8 hamachi.cc hamachi.com logmein.com'

config policy
option comment 'he.net'
option gateway 'wan'
option remote_addresses '66.220.2.74 he.net tunnelbroker.net'

config policy
option gateway 'wan'
option comment 'Plex Remote'
option remote_addresses 'plex.tv my.plexapp.com'

config policy
option gateway 'wan'
option comment 'Media Devices'
option local_addresses 'firehdx nexusplayer 192.168.1.81/28'

THIS VERSION IS CONFIG-INCOMPATIBLE with earlier versions, please use --force-maintainer option when installing/upgrading, like: opkg update; opkg install --force-maintainer ./openvpn-policy-routing-5.0-1.ipk; opkg install ./luci-app-openvpn-policy-routing-15_all.ipk

ZIP of IPKs can be downloaded here: https://www.melmac.net/openwrt/packages/OPR-5.0.1.zip, I would appreciate feedback before I push it to my repo.

PS. If things don't work, try /etc/init.d/openvpn-policy-routing reload first and post/PM me the output of /etc/init.d/openvpn-policy-routing support.
PPS. IPv6 support is still disabled for now.

How did you roll back to the previous version 4.1.5-9? Mine isn't working either, but unable to roll back

I like the paste.ee functionality!

One possible thing to have a look at when it comes to the service stopping - if there's any activity with fw3 all iptables rules get reset. Anybody messing with their firewall settings at any point is likely to get the OPR rules nuked in the process, without informing OPR of it happening.

The service_triggers() in the init.d weren't enough to get it to work when I was doing my routing package (even when listening for a "firewall" config update as well) - you'll likely need to set up an entry in /etc/config/ucitrack such that the firewall config affects OPR. If anyone makes a change in luci to the firewall config, this will instruct luci to invoke /etc/init.d/openvpn-policy-routing reload, once the firewall has done its thing... and they'll even get a nice notification of OPR doing its thing when they hit "Save and Apply". Hope it helps!

@tzarc -- thanks Nick, I'm not very familiar with OpenWrt makefiles, if you can share how do you do the ucitrack in post-install I'd be grateful.

I've pushed 5.0.2 to the repo, this version is config-incompatible with the previous versions! If you're upgrading, please run: opkg update; opkg upgrade --force-maintainer openvpn-policy-routing; opkg upgrade luci-app-openvpn-policy-routing and then re-create your config.

@stangri Is domain based policy routing no longer supported in the new version of OPR? If so how how may we go about setting it so that certain domains go to the selected interface?

@stangri

Ohh wait, disregard. I think I figured it out by looking at your example config from a few posts above. Just added a new policy, named it Domains and set the remote addresses to the domains I wanted separated by spaces, and also set the gateway to the interface I wanted the domains to go through. super simple thanks. Sorry I didn't wait just 5 mins to figure it out on my own before asking in the previous post.

1 Like

And yet again, I've just pushed a version (5.0.1) which is config-incompatible with older versions. This time the only change was that gateway was changed to interface in the config to better reflect the actual values stored there.

However, there are some other important (under the hood) changes:

  1. I've switched to the table IDs in the 200-range which should fix OPR not working properly on stable (17.01.x) builds.
  2. For policies with only local or only remote addresses/domain names I try to add them to ipset first. That significantly improved the start up time if you have tens/dozens domain names in your policies. As it also results in a fewer iptables rules, it reduced the processing penalty for each frame if you have tens/dozens of domain names in policies.

So if you've tried an older OPR version and it didn't work (or stopped working at some point) or if you have many domain names in your policies and want to speed things up, run: opkg update; opkg upgrade --force-maintainer openvpn-policy-routing; opkg upgrade luci-app-openvpn-policy-routing .

PS. Even tho I'm using ipsets internally, domains are still defined in regular policies and there's still no need for dnsmasq-full.

Updated to new version OPR and get the following error on boot-up. It is not the same IP each time and it appears to be random. Reload has no errors.

ERROR: iptables -t mangle -I OPR_CHAIN 1 -j MARK --set-xmark 0x010000/0xff0000 -s 192.168.2.190 -m comment --comment GalaxyTabA6

1 Like

I get those sometimes too, I can't figure out why.

@stangri

I've been trying out this new way of setting my DNS server that someone suggested over on @davidc502's forum. Basically you put your custom DNS server under "DNS forwardings" on the "DHCP and DNS" settings page (when doing it through luci interface). You also must check the box for "Ignore resolve file" on the "Resolv and Host Files" tab (this is supposed to tell dnsmasq to ignore the preconfigured resolve file's that the system gets from your ISP and use the custom settings you have set in DNS forwarding). This all works fine, until I install OPR and setup my policies and I cant figure it out.

Well it doesn't completely break the system, all my clients are able to access the internet and ping each other perfectly fine. The problem is that now when I try to access the router through web browser it is extremely slow to load each page. everything will work but takes very long to load and change settings. And also when I ssh to the router I am unable to opkg update the router (this is how I discovered the issue in the first place). also while ssh'd to the router I am able to ping addresses like "8.8.8.8" "8.8.4.4", but unable to ping domain names such as "google.com". any Idea what might be going on here?

Thanks by the way, I think this forum and package are a great asset to the OpenWRT/ LEDE community. and I appreciate all the hard work you and everyone else put into it.

I don't think that what that setting is for. I run the same config (custom DNS servers in DNS forwardings) but without ignoring local resolv file and I do not experience the issue you've mentioned.

1 Like

@stangri
I don't think that what that setting is for. I run the same config (custom DNS servers in DNS forwardings) but without ignoring local resolv file and I do not experience the issue you've mentioned.

See I figured that Ignore resolv file does not need to be checked either, because my DNS settings work just fine without it being checked, no DNS leaks when I perform DNS leak test. However @starcms on @davidc502's forum seems to think differently on this topic. I normally follow what ever yourself or starcms recomment but since you conflict with each other on this one I'll have to go by what works which is unchecking ignore resolv file.

Thanks for the input stangri, really appreciate the help.

Tested OPR for a week and can't get how to test policy enforcement correctly.
Case 1. Policy enforcement is strict. openvpn running and connected. Test VM routed via tunnel.
If I disastrously disable openvpn like '/etc/init.d/openvpn stop' test VM goes via wan.

Case 2. Policy enforcement lazy. openvpn running and connected. Test VM routed via tunnel.
I block openvpn traffic with firewall. Test VM looses all connectivity.

Case 3. Policy enforcement is strict. openvpn stopped and I add rules to block openvpn traffic. Start openvpn, restart OPR. openvpn can't connect. Test VM goes via wan.

opkg list_installed | grep policy
luci-app-openvpn-policy-routing - git-17.220.63907-bbcf73f-16
openvpn-policy-routing - 5.0.1-1

QoS app uninstalled as I read it could distort OPR packet marking.

How can I correctly test policy enforcement? What information to provide also?

Just FYI:

I tried to upgrade to R3492 from R3491 from here: http://downloads.lede-project.org/releases/17.01-SNAPSHOT/targets/mvebu/generic/

I use the image builder found there with the following packages:

make image PROFILE=linksys-wrt1900ac PACKAGES="-dnsmasq dnsmasq-full vsftpd-tls libustream-mbedtls luci-app-ddns ipset luci-app-openvpn-policy-routing luci-app-advanced-reboot block-mount collectd collectd-mod-cpu collectd-mod-interface collectd-mod-iwinfo collectd-mod-load collectd-mod-memory collectd-mod-network collectd-mod-sensors collectd-mod-rrdtool curl e2fsprogs f2fs-tools f2fsck iptables-mod-conntrack-extra iptables-mod-ipopt iwinfo kmod-crypto-aead kmod-crypto-authenc kmod-crypto-cbc kmod-crypto-crc32c kmod-crypto-deflate kmod-crypto-des kmod-crypto-ecb kmod-crypto-echainiv kmod-crypto-hash kmod-crypto-hmac kmod-crypto-iv kmod-crypto-manager kmod-crypto-md5 kmod-crypto-null kmod-crypto-pcompress kmod-crypto-rng kmod-crypto-sha1 kmod-crypto-sha256 kmod-crypto-wq kmod-fs-ext4 kmod-fs-f2fs kmod-fs-hfs kmod-fs-hfsplus kmod-fs-msdos kmod-fs-ntfs kmod-fs-vfat kmod-hwmon-core kmod-hwmon-pwmfan kmod-ifb kmod-ipt-conntrack-extra kmod-ipt-ipopt kmod-ipt-ipset kmod-lib-crc16 kmod-lib-zlib-deflate kmod-lib-zlib-inflate kmod-mmc kmod-mwifiex-sdio kmod-nf-conntrack-netlink kmod-nfnetlink kmod-nls-base kmod-nls-cp437 kmod-nls-iso8859-1 kmod-nls-utf8 kmod-sched kmod-sched-cake kmod-sched-connmark kmod-sched-core kmod-scsi-core kmod-tun kmod-usb-core kmod-usb-ledtrig-usbport kmod-usb-storage kmod-usb-storage-extras kmod-usb-uhci kmod-usb2 libcurl libext2fs libf2fs libgmp libgnutls libiwinfo libiwinfo-lua libltdl liblua liblzo libmnl libncurses libnetfilter-conntrack libnettle libnfnetlink libopenssl libpcre librrd1 librt libsensors libsysfs libubus-lua libuci-lua libusb-1.0 libuuid lm-sensors lua luci luci-app-firewall luci-app-openvpn luci-app-samba luci-app-sqm luci-app-statistics luci-app-upnp luci-base luci-lib-ip luci-lib-jsonc luci-lib-nixio luci-mod-admin-full luci-proto-ipv6 luci-proto-ppp luci-theme-bootstrap luci-theme-material miniupnpd mkf2fs mtd mwifiex-sdio-firmware nano netifd odhcp6c odhcpd openssl-util openvpn-easy-rsa openvpn-openssl opkg ppp ppp-mod-pppoe procd procd-nand rpcd rrdtool1 samba36-server sqm-scripts sqm-scripts-extra swconfig sysfsutils tc terminfo tune2fs ubi-utils uboot-envtools ubox ubus ubusd uci uclient-fetch uhttpd uhttpd-mod-ubus usbutils usign wget wpad zlib"

I have built the WRT1900AC V1 image this way for a while now but I had to revert to R3491 (1 prior rev) to have OPR work today.

Only the VPN entries (1 WAN, 2 VPN) get written in the Firewall status, the many static IPs I have setup do not get inserted into the firewall although it looks like OPR is working correctly in the system log.

I will build the next rev when it comes out to see if that fixes the issue.

That's an unexpected behavior, but it's been a while since I tested that. Let me re-test and I'll get back to you.

QoS shouldn't affect OPR.

OPR is now powered by magic. :wink: For simple rules (a domain or a local device or an IP without any ports) OPR tries to insert the IP to proper ipset and only if it fails it adds an iptables rule. If you run support you'll see a lot of static IPs in the ipsets. From my testing it's a faster method.

1 Like

I've made a mistake in handling ipsets in early 5.0.1 builds resulting in local device policies possibly not working.

5.0.1-5 has proper support for ipsets for both local and remote addresses (IPv4 only so far) with clarifying comments and also introduced support for a new config settings "ipset_enabled" -- if manually created and set to 0, all policies will result in iptables rules and ipsets won't be used at all. Reason for this setting is that from what I've read, putting domains into ipset might result only in first of its resolved ip addresses added to ipset. So if you notice that some domains are not working as expected, try doing: uci set openvpn-policy-routing.config.ipset_enabled=0; uci commit;