Is all the RAM being used mainly cache?
Please run the following and take a look to see if the amount of RAM used has gone down.
sync; echo 1 > /proc/sys/vm/drop_caches
sync; echo 2 > /proc/sys/vm/drop_caches
sync; echo 3 > /proc/sys/vm/drop_caches Unfortunately this doesn't seem to be the issue. That does increase the MemFree a little bit, but we're still well beyond what should be being used.
Before sync:
MemTotal: 510920 kB
MemFree: 30436 kB
MemAvailable: 21080 kB
Buffers: 6488 kB
Cached: 19220 kB
SwapCached: 0 kB
Active: 22908 kB
Inactive: 6804 kB
Active(anon): 4944 kB
Inactive(anon): 256 kB
Active(file): 17964 kB
Inactive(file): 6548 kB
Unevictable: 4 kB
Mlocked: 4 kB
HighTotal: 0 kB
HighFree: 0 kB
LowTotal: 510920 kB
LowFree: 30436 kB
SwapTotal: 0 kB
SwapFree: 0 kB
Dirty: 0 kB
Writeback: 0 kB
AnonPages: 4012 kB
Mapped: 6140 kB
Shmem: 1196 kB
Slab: 179416 kB
SReclaimable: 5928 kB
SUnreclaim: 173488 kB
KernelStack: 672 kB
PageTables: 348 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 255460 kB
Committed_AS: 9704 kB
VmallocTotal: 507904 kB
VmallocUsed: 0 kB
VmallocChunk: 0 kB
After sync:
MemTotal: 510920 kB
MemFree: 50684 kB
MemAvailable: 30400 kB
Buffers: 240 kB
Cached: 7224 kB
SwapCached: 0 kB
Active: 7796 kB
Inactive: 3688 kB
Active(anon): 4272 kB
Inactive(anon): 944 kB
Active(file): 3524 kB
Inactive(file): 2744 kB
Unevictable: 4 kB
Mlocked: 4 kB
HighTotal: 0 kB
HighFree: 0 kB
LowTotal: 510920 kB
LowFree: 50684 kB
SwapTotal: 0 kB
SwapFree: 0 kB
Dirty: 0 kB
Writeback: 0 kB
AnonPages: 4012 kB
Mapped: 6128 kB
Shmem: 1196 kB
Slab: 175796 kB
SReclaimable: 2312 kB
SUnreclaim: 173484 kB
KernelStack: 696 kB
PageTables: 344 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 255460 kB
Committed_AS: 9704 kB
VmallocTotal: 507904 kB
VmallocUsed: 0 kB
VmallocChunk: 0 kB
Wireless router crashed and the memory at the end looked like this:
MemTotal: 510920 kB
MemFree: 18100 kB
MemAvailable: 0 kB
Buffers: 1580 kB
Cached: 4412 kB
SwapCached: 0 kB
Active: 6748 kB
Inactive: 3300 kB
Active(anon): 4328 kB
Inactive(anon): 924 kB
Active(file): 2420 kB
Inactive(file): 2376 kB
Unevictable: 4 kB
Mlocked: 4 kB
HighTotal: 0 kB
HighFree: 0 kB
LowTotal: 510920 kB
LowFree: 18100 kB
SwapTotal: 0 kB
SwapFree: 0 kB
Dirty: 0 kB
Writeback: 0 kB
AnonPages: 4064 kB
Mapped: 2416 kB
Shmem: 1196 kB
Slab: 176892 kB
SReclaimable: 2288 kB
SUnreclaim: 174604 kB
KernelStack: 728 kB
PageTables: 344 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 255460 kB
Committed_AS: 10004 kB
VmallocTotal: 507904 kB
VmallocUsed: 0 kB
VmallocChunk: 0 kB
It'll be a couple more days until it bugs out again and starts leaking afaict
Did you try enabling kernel threads in htop and then ordering by memory usage? That should give you leaker.
I was still also checking ps, which displays the kernel threads, and no process appeared to be leaking any memory, it looks like most of the leaking (if not all) is to slab allocation. Unfortunately there's no /proc/slabinfo to peer into in this kernel build
What other modules are installed on this build. This would be on top of what the build already comes with.
Nothing at all, normally I actually remove a lot of modules, but for the purpose of testing I did not when I upgraded to the latest build a couple days ago. I run 2 SSIDs on each radio, 1 on vlan 1 and the other on vlan 2, other than that it runs as a dumb AP, no dhcp, no dns, nothing. My server handles all the services for dns, dhcp, routing, pppoe, etc.
(honestly the only reason I even need openwrt/lede is for vlans and multiple ssids, Nest Smoke Detectors send out router advertisements for ipv6 to set up their own network, which were screwing up my network, so I had to isolate them)
/etc/config/network
config interface 'loopback'
option ifname 'lo'
option proto 'static'
option ipaddr '127.0.0.1'
option netmask '255.0.0.0'
config globals 'globals'
option ula_prefix 'fddf:d092:9cae::/48'
config interface 'lan'
option type 'bridge'
option ifname 'eth0.1'
option proto 'static'
option netmask '255.255.255.0'
option ipaddr '192.168.1.249'
option gateway '192.168.1.1'
option dns '192.168.1.1 8.8.8.8'
config switch
option name 'switch0'
option reset '1'
option enable_vlan '1'
config switch_vlan
option device 'switch0'
option vlan '1'
option ports '0 1 2 3 4 5t'
option vid '1'
config switch_vlan
option device 'switch0'
option vlan '2'
option ports '4t 6t'
option vid '2'
config interface 'VLAN2'
option type 'bridge'
option proto 'static'
option ifname 'eth1.2'
option ipaddr '192.168.254.249'
option netmask '255.255.255.0'
option gateway '192.168.254.1'
/etc/config/wireless
config wifi-device 'radio0'
option type 'mac80211'
option hwmode '11a'
option htmode 'VHT80'
option channel '44'
option country 'CA'
option path 'soc/soc:pcie/pci0000:00/0000:00:01.0/0000:01:00.0'
config wifi-iface 'default_radio0'
option device 'radio0'
option network 'lan'
option mode 'ap'
option ssid 'a'
option encryption 'psk2'
option key '------'
config wifi-device 'radio1'
option type 'mac80211'
option hwmode '11g'
option htmode 'HT20'
option channel '6'
option country 'CA'
option path 'soc/soc:pcie/pci0000:00/0000:00:02.0/0000:02:00.0'
config wifi-iface 'default_radio1'
option device 'radio1'
option network 'lan'
option mode 'ap'
option ssid 'a'
option encryption 'psk2'
option key '-------'
config wifi-iface
option device 'radio0'
option mode 'ap'
option ssid 'a-nest'
option network 'VLAN2'
option encryption 'psk2'
option key '------'
config wifi-iface
option device 'radio1'
option mode 'ap'
option ssid 'a-nest'
option network 'VLAN2'
option encryption 'psk2'
option key '-----'
From this point you might try removing even more modules.
Have you removed luci-app-statistics?
opkg remove --autoremove luci-app-statistics
Wouldn't that have had some indication in ps from a process as having taken up a lot of memory if it was part of luci, or anything except some of the kmod packages?
@davidc502 Any chance you can add CONFIG_SLABINFO=y to the kernel whenever you do a new build next so that I can try to get further insight into what's leaking, since it seems to be kernel related?
No problem..
@david -
i know there's a lot going on with openwrt/lede, the forums, new forks, and your builds. i appreciate your efforts tremendously.
your 5/18 build seemed to have more wifi probs on my wrt32x than the previous build had on my wrt3200acm (same silicon).
for what it's worth, i have loaded 5/23 lede 18.06 snapshot onto my wrt32x and it has been up for 4 hours now. i'll provide further info later. is there anything in particular that i can do to help you?
thanks again
@davidc502
Can you please include this feed in your builds please https://github.com/InkblotAdmirer/custom_feed.
This feed has a dnscrypt-proxy-v2 package.
I have never used dnscrypt-proxy-v2.
Because of curiosity, I just took about 10 minutes to download the files, and configure. Seems to work okay so far.
I have no idea what all is involved to create a custom feed to complete the download/installation of files during the compile process.
@davidc502
I don't know either 
Perhaps just build the package and host it: https://github.com/InkblotAdmirer/custom_feed/tree/master/dnscrypt-proxy-v2
It's going to take a deeper understanding... I think I can create the custom feed, and perhaps download the right binary and configuration files, but after that........
We have a lot of people who use the old dnscrypt-proxy...
I'm thinking I can add the dnscrypt-proxy binary to the /root directory... Will look further.
The actual package is cleverly named dnscrypt-proxy-v2. I believe anyone who wants to try it has to uninstall the original package.
Thank you for your consideration.
That package should work with an existing dnscrypt-proxy config file, unless blacklisting is enabled. It would be possible to convert syntax but the logic to support backward-compatibility is already pretty convoluted and I want to limit the number of test cases.
If dnscrypt-proxy-v2 is installed over dnscrypt-proxy it will create the correct /etc/dnscrypt-proxy folder (which is now used to store the configuration files actually used by the program) and it will create /etc/config/dnscrypt-proxy-opkg with the new syntax. The user would be responsible for one of the following:
- rename /etc/config/dnscrypt-proxy prior to installing the -v2 package
or - convert their configuration for the original /etc/config/dnscrypt-proxy into the new format
-v2 only listens on a single port now, and automatically selects from the selected servers based on periodic response time checks.
Information on how it works, differences from original, and how to configure should be in the readme of the custom feed link.
1 Like
Thank you for the information. Do we have to tell dnsmasq to forward requests to it?
That is no different than the original, other than you don't run multiple instances of the proxy. You configure multiple servers in the .toml configuration file and listen on a single port. The proxy then picks the best server from the list.
I only mentioned it because if you previously had multiple servers configured, dnsmasq would listen for each on a different port. Thus, dnsmasq should be reconfigured to no longer listen on the now unused additional ports.