Adblock support thread

@felipee07 memory optimized adblock version 2.3.2 is now in trunk, thanks for your pre-tests!

1 Like

I've uploaded a new experimental pre-release 2.3.99-pre0 to my google drive account (see first post). There is only one major change:

  • add tld compression,
    remove bloated host entries from block lists
    and save up to 10 MByte RAM plus better performance for dns backends

The new "top level domain compression" removes up to 40 thousand needless host entries from the block lists and lowered the memory footprint for the dns backends by 8-10 MByte. :sunglasses:

Example (without compression):

Sun Feb 26 20:13:56 2017 user.notice adblock-[2.3.99-pre0] info : start adblock processing ...
Sun Feb 26 20:14:52 2017 user.notice adblock-[2.3.99-pre0] info : block lists with overall 148505 domains loaded successfully (LEDE Reboot SNAPSHOT r3544-55ffc38004)

Example (with compression):

Sun Feb 26 20:08:43 2017 user.notice adblock-[2.3.99-pre0] info : start adblock processing ...
Sun Feb 26 20:10:00 2017 user.notice adblock-[2.3.99-pre0] info : block lists with overall 112030 domains loaded successfully (LEDE Reboot SNAPSHOT r3544-55ffc38004)

Example for list adaway (uncompressed vs. compressed):

file:///home/dirk/Pictures/compress.png

Happy testing! :slight_smile:
Dirk

1 Like

I found a small error in my log files:

Tue Mar 14 06:00:00 2017 user.notice adblock-[2.4.0-2] info : start adblock processing ...
Tue Mar 14 06:03:16 2017 daemon.err adblock.sh[1930]: awk: bad regex 'se[server.d9ae99824.': Missing ']'
Tue Mar 14 06:04:05 2017 user.notice adblock-[2.4.0-2] info : block lists with overall 84480 domains loaded successfully (LEDE Reboot 17.01.0 r3205-59508e3)

Yes, these errors are winhelp related, the latest update of this list includes several typos like that ...

wrong: > 0.0.0.0 d9ae99824.se[server down?]
correct: > 0.0.0.0 d9ae99824.se #[server down?]

I've already informed the author of the list! :relaxed:
Anyway, the adblock "error" is harmless ... it just says that awk can't process this single record - the rest of the list is OK. I'll mute those warnings with the next release.

1 Like

@brokenpipe
The winhelp author fixed his list a few minutes ago.

The blocklists that are enabled by default, does it also block ads that are used on youtube or do I manually need to add them to the blacklist?

@dibdot Dirk, I have installed your latest 2.5.0. Congrats on reaching another great milestone with your adblock tool!

I do have one question to clarify. I see that major changes has occurred with regard to adblock and luci. I can't seem to find luci-app-adblock package anymore in the snapshots. Does this mean that luci-app-adblock is built directly into official luci now? Or is that package failing to compile?

Thank you.

It was failing to compile to a missing like in Makefile. The luci app "disappeared". It should be fixed now.

Try building again after selecting it again in menuconfig.

Sorry for any inconvenience that this faulty commit may have caused.
Thanks @hnyman that adblock 2.5.0 finally reached LEDE trunk (with new LuCI frontend)! :wink:

To get a quick impression of the new GUI take a look ...

  • New overview page with additional runtime information plus Suspend/Resume button:

  • In "Advanced" section you can directly edit blacklist / whitelist and your adblock configuration:

  • Also you can query the adblock files for certain domains:

Hope you enjoy it!
Dirk

3 Likes

Thank you both. I've got the package now but it seems to be creating an error:

From: http://192.168.1.1/cgi-bin/luci/admin/services/adblock

/usr/lib/lua/luci/dispatcher.lua:460: Failed to execute cbi dispatcher target for entry '/admin/services/adblock'.
The called action terminated with an exception:
/usr/lib/lua/luci/cbi.lua:53: Model 'adblock' not found!
stack traceback:
	[C]: in function 'assert'
	/usr/lib/lua/luci/dispatcher.lua:460: in function 'dispatch'
	/usr/lib/lua/luci/dispatcher.lua:141: in function </usr/lib/lua/luci/dispatcher.lua:140>

This is latest luci-app-adblock_git-17.085.52855-65a089d-1_all.ipk from snapshots directory.

probably a luci caching issue, try the following:

/etc/init.d/uhttpd stop
rm -f /tmp/luci-indexcache
rm -rf /tmp/luci-modulecache
/etc/init.d/uhttpd start

Your suspicion of my issue being a caching issue was correct and the commands which you specific worked perfectly. Thank you, sir.

By the way, this new user interface is a dream come true. All of the logging and features all within well organized tabs. Great work by yourself and @hnyman

Hi
I have a some questions about the tld compression feature.
With 2.5 it is enabled by default now?

Does it only compress when the top domain is in the list of blocked domains? Otherwise it would be quite bad?
For example:
adhost1.trusteddomain.com
adhost2.trusteddomain.com
adhost3.trusteddomain.com
.....
So it gets compressed to trusteddomain.com?
But i dont want trusteddomain.com to be blocked.

No, it's enabled by default since release 2.4.0.

yes, see example screenshot a few posts above.

thank you!

My whitelist and blacklist aren't working.
Config file is default.
Any hints where should I look for?

Wed Mar 29 18:31:09 2017 user.notice adblock-[2.5.0] info : start adblock processing ...
Wed Mar 29 18:31:09 2017 user.notice adblock-[2.5.0] debug: name: adaway, enabled: 1, backup: 0, dns: dnsmasq, fetch: /usr/bin/wget, memory: 513280
Wed Mar 29 18:31:10 2017 user.notice adblock-[2.5.0] debug: name: adaway, mode: backup, count: 379, in_rc: 0, out_rc: 0
Wed Mar 29 18:31:10 2017 user.notice adblock-[2.5.0] debug: name: adguard, mode: remove, count: 0, in_rc: 0, out_rc: 0
Wed Mar 29 18:31:10 2017 user.notice adblock-[2.5.0] debug: name: blacklist, mode: remove, count: 0, in_rc: 0, out_rc: 0
Wed Mar 29 18:31:10 2017 user.notice adblock-[2.5.0] debug: name: disconnect, mode: remove, count: 0, in_rc: 0, out_rc: 0
Wed Mar 29 18:31:10 2017 user.notice adblock-[2.5.0] debug: name: dshield, mode: remove, count: 0, in_rc: 0, out_rc: 0
Wed Mar 29 18:31:10 2017 user.notice adblock-[2.5.0] debug: name: feodo, mode: remove, count: 0, in_rc: 0, out_rc: 0
Wed Mar 29 18:31:10 2017 user.notice adblock-[2.5.0] debug: name: hphosts, mode: remove, count: 0, in_rc: 0, out_rc: 0
Wed Mar 29 18:31:10 2017 user.notice adblock-[2.5.0] debug: name: malware, mode: remove, count: 0, in_rc: 0, out_rc: 0
Wed Mar 29 18:31:10 2017 user.notice adblock-[2.5.0] debug: name: malwarelist, mode: remove, count: 0, in_rc: 0, out_rc: 0
Wed Mar 29 18:31:10 2017 user.notice adblock-[2.5.0] debug: name: openphish, mode: remove, count: 0, in_rc: 0, out_rc: 0
Wed Mar 29 18:31:10 2017 user.notice adblock-[2.5.0] debug: name: palevo, mode: remove, count: 0, in_rc: 0, out_rc: 0
Wed Mar 29 18:31:10 2017 user.notice adblock-[2.5.0] debug: name: ransomware, mode: remove, count: 0, in_rc: 0, out_rc: 0
Wed Mar 29 18:31:10 2017 user.notice adblock-[2.5.0] debug: name: rolist, mode: remove, count: 0, in_rc: 0, out_rc: 0
Wed Mar 29 18:31:10 2017 user.notice adblock-[2.5.0] debug: name: ruadlist, mode: remove, count: 0, in_rc: 0, out_rc: 0
Wed Mar 29 18:31:10 2017 user.notice adblock-[2.5.0] debug: name: securemecca, mode: remove, count: 0, in_rc: 0, out_rc: 0
Wed Mar 29 18:31:10 2017 user.notice adblock-[2.5.0] debug: name: shalla, mode: remove, count: 0, in_rc: 0, out_rc: 0
Wed Mar 29 18:31:10 2017 user.notice adblock-[2.5.0] debug: name: spam404, mode: remove, count: 0, in_rc: 0, out_rc: 0
Wed Mar 29 18:31:10 2017 user.notice adblock-[2.5.0] debug: name: sysctl, mode: remove, count: 0, in_rc: 0, out_rc: 0
Wed Mar 29 18:31:10 2017 user.notice adblock-[2.5.0] debug: name: whocares, mode: remove, count: 0, in_rc: 0, out_rc: 0
Wed Mar 29 18:31:10 2017 user.notice adblock-[2.5.0] debug: name: winspy, mode: remove, count: 0, in_rc: 0, out_rc: 0
Wed Mar 29 18:31:10 2017 user.notice adblock-[2.5.0] debug: name: winhelp, mode: remove, count: 0, in_rc: 0, out_rc: 0
Wed Mar 29 18:31:10 2017 user.notice adblock-[2.5.0] debug: name: yoyo, mode: remove, count: 0, in_rc: 0, out_rc: 0
Wed Mar 29 18:31:10 2017 user.notice adblock-[2.5.0] debug: name: zeus, mode: remove, count: 0, in_rc: 0, out_rc: 0
Wed Mar 29 18:40:58 2017 user.notice adblock-[2.5.0] info : block lists with overall 379 domains loaded successfully (LEDE-BETA Reboot SNAPSHOT r3792-0685f2a)

Yes, please enable the blacklist source in your config (it's disabled by default).
If your whitelist does not work, please post your whitelist entries - thanks.

@dibdot good work with the procd and LuCI updates.

Few minutes ago, I've pushed release 2.6.0 to package snapshot repo. The new release includes the following changes:

- no longer misuse ubus/procd service object for adblock runtime
  information, now save all required information directly
  in a JSON file/format (/tmp/adb_runtime.json)
- new 'status' init command to print runtime information
- add a configurable interface trigger timeout for
  nested or slow modem/router setups,
  set 'adb_triggerdelay' accordingly (default 1 second)
- add support for pure http download utilities like wget-nossl
  or uclient-fetch without libustream-ssl (http downloads only!)
- fix stop action
- fix enabled/disabled action
- fix country code in regional list for china

LuCI frontend was updated too ...

Have fun!
Dirk

3 Likes

I just updated to adblock_2.6.0-1 from 2.5.1-1 & luci-app-adblock_git-17.105.47818-76ee1ba-1 from 17.099.31049-a162384-1 and got this error going to Services > Adblock:

/usr/lib/lua/luci/dispatcher.lua:433: Failed to execute firstchild dispatcher target for entry '/admin/services/adblock'. The called action terminated with an exception: /usr/lib/lua/luci/dispatcher.lua:433: Failed to execute cbi dispatcher target for entry '/admin/services/adblock/tab_from_cbi'. The called action terminated with an exception: /usr/lib/lua/luci/model/cbi/adblock/overview_tab.lua:7: module 'luci.jsonc' not found: no field package.preload['luci.jsonc'] no file './luci/jsonc.lua' no file '/usr/share/lua/luci/jsonc.lua' no file '/usr/share/lua/luci/jsonc/init.lua' no file '/usr/lib/lua/luci/jsonc.lua' no file '/usr/lib/lua/luci/jsonc/init.lua' no file './luci/jsonc.so' no file '/usr/lib/lua/luci/jsonc.so' no file '/usr/lib/lua/loadall.so' no file './luci.so' no file '/usr/lib/lua/luci.so' no file '/usr/lib/lua/loadall.so' stack traceback: [C]: in function 'assert' /usr/lib/lua/luci/dispatcher.lua:433: in function 'dispatch' /usr/lib/lua/luci/dispatcher.lua:168: in function </usr/lib/lua/luci/dispatcher.lua:167>

Clearing the luci index & module cache worked before when this happened, but not this time. I also tried removing & reinstalling both packages.

GLi GL-ARM300M
OpenWrt Chaos Calmer 15.05 / LuCI 15.05-54-gfd60055 Release (git-15.363.78009-956be55)