Quick post. Figuring out awk has driven me a little nuts and I’m running low on vodka.

I’ve been trying to cross compile FTLdns from the Pi-Hole project to run on top of the Orbi. Needless to say, that’s been a bit of a project.

While researching the possibility of running Pi-Hole natively on an OpenWRT platform, I’ve discovered that there’s an adblocker that runs natively on OpenWRT. This is to be found in the adblock package from friendly neighborhood OpenWRT repository.

If there’s one thing that’s less fun than figuring out awk, it is trying to decode someone else’s shell scripts.

After a couple false starts (couple? HAHAHAHAHA) what I figured out is that the adblock package has a file, adblock.conf, that contains a list of repositories of adservers, trackers, and the like. The stanzas of adblock.conf look like this:

config source 'adaway' option enabled '1' option adb_src 'https://adaway.org/hosts.txt' option adb_src_rset '\$0~/^127\.0\.0\.1[[:space:]]+([[:alnum:]_-]+\.){1,}[[:alpha:]]+([[:space:]]|$)/{print tolower(\$2)}' option adb_src_desc 'focus on mobile ads, infrequent updates, approx. 400 entries'

You guys can figure it out: source name, whether enabled or not, the URL of the list, a customized awk one-liner to extract a hostname list, and a brief summary.

The management of this package is integrated into OpenWRT’s luci web gui. We don’t have that on the Orbi, but we do have ssh, and we’re goddamn happy to live like real men and women, using the command line like $DEITY intended us to.

This is a 90% solution, at the moment. It works for me, and I’d like to get it on WordPress so, in the case of a firmware update that nukes all my customizations, I have a backup somewhere.

I’ve changed some things around. I’m using a blank partition that was hidden on the Orbi, mounted on /usr/local, and have re-written all my scripts to be started from /usr/local/init. If you’re using /mnt/circle and /mnt/bitdefender like I was a while ago you’re going to have to alter the appropriate lines to make things work.

I’ve downloaded the adblock package from here, and extracted it in /usr/local/adblock. Edit /usr/local/adblock/adblock.conf and enable the repositories you want to by changing the enabled option to 1.

I made this shell script to parse the adblock.conf file. You’ll need to create the directory /usr/local/adblock/tmp. I call it read_adblockconf.sh:

#!/bin/sh DIR=/usr/local/adblock CONF=$DIR/adblock.conf TMPLIST=$DIR/tmp/tmplist echo > $TMPLIST BLOCKLIST=$DIR/blocklist SOURCENAME= URL= CONFIG= ENABLED=0 OPTION= FILTER= while read -r line; do FIRST=$( echo $line | cut -f 1 -d ' ' ) SECOND=$( echo $line | cut -f 2 -d ' ' ) THIRD=$( echo $line | cut -f 3- -d ' ' ) if [ $FIRST ] && [ $FIRST = 'config' ] && \ [ $SECOND ] && [ $SECOND = 'source' ]; then SOURCENAME=$THIRD fi if [ $SOURCENAME ] && \ [ $FIRST ] && [ $FIRST = 'option' ] && \ [ $SECOND ] && [ $SECOND = 'enabled' ] && \ [ $THIRD ] && [ $(echo $THIRD | grep 1) ]; then echo $SOURCENAME is enabled ENABLED=1 fi if [ $SOURCENAME ] && \ [ $ENABLED -eq 1 ] && \ [ $FIRST ] && [ $FIRST = 'option' ] && \ [ $SECOND ] && [ $SECOND = 'adb_src' ] && \ [ $THIRD ]; then URL=$THIRD fi if [ $SOURCENAME ] && \ [ $ENABLED -eq 1 ] && \ [ $FIRST ] && [ $FIRST = 'option' ] && \ [ $SECOND ] && [ $SECOND = 'adb_src_rset' ] && \ [ ! -z "$THIRD" ]; then eval "FILTER=\"${THIRD}\"" eval "wget -O - $URL | awk ${FILTER} >> $TMPLIST" SOUCENAME= URL= ENABLED=0 FILTER= fi done < $CONF echo > $BLOCKLIST while read -r hostname; do echo 127.0.0.1 $hostname >> $BLOCKLIST done < $TMPLIST

Briefly, what this does is downloads and parses the blocklists into /usr/local/adblock/blocklist, which is a list of names (in /etc/hosts format) to resolve to localhost (127.0.0.1), to prevent your browser from resolving them to an ip address on the actual internet.

Now, I have to alter my fix_dns.sh script from a prior blog entry, in order to grab the blocklist file and concatenate into /etc/hosts:

#!/bin/sh UPDATE_INTERVAL=15 echo "#!/bin/sh md5=0 updatehost () { md5=\`md5sum /tmp/udhcpd_clients | cut -f 1 -d ' '\` echo 127.0.0.1 localhost > /etc/hosts echo 192.168.86.1 orbi >> /etc/hosts echo 192.168.86.3 albedo >> /etc/hosts while read -r LINE; do echo \$( echo \$LINE | cut -f 2 -d \; ) \$( echo \$LINE | cut -f 3 -d \; | tr '[ABCDEFGHIJKLMNOPQRSTUVWXYZ]' '[abcdefghijklmnopqrstuvwxyz]' ) >> /etc/hosts done < /tmp/udhcpd_clients if [ -f /usr/local/adblock/blocklist ]; then while read -r LINE; do echo \$LINE >> /etc/hosts done < /usr/local/adblock/blocklist fi killall -SIGHUP dnsmasq } initialize () { name=\`basename \"\$0\"\` mypid=\$\$ pid=\`pidof \$name\` if [ ! \"\$mypid\" == \"\$pid\" ]; then for i in \$pid; do if [ \$i != \$mypid ]; then echo killing pid \$i kill \$i fi done fi } initialize updatehost while ( true ); do tmpmd5=\`md5sum /tmp/udhcpd_clients | cut -f 1 -d ' '\` if [ \$md5 != \$tmpmd5 ]; then md5=\$tmpmd5 echo change detected. updating... updatehost fi sleep $UPDATE_INTERVAL done" > /tmp/updatehosts.sh chmod a+x /tmp/updatehosts.sh /tmp/updatehosts.sh & cp /etc/dnsmasq.conf /tmp/dnsmasq.conf.old sed -e 's/#*no-hosts/#no-hosts/' < /tmp/dnsmasq.conf.old > /etc/dnsmasq.conf /etc/init.d/dnsmasq restart

The only new thing here is the stanza:

if [ -f /usr/local/adblock/blocklist ]; then while read -r LINE; do echo \$LINE >> /etc/hosts done < /usr/local/adblock/blocklist fi killall -SIGHUP dnsmasq

If the script finds the file /usr/local/adblock/blocklist, it concatenates it into /etc/hosts, then forces dnsmasq to re-read /etc/hosts by passing it a SIGHUP.

To do: run read_adblockconf.sh on a regular basis to update the blockfile.