What's new
  • SNBForums Code of Conduct

    SNBForums is a community for everyone, no matter what their level of experience.

    Please be tolerant and patient of others, especially newcomers. We are all here to share and learn!

    The rules are simple: Be patient, be nice, be helpful or be gone!

It should be possible to compromise and attempt removing the entware-setup.sh parts shouldn't it?

Code:
# Remove entware-setup.sh additions
# Usage: entware_clean
entware_clean() {
    # Quote strings suitable for use with sed
    # Usage: sed_quote STRING
    sed_quote() { printf '%s\n' "$1" | sed 's/[]\/$*.^&[]/\\&/g'; }

    if [ -f '/jffs/scripts/services-start' ]; then
        #RC='/opt/etc/init.d/rc.unslung'
        #
        #i=30
        #until [ -x "$RC" ] ; do
        #  i=$(($i-1))
        #  if [ "$i" -lt 1 ] ; then
        #    logger "Could not start Entware"
        #    exit
        #  fi
        #  sleep 1
        #done
        #$RC start
        sed -i "/^$(sed_quote "RC='/opt/etc/init.d/rc.unslung'")$/,/^$(sed_quote "\$RC start")$/d" '/jffs/scripts/services-start'
    fi

    if [ -f '/jffs/scripts/services-stop' ]; then
        #/opt/etc/init.d/rc.unslung stop
        sed -i "/^$(sed_quote '/opt/etc/init.d/rc.unslung stop')/d" '/jffs/scripts/services-stop'
    fi

    if [ -f '/jffs/scripts/post-mount' ]; then
        #if [ -d "$1/entware" ] ; then
        #  ln -nsf $1/entware /tmp/opt
        #fi
        sed -i "/^$(sed_quote "if [ -d \"\$1/entware\" ] ; then")$/,+2d" '/jffs/scripts/post-mount'
    fi

    # Remove empty/broken scripts
    local SCRIPT
    for SCRIPT in services-start services-stop post-mount; do
        [ ! -f "/jffs/scripts/$SCRIPT" ] && continue
        if [ "$(grep -csvE '^[[:space:]]*(#|$)' "/jffs/scripts/$SCRIPT")" = '0' ]; then
            rm -f "/jffs/scripts/$SCRIPT"
        elif ! sh -n "/jffs/scripts/$SCRIPT" 2>/dev/null; then
            echo "Renaming unconvertible script (/jffs/scripts/$SCRIPT -> /jffs/scripts/obsolete_$SCRIPT)"
            mv "/jffs/scripts/$SCRIPT" "/jffs/scripts/obsolete_$SCRIPT"
        fi
    done
}
 
During updating the lists I noticed this in syslog:
Code:
Feb 26 18:33:47 Diversion: http://winhelp2002.mvps.org/hosts.txt not hosts file, no backup found
Feb 26 18:33:48 Diversion: http://www.malwaredomainlist.com/hostslist/hosts.txt not hosts file, no backup found
Feb 26 18:37:19 Diversion: https://pgl.yoyo.org/adservers/serverlist.php?showintro=0;hostformat=hosts not hosts file, no backup found
I never noticed this before, and the links still work in my browser. Why does diversion say so?
 
During updating the lists I noticed this in syslog:
Code:
Feb 26 18:33:47 Diversion: http://winhelp2002.mvps.org/hosts.txt not hosts file, no backup found
Feb 26 18:33:48 Diversion: http://www.malwaredomainlist.com/hostslist/hosts.txt not hosts file, no backup found
Feb 26 18:37:19 Diversion: https://pgl.yoyo.org/adservers/serverlist.php?showintro=0;hostformat=hosts not hosts file, no backup found
I never noticed this before, and the links still work in my browser. Why does diversion say so?
Check your "update-blocking-list.log", it'll give you a clue.
Open diversion, sf-->5
 
During updating the lists I noticed this in syslog:
Code:
Feb 26 18:33:47 Diversion: http://winhelp2002.mvps.org/hosts.txt not hosts file, no backup found
Feb 26 18:33:48 Diversion: http://www.malwaredomainlist.com/hostslist/hosts.txt not hosts file, no backup found
Feb 26 18:37:19 Diversion: https://pgl.yoyo.org/adservers/serverlist.php?showintro=0;hostformat=hosts not hosts file, no backup found
I never noticed this before, and the links still work in my browser. Why does diversion say so?
were those copied over to the file or were they inputted individually in the diversion in the customize list option via ssh terminal? You need to input them individually via the terminal and not manually by copying them into host list file. I noticed this issue when i attempted to copy mine manually to that list as well instead of inputting them individually into the terminal like i should.


if you copied them all to the host list file then diversion assumes you already had a downloaded backup and when it doesn't find it, it gives this error.
 
Last edited:
I removed the lines in diversion and re-added them (as suggested) through the menu. I still get these errors:
Code:
Feb 27 08:23:01 Diversion: http://winhelp2002.mvps.org/hosts.txt not hosts file, no backup found
Feb 27 08:23:01 Diversion: http://www.malwaredomainlist.com/hostslist/hosts.txt not hosts file, no backup found
Feb 27 08:26:05 Diversion: https://pgl.yoyo.org/adservers/serverlist.php?showintro=0;hostformat=hosts not hosts file, no backup found
 
I removed the lines in diversion and re-added them (as suggested) through the menu. I still get these errors:
Code:
Feb 27 08:23:01 Diversion: http://winhelp2002.mvps.org/hosts.txt not hosts file, no backup found
Feb 27 08:23:01 Diversion: http://www.malwaredomainlist.com/hostslist/hosts.txt not hosts file, no backup found
Feb 27 08:26:05 Diversion: https://pgl.yoyo.org/adservers/serverlist.php?showintro=0;hostformat=hosts not hosts file, no backup found
While evaluating a new feature I found that
- http://winhelp2002.mvps.org/hosts.txt and
- http://www.malwaredomainlist.com/hostslist/hosts.txt
has DOS line endings which is then incorrectly interpreted as a non-valid file and rejected.
This is fixed locally and will be available with the next Diversion update.

https://pgl.yoyo.org/adservers/serverlist.php?showintro=0;hostformat=hosts is not a valid format, use this instead:
https://pgl.yoyo.org/adservers/serv...&showintro=0&mimetype=plaintext&useip=0.0.0.0
 
Diversion, unfortunately, is extremely sloppy with scripts installed before it - it just deletes them. I mean post-mount and write to the crontab. You can also check the presence of previously installed programs during installation and not touch the records made earlier?
If the known Entware user scripts contain remnants or part of the old-style Entware code, Diversion will remove that file to have a clean slate.
Yours must have had such entries.
See also below.

It should be possible to compromise and attempt removing the entware-setup.sh parts shouldn't it?

Code:
# Remove entware-setup.sh additions
# Usage: entware_clean
entware_clean() {
    # Quote strings suitable for use with sed
    # Usage: sed_quote STRING
    sed_quote() { printf '%s\n' "$1" | sed 's/[]\/$*.^&[]/\\&/g'; }

    if [ -f '/jffs/scripts/services-start' ]; then
        #RC='/opt/etc/init.d/rc.unslung'
        #
        #i=30
        #until [ -x "$RC" ] ; do
        #  i=$(($i-1))
        #  if [ "$i" -lt 1 ] ; then
        #    logger "Could not start Entware"
        #    exit
        #  fi
        #  sleep 1
        #done
        #$RC start
       sed -i "/^$(sed_quote "RC='/opt/etc/init.d/rc.unslung'")$/,/^$(sed_quote "\$RC start")$/d" '/jffs/scripts/services-start'
    fi

    if [ -f '/jffs/scripts/services-stop' ]; then
        #/opt/etc/init.d/rc.unslung stop
        sed -i "/^$(sed_quote '/opt/etc/init.d/rc.unslung stop')/d" '/jffs/scripts/services-stop'
    fi

    if [ -f '/jffs/scripts/post-mount' ]; then
        #if [ -d "$1/entware" ] ; then
        #  ln -nsf $1/entware /tmp/opt
        #fi
        sed -i "/^$(sed_quote "if [ -d \"\$1/entware\" ] ; then")$/,+2d" '/jffs/scripts/post-mount'
    fi

    # Remove empty/broken scripts
    local SCRIPT
    for SCRIPT in services-start services-stop post-mount; do
        [ ! -f "/jffs/scripts/$SCRIPT" ] && continue
        if [ "$(grep -csvE '^[[:space:]]*(#|$)' "/jffs/scripts/$SCRIPT")" = '0' ]; then
            rm -f "/jffs/scripts/$SCRIPT"
        elif ! sh -n "/jffs/scripts/$SCRIPT" 2>/dev/null; then
            echo "Renaming unconvertible script (/jffs/scripts/$SCRIPT -> /jffs/scripts/obsolete_$SCRIPT)"
            mv "/jffs/scripts/$SCRIPT" "/jffs/scripts/obsolete_$SCRIPT"
        fi
    done
}
Thanks for the code. I have thought for about a second implementing something like this into Diversion when I rewrote AB-Solution to what became Diversion.
Since I used a much more elaborate jffs code in AB-Solution to cater for almost every (user)configuration, the resulting Entware code in the /jffs/scripts/ folder could be in a number of different formats.

Expecting that after a certain string 2 more lines with the accompanying code follows is not a given. Your code example relies on that. Unfortunately, this would result in remnants or broken user scripts.
I wanted and still do avoid this by simply removing files containing such old-style Entware entries.
My decision back in early 2018 to go that route was obviously the right choice. There are few if any complaints of users saying Entware did not start.
 
You can lead a horse to water, they say. That aside.. if you know enough, and care, about how your event scripts work, you should surely be able to understand what the installation messages mean pertaining to them. Would everyone understand? Of course not, which brings us to the answer to the last question - support.

At the risk of putting words into someone else’s mouth, the install process for diversion (and entware for that matter) create a known good configuration file without pulling in someone’s potential nonsense that breaks things. This is the best approach for the common user and the easiest for the script writer to “support” (i.e. provide assistance when something goes wrong). It’s implausible to accommodate all possible edits/configurations people could be trying to bring in because it’s impossible to support.

The diversion project to me seeks to be the lowest common denominator for as wide an audience as possible. If you don’t like the way it installs, your choices as I see it would be to either reinstate your personal changes or not install it to begin with.
Well said, thanks.
 
Expecting that after a certain string 2 more lines with the accompanying code follows is not a given. Your code example relies on that. Unfortunately, this would result in remnants or broken user scripts.
I wanted and still do avoid this by simply removing files containing such old-style Entware entries.
My decision back in early 2018 to go that route was obviously the right choice. There are few if any complaints of users saying Entware did not start.

It should be pretty safe to assume people are using entware-setup.sh code, any broken scripts would be caught with sh -n. The bigger problem would be the services-start sed statement not having a matching end block which would mean the file gets truncated. This could be fixed with
Code:
sed -ni "/^$(sed_quote "RC='/opt/etc/init.d/rc.unslung'")$/{h;bx;};p;b;:x;n;/^$(sed_quote "\$RC start")$/d;H;\${x;p};bx" '/jffs/scripts/services-start'

But if you want a completely fail-safe method you'd want to match the block of code exactly. Should be possible in sed but.. it'd be ugly. The easiest way is to just load the script into a variable and use the built in replace.

Code:
SCRIPT="$(cat /jffs/scripts/services-start; printf _)"
SCRIPT="${SCRIPT%_}"

REMOVE='
RC='\''/opt/etc/init.d/rc.unslung'\''

i=30
until [ -x "$RC" ] ; do
  i=$(($i-1))
  if [ "$i" -lt 1 ] ; then
    logger "Could not start Entware"
    exit
  fi
  sleep 1
done
$RC start
'

NEWLINE='
'

printf '%s' "${SCRIPT//"$REMOVE"/"$NEWLINE"}" >/jffs/scripts/services-start
 
It should be pretty safe to assume people are using entware-setup.sh code, any broken scripts would be caught with sh -n. The bigger problem would be the services-start sed statement not having a matching end block which would mean the file gets truncated. This could be fixed with
Code:
sed -ni "/^$(sed_quote "RC='/opt/etc/init.d/rc.unslung'")$/{h;bx;};p;b;:x;n;/^$(sed_quote "\$RC start")$/d;H;\${x;p};bx" '/jffs/scripts/services-start'

But if you want a completely fail-safe method you'd want to match the block of code exactly. Should be possible in sed but.. it'd be ugly. The easiest way is to just load the script into a variable and use the built in replace.

Code:
SCRIPT="$(cat /jffs/scripts/services-start; printf _)"
SCRIPT="${SCRIPT%_}"

REMOVE='
RC='\''/opt/etc/init.d/rc.unslung'\''

i=30
until [ -x "$RC" ] ; do
  i=$(($i-1))
  if [ "$i" -lt 1 ] ; then
    logger "Could not start Entware"
    exit
  fi
  sleep 1
done
$RC start
'

NEWLINE='
'

printf '%s' "${SCRIPT//"$REMOVE"/"$NEWLINE"}" >/jffs/scripts/services-start
Thanks but I'll leave it as it is, for the sanity of me and the enthusiastic and helpful board members minds (thanks guys / gals !).
 
@Rhialto, you will need to reboot the router. :)
 
While evaluating a new feature I found that
- http://winhelp2002.mvps.org/hosts.txt and
- http://www.malwaredomainlist.com/hostslist/hosts.txt
has DOS line endings which is then incorrectly interpreted as a non-valid file and rejected.
It’s been long enough ago that I forget the culprit, but one of the popular lists at the time was including a very unusual character (it looked like a “G” and was blocking a fake Google.com) in one of its blocked domains that was making dnsmasq fail to load similarly. I went to a sed line to strip the non standard stuff, I’m curious about how you handle it.
Code:
sed -i 's/[^a-zA-Z0-9#-.]//g' blocklist
 
Can someone explain me how does primary and secondary lists work?
The secondary one works as a backup if the primary doesn't work?
Or are they used both at same time?

Thanks in advance
 
Hello.
I am not a CLI fan but I can manage some things.
I updated the latest fw of Merlin on RT-AC66Ub1.

So I played with amtm interface and features.
I installed Diversion. Everything seemed ok with the a d b l o c k e r but...
The Youtube android app seems not updating the history/recent of the played videos (in the library/history section)
On the PC/laptop browser it is working correctly.
I did not checked the situation of accessing YT from phone browser.

I disabled the a d b l o c k e r via amtm menu and the played videos appeared in the history.
And the YT app works fine now.

I don't know if somebody else observed this issue.
Can somebody check this?
 
Last edited:
Hello.
I am not a CLI fan but I can manage some things.
I updated the latest fw of Merlin on RT-AC66Ub1.

So I played with amtm interface and features.
I installed Diversion. Everything seemed ok with the a d b l o c k e r but...
The Youtube android app seems not updating the history/recent of the played videos (in the library/history section)
On the PC/laptop browser it is working correctly.
I did not checked the situation of accessing YT from phone browser.

I disabled the a d b l o c k e r via amtm menu and the played videos appeared in the history.
And the YT app works fine now.

I don't know if somebody else observed this issue.
Can somebody check this?


From the pie hole page they list some whitelist items to enable YouTube history to work.


https://discourse.pi-hole.net/t/commonly-whitelisted-domains/212

Maybe a good link to add to the original page as a sticky.


There is also a whitelist shared here for these items:

https://github.com/anudeepND/whitelist/blob/master/README.md
 
Whitelisting device-metrics-us.amazon.com and rebooting my echos solved it.
I have a ONeplus 7T, had problems with the Amazon Shopping app, in addition I had to add:
Code:
device-metrics-us.amazon.com
aan.amazon.com
fls-na.amazon.com
 

Similar threads

Support SNBForums w/ Amazon

If you'd like to support SNBForums, just use this link and buy anything on Amazon. Thanks!

Sign Up For SNBForums Daily Digest

Get an update of what's new every day delivered to your mailbox. Sign up here!
Top