Solved Unable to log into router (AX86U) after reboot

[iTyPsIDg]

I have everything I'm running in my signature. I added dnsmon to init-start. Yesterday, I decided to give the GeForce NOW QoS a try, I lost access to the router and ended up having to wipe the settings (WPS on boot method) and rebuild from scratch. Today, I lost power for about 20 seconds and the same thing happened, but I'm using Cake QoS. As a result, I don't think this has anything to do with QoS.

I followed eibgrad's recommendations here, along with his dnsmon tool, to ensure that all of my DNS traffic goes through my VPN. I want all traffic to go through the VPN so streaming services don't block me; I'm a remote worker from the US but I travel everywhere with the router.

Is init-start the wrong place for the script? I'm trying to figure out where I should launch vpnmon and dnsmon via screen for automatic monitoring upon reboot. I also want to make sure I can still access the router if a reboot happens.

Do I need to wipe again just to get internet access? It seems bizarre that I can't access the GUI or SSH to the router after this reboot. I want to prevent having to rebuild constantly when a power blip happens.

SOLVED: http://www.snbforums.com/threads/unable-to-log-into-router-ax86u-after-reboot.77831/post-751296

 

[eibgrad]

Seems to me you're describing two different issues; the fact you have problems accessing the router after a power failure, and when enabling QoS, vs. how to configure dnsmon and vpnmon. On the face of it, they don't seem related.

Frankly, I could have done a better job describing the process of using screen to run dnsmon. But that wasn't the primary focus of the utility at the time.

Since Entware is typically installed to USB, it means you can't just assume the Entware directories will be available at the time the init-start script is started. You have to WAIT until the USB drive and /opt directories are mounted. And you can do that one of two ways; either use a post-mount script, or wait in the init-start script for the /opt/sbin/screen utility to be accessible (notice this requires the full path to screen as well). The following uses the init-start script.

(
until [ -f /opt/sbin/screen ]; do sleep 10; done
/opt/sbin/screen -dmS dnsmon /jffs/scripts/merlin-dns-monitor.sh
) &

I'll leave the alternative (using a post-mount script) as an exercise for the user.

What's between the parentheses ( ) will run in the background and wait for the screen utility to become available, then start dnsmon. In the meantime, the rest of the init-start script continues normally. Notice too I include the full path to screen, since it's part of Entware. I don't believe the system path includes any of the /opt directories. I updated the dnsmon documentation to reflect that need for the full path.

But again, I don't see how any of these dnsmon issues would impact your ability to gain access to the router when it comes to those other issues.

 

[iTyPsIDg]

I'll try a restart tomorrow to see if I have the same issue. I can't be sure, but I think changing the QoS setting yesterday caused the router to restart. In the interest of giving all the details, I included what steps I took.

Only three things are different since this issue started. I'm mainly hoping to narrow it down so I can figure out the cause. VPNMON is new, but I don't have it configured to start automatically. DNSMON is new, and I did have it set to start automatically in init-start as follows:

screen -dmS dnsmon /jffs/scripts/merlin-dns-monitor.sh

I found your DNSMON script because I wanted to solve the issue with Netflix knowing I wasn't in the US. Oddly, Netflix would seem to indicate I was in the US (Top 10 shows in the US); however, it wouldn't find certain titles that I know should show up for the US. So I also configured the routes in VPN client 1 custom configuration:

route 103.86.96.100
route 103.86.99.100

In addition, I set the VPN Director Policy to route traffic to those IPs through the VPN.

At present, I don't have DNSMON or VPNMON set to automatically start. I imagine that if the issue occurs again, it could be related to the routes for DNS to go through the VPN? Maybe if the VPN connection doesn't start then it hangs my configuration? Or maybe it was because, as you said, screen is on the USB and I tried using it too early. If I could have ever logged in to the router to check, I could learn more about where the problem occurred.

I rebuilt again today since people needed to come over and use internet for a call. They've been paying $15/day to go to a coworking space nearby.

Thank you for the updated command for init-start.

 

[Martinski]

Since Entware is typically installed to USB, it means you can't just assume the Entware directories will be available at the time the init-start script is started. You have to WAIT until the USB drive and /opt directories are mounted ...

(
until [ -f /opt/sbin/screen ]; do sleep 10; done
/opt/sbin/screen -dmS dnsmon /jffs/scripts/merlin-dns-monitor.sh
) &

As coded above, there's a chance that the "until" loop may become an endless loop if the attached USB drive partition where Entware was installed fails to mount for any reason. Yes, it might be a small chance of failure, but the possibility is not zero. It won't affect the init-start script execution because the loop is in a section of code that is enclosed in parentheses followed by the ampersand symbol so it gets invoked in the background and the script continues, as @eibgrad already pointed out.

One possible solution is to have a counter/timer within the "until" loop so that if a reasonably long timeout is ever reached, we exit the loop regardless of the state of the other conditional. Then you must check again if the Entware binary indeed exists.

Example:

(
# Set Maximum Timeout value as you see fit #
MaxTimeout=600
SleepSecs=2
TimerSecs=0

OptBinFile="/opt/sbin/screen"
ScriptFile="/jffs/scripts/merlin-dns-monitor.sh"

until [ -f $OptBinFile ] || [ $TimerSecs -ge $MaxTimeout ]
do sleep $SleepSecs ; TimerSecs=$(($TimerSecs + $SleepSecs)) ; done

if [ ! -f $OptBinFile ]
then echo "ERROR: [$OptBinFile] NOT found." ; exit 1 ; fi

echo "Launching $ScriptFile"
$OptBinFile -dmS dnsmon $ScriptFile
) &

As a general rule, once I have this many lines of code, I prefer to put the whole section into its own script file. As a bonus, now the new script can be called from any of the event-driven scripts that fit the purpose. Also, more code can be added to the new script so it can actually log useful messages in case of specific errors/failures.

Example "call-dns-monitor.sh" script:

#!/bin/sh
########################
## call-dns-monitor.sh
########################

LogCmd="/usr/bin/logger -sct "$0_$$""

_DoExit_() { $LogCmd "EXIT $1" ; exit $1 ; }

$LogCmd "START"

# Set Maximum Timeout value as you see fit #
MaxTimeout=600
SleepSecs=2
TimerSecs=0

OptBinFile="/opt/sbin/screen"
ScriptFile="/jffs/scripts/merlin-dns-monitor.sh"

if [ ! -f $ScriptFile ]
then $LogCmd "ERROR: [$ScriptFile] NOT found." ; _DoExit_ 1 ; fi

until [ -f $OptBinFile ] || [ $TimerSecs -ge $MaxTimeout ]
do sleep $SleepSecs ; TimerSecs=$(($TimerSecs + $SleepSecs)) ; done

if [ ! -f $OptBinFile ]
then $LogCmd "ERROR: [$OptBinFile] NOT found." ; _DoExit_ 2 ; fi

$LogCmd "Launching $ScriptFile"
$OptBinFile -dmS dnsmon $ScriptFile

_DoExit_ 0

#EOF#

Example call from within another script:

if [ -f /jffs/scripts/call-dns-monitor.sh ]
then /jffs/scripts/call-dns-monitor.sh & ; fi

 

[iTyPsIDg]

As coded above, there's a chance that the "until" loop may become an endless loop if the attached USB drive partition where Entware was installed fails to mount for any reason. Yes, it might be a small chance of failure, but the possibility is not zero. It won't affect the init-start script execution because the loop is in a section of code that is enclosed in parentheses followed by the ampersand symbol so it gets invoked in the background and the script continues, as @eibgrad already pointed out.

One possible solution is to have a counter/timer within the "until" loop so that if a reasonably long timeout is ever reached, we exit the loop regardless of the state of the other conditional. Then you must check again if the Entware binary indeed exists.

Example:

(
# Set Maximum Timeout value as you see fit #
MaxTimeout=600
SleepSecs=2
TimerSecs=0

OptBinFile="/opt/sbin/screen"
ScriptFile="/jffs/scripts/merlin-dns-monitor.sh"

until [ -f $OptBinFile ] || [ $TimerSecs -ge $MaxTimeout ]
do sleep $SleepSecs ; TimerSecs=$(($TimerSecs + $SleepSecs)) ; done

if [ ! -f $OptBinFile ]
then echo "ERROR: [$OptBinFile] NOT found." ; exit 1 ; fi

echo "Launching $ScriptFile"
$OptBinFile -dmS dnsmon $ScriptFile
) &

As a general rule, once I have this many lines of code, I prefer to put the whole section into its own script file. As a bonus, now the new script can be called from any of the event-driven scripts that fit the purpose. Also, more code can be added to the new script so it can actually log useful messages in case of specific errors/failures.

Example "call-dns-monitor.sh" script:

#!/bin/sh
########################
## call-dns-monitor.sh
########################

LogCmd="/usr/bin/logger -sct "$0_$$""

_DoExit_() { $LogCmd "EXIT $1" ; exit $1 ; }

$LogCmd "START"

# Set Maximum Timeout value as you see fit #
MaxTimeout=600
SleepSecs=2
TimerSecs=0

OptBinFile="/opt/sbin/screen"
ScriptFile="/jffs/scripts/merlin-dns-monitor.sh"

if [ ! -f $ScriptFile ]
then $LogCmd "ERROR: [$ScriptFile] NOT found." ; _DoExit_ 1 ; fi

until [ -f $OptBinFile ] || [ $TimerSecs -ge $MaxTimeout ]
do sleep $SleepSecs ; TimerSecs=$(($TimerSecs + $SleepSecs)) ; done

if [ ! -f $OptBinFile ]
then $LogCmd "ERROR: [$OptBinFile] NOT found." ; _DoExit_ 2 ; fi

$LogCmd "Launching $ScriptFile"
$OptBinFile -dmS dnsmon $ScriptFile

_DoExit_ 0

#EOF#

Example call from within another script:

if [ -f /jffs/scripts/call-dns-monitor.sh ]
then /jffs/scripts/call-dns-monitor.sh & ; fi

Thank you!

Updated news: I rebooted and everything worked as expected. I think we can safely conclude that using the routes for the NodVPN DNS servers in the custom config for the client, and also the VPN Director rules, did not cause the problem.

I think my lack of understanding about init-start and the fact that the USB wasn't mounted was the likely culprit.

 

[iTyPsIDg]

OK, I added the script from Martinski to post-mount (created the file and then called it from post-mount) and did another reboot. While DNSMON and VPNMON didn't start, at least my router survives reboots.

 

[Martinski]

OK, I added the script from Martinski to post-mount (created the file and then called it from post-mount) and did another reboot. While DNSMON and VPNMON didn't start, at least my router survives reboots.

First, make sure of the following:

1) You copied exactly the full contents of the "call-dns-monitor.sh" script from the post to your "/jffs/scripts/call-dns-monitor.sh" file on the router.

If using a Windows PC, the utility WinSCP is highly recommended to transfer file contents.

2) The new script file contains ONLY Unix-style line endings (LF), NOT Windows DOS-style (CR+LF).

dos2unix /jffs/scripts/call-dns-monitor.sh

3) The new script file has the required executable permissions.

chmod a+x /jffs/scripts/call-dns-monitor.sh

4) When the script is invoked directly, do you see some output on the terminal window?

For example:

<MYPROMPT># /jffs/scripts/call-dns-monitor.sh
/jffs/scripts/call-dns-monitor.sh_32749: START
/jffs/scripts/call-dns-monitor.sh_32749: Launching /jffs/scripts/merlin-dns-monitor.sh
/jffs/scripts/call-dns-monitor.sh_32749: EXIT 0

What do you see when you run the script by itself on your router?

 

[Martinski]

@iTyPsIDg,

Immediately after a reboot is completed, you can check the system logs to see if the script has left its own trace messages behind:

cat /jffs/syslog.log | grep call-dns-monitor.sh

If no trace messages are found, it's very unlikely that the script was actually executed during the reboot.

If it has been a while since last reboot but less than 48 hours, and not quite a lot of activity has been logged, you might get lucky and still find trace messages left during the reboot here:

cat /jffs/syslog.log-1 | grep call-dns-monitor.sh

 

[iTyPsIDg]

I'm running on a Mac. I SSHed to the router and copy/pasted what you wrote into vi. When I run your scripts manually, they work. They aren't launching from post-mount for some reason. I tried the cat commands, but those are directories for me.

 

[Martinski]

I'm running on a Mac. I SSHed to the router and copy/pasted what you wrote into vi. When I run your scripts manually, they work.

OK, that's a good sign.
You said "scripts"? As in more than one?
One for DNSMON & one for VPNMON, is that what you mean?

I tried the cat commands, but those are directories for me.

OK, that's surprising.
Is that normal and as intended for your router model RT-AX86U?
I don't know. I have the RT-AC86U model which has those syslogs as files, not directories.

The fact that you have those directories completely empty also doesn't look right to me either.

It's possible that your router has a different file directory structure, but I'd say you need to confirm whether what you're seeing is correct for your router. Perhaps, other forum members with the same router model as yours can check and clear any doubts.

 

[Martinski]

When I run your scripts manually, they work. They aren't launching from post-mount for some reason.

Calling the script (any script actually) from /jffs/scripts/post-mount does have one big caveat:

During reboot, the post-mount script gets called for every disk partition on each USB disk drive that is plugged in and gets properly mounted.

This means that if you have a single USB drive that has a single disk partition, post-mount will be executed once. No problem here.

However, if you have one USB drive with multiple partitions, or more than one USB drive plugged in, post-mount gets executed multiple times and, as a result, each script called during its run also gets executed multiple times. This may not be good at all if the script is meant to run and do what it needs to do only once during reboot, but also depends on the order in which the partitions get mounted.

Do you have multiple USB disk drive partitions being mounted?

 

[iTyPsIDg]

OK, that's a good sign.
You said "scripts"? As in more than one?
One for DNSMON & one for VPNMON, is that what you mean?


OK, that's surprising.
Is that normal and as intended for your router model RT-AX86U?
I don't know. I have the RT-AC86U model which has those syslogs as files, not directories.

The fact that you have those directories completely empty also doesn't look right to me either.

It's possible that your router has a different file directory structure, but I'd say you need to confirm whether what you're seeing is correct for your router. Perhaps, other forum members with the same router model as yours can check and clear any doubts.

Haha, yeah, sorry. It was so minor to modify what you wrote to work for both. I half considered making it take a parameter so I could use one file.

I have no idea if that's normal for this router. I just replaced my AC86U with the AX86U two weeks ago.

Your reply about post-mount is helpful. I need to get it out of there. I have two USB drives attached and a total of three partitions.

 

[Martinski]

Haha, yeah, sorry. It was so minor to modify what you wrote to work for both. I half considered making it take a parameter so I could use one file.

OK, got it. Yeah, that's the benefit of writing code that can be reused for similar purposes/targets by simply changing initialized variables, or adding a parameter that resets one of the initialized variables, as you suggested. The latter would be my personal preference, but it's up to you.

Your reply about post-mount is helpful. I need to get it out of there. I have two USB drives attached and a total of three partitions.

One solution is to check the parameter passed to post-mount to see if it matches the disk partition on which Entware was installed.

For example:

if [ -x ${1}/entware/bin/opkg ] && \
   [ -f /jffs/scripts/call-dns-monitor.sh ]
then /jffs/scripts/call-dns-monitor.sh & ; fi

Also, it's recommended to put this code *after* the one where Entware services are started. So simply add the above lines at the end of whatever is already there in the existing script, but *before* any "exit" statement, if found.

 

[iTyPsIDg]

Bad news: It happened again. Good news: I learned a bit more about nsrum and getting everything back to the way it was was significantly faster.

I noticed, while using DNSMON, that after one of my reboots the VPN Director rules weren't there for the NordVPN DNS servers. Maybe I didn't click save, or something else happened that made me think the VPN Director and VPN Custom Configuration weren't the culprits. So I don't think we can conclude that that isn't the cause yet. I added them back in and then had a brown-out this morning and I imagine the router restarted.

I'm going to forgo automatically starting the scripts for now and only see how the VPN Director and VPN Client Custom Configuration rules impact my connections.

I did have the calls to those scripts after everything else; however, vpnmgr seems to always move to the end of post-mount.

I used to use Unbound and x3mRouting to prevent DNS leaks (according to dnsleaktest.com). Using that configuration never worked with the streaming services. I wonder if I use x3mRouting only, instead of putting the routes in the VPN Client Custom Configuration and putting the DNS IPs in VPN Director rules would be better? Maybe that would allow the process to happen a little later so a full reboot doesn't lock me out of the router and cause me to start fresh?

 

[iTyPsIDg]

In the interest of maybe giving too much information, but hopefully solving the problem if anyone else ever encounters it in the future, here are some screenshots:

Spoiler: VPN Director

Spoiler: VPN Client 1

Spoiler: vpnmgr Client List

I eventually want to rely on VPNMON to handle the schedule for client 1, while vpnmgr can continue to handle the schedule for client 2.

I want DNSMON to always run just to make it easy to quickly check and see if everything is configured correctly so that I'm not leaking DNS.

Again, for now I am running neither VPNMON nor DNSMON because I want to see if this is a problem with a script hanging during reboot or if it is a problem with the router thinking it isn't connected to the internet because of these routes so it never connects to the VPN.

 

[iTyPsIDg]

OK, so it has nothing to do with VPNMON, DNSMON, or any scripts in jffs.

I just rebooted again and the problem recurred. I think it's safe to say that the router doesn't like something about putting the WAN DNS servers in VPN Director and/or the Custom Configuration for the VPN Client. It ends up failing to fully boot back up if I set it up that way. Which brings me back to, maybe x3mRouting is a more elegant solution to accomplish the same goal without causing the router to brick every time it reboots?

 

[iTyPsIDg]

OK, got it. Yeah, that's the benefit of writing code that can be reused for similar purposes/targets by simply changing initialized variables, or adding a parameter that resets one of the initialized variables, as you suggested. The latter would be my personal preference, but it's up to you.


One solution is to check the parameter passed to post-mount to see if it matches the disk partition on which Entware was installed.

For example:

if [ -x ${1}/entware/bin/opkg ] && \
   [ -f /jffs/scripts/call-dns-monitor.sh ]
then /jffs/scripts/call-dns-monitor.sh & ; fi

Also, it's recommended to put this code *after* the one where Entware services are started. So simply add the above lines at the end of whatever is already there in the existing script, but *before* any "exit" statement, if found.

Now that I'm thinking about this more, wouldn't my post-mount cause all kinds of issues because I have three partitions?

Spoiler: post-mount

#!/bin/sh
swapon /tmp/mnt/Core-Files/myswap.swp # Added by amtm
. /jffs/addons/diversion/mount-entware.div # Added by Diversion
/jffs/scripts/ntpmerlin startup "$@" & # ntpMerlin
/jffs/scripts/connmon startup "$@" & # connmon
/jffs/scripts/uiDivStats startup "$@" & # uiDivStats
cru a logrotate "5 0 * * * /opt/sbin/logrotate /opt/etc/logrotate.conf >> /opt/tmp/logrotate.daily 2>&1" # added by scribe
/jffs/scripts/uiScribe startup "$@" & # uiScribe
/jffs/scripts/spdmerlin startup "$@" & # spdMerlin
/jffs/scripts/vpnmgr startup "$@" & # vpnmgr_startup

if [ -x ${1}/entware/bin/opkg ] && [ -f /jffs/scripts/call-monitors.sh ]; then
  /jffs/scripts/call-monitors.sh DNS &
  /jffs/scripts/call-monitors.sh VPN &
fi

Also, I went ahead and modified your script slightly to use parameters.

Spoiler: call-monitors.sh

#!/bin/sh
########################
## call-monitors.sh
########################

LogCmd="/usr/bin/logger -sct "$0_$$""

_DoExit_() {
  $LogCmd "EXIT $1"
  exit $1
}

$LogCmd "START"

# Set Maximum Timeout value as you see fit #
MaxTimeout=600
SleepSecs=2
TimerSecs=0

OptBinFile="/opt/sbin/screen"

if [ "$1" == "DNS" ]; then
  ScriptFile="sh /jffs/scripts/merlin-dns-monitor.sh"
  ScreenName="dnsmon"
elif [ "$1" == "VPN" ]; then
  ScriptFile="sh /jffs/scripts/vpnmon-r2.sh -monitor"
  ScreenName="vpnmon-r2"
else
  $LogCmd "ERROR: Incorrect argument passed. [$1]"
  _DoExit_ 2
fi

if [ ! -f $ScriptFile ]; then
  $LogCmd "ERROR: [$ScriptFile] NOT found."
  _DoExit_ 1
fi

until [ -f $OptBinFile ] || [ $TimerSecs -ge $MaxTimeout ]; do
  sleep $SleepSecs
  TimerSecs=$(($TimerSecs + $SleepSecs))
done

if [ ! -f $OptBinFile ]; then
  $LogCmd "ERROR: [$OptBinFile] NOT found."
  _DoExit_ 2
fi

$LogCmd "Launching $ScriptFile"
$OptBinFile -dmS $ScreenName $ScriptFile

_DoExit_ 0

#EOF#

Both monitoring tools now start on a reboot of the router.

 

[Martinski]

Now that I'm thinking about this more, wouldn't my post-mount cause all kinds of issues because I have three partitions?

Because of your setup, the post-mount script gets called 3 times during every reboot, so each line/command in the script is executed the same number of times, except the ones protected by the "if" conditional for "Entware."

WRT the swapon cmd, you simply get an error after the 1st successful call, and nothing bad really happens.

WRT the cru cmd, it simply replaces the existing cron job after the 1st successful call, so you're safe there.

However, for the rest of the calls to 3rd-party scripts, I cannot tell you whether those are safe to be called multiple times. I don't have them installed in my own router at home, and I don't know whether each script has code within to "protect" itself from running more than once when called. It may or may not be "bad" at all to execute each of them multiple times during the reboot process; I just don't know.

If you continue to have your login issues after reboots, I'd seriously consider doing a full factory defaults reset (including reformatting the JFFS partition), and then manually set up the router with the simplest configuration you can have (i.e. no restore with previous config, no add-ons, no VPN server/clients, none of the extra "stuff"), and run it like that for a few days, testing to see if the login issue persists after reboots.

If the initial simple setup is OK, then methodically install & set up only one extra feature/add-on at a time, and run it like that for a few days, testing at each junction for the login issue. "Rinse & Repeat" for each of the add-on scripts you want to have. Hopefully, through this slow process of integration, you'll find the culprit or, at least, get more clues to what or where the problem might be.

 

[Martinski]

Also, I went ahead and modified your script slightly to use parameters.

I'd suggest a couple of simple changes in your modified script:

FROM:

if [ $1 == "DNS" ]; then
...
...
elif [ $1 == "VPN" ]; then
...
...

TO:

if [ "$1" = "DNS" ]; then
...
...
elif [ "$1" = "VPN" ]; then
...
...

Your original "if" statements will work on the router without real problems; it's just that the current syntax triggers my "S/W Developer's OCD tendencies."

For example, try calling your current script *as is* manually *without* any arguments, or with an argument like "DNS VPN" (yes, blank space in between), and see for yourself what you get in such cases.

/jffs/scripts/call-monitors.sh "DNS VPN"

Yeah, nothing "bad" really happens; it's simply a couple of runtime errors on each call without adverse effects. Granted, those are not the expected input and are unlikely to happen in your particular use case; but any test errors, however minor, are "frowned upon" in my world because the source code would not be considered "clean code."

Obviously, you're free to take or ignore my advice as you deemed appropriate.

 

[ColinTaylor]

I'm running on a Mac. I SSHed to the router and copy/pasted what you wrote into vi. When I run your scripts manually, they work. They aren't launching from post-mount for some reason. I tried the cat commands, but those are directories for me.

View attachment 40138

That is wrong. /jffs/syslog.log and /jffs/syslog.log-1 should not be directories. They should be copies of the syslog files that the router periodically updates from the files in /tmp. Delete those directories and reboot the router. Then check that they are now regular files.