R9000 running Voxel's 1.0.4.42HF rebooting randomly.

[kamoj]

The "Top ever" temperature is saved in nvram so it survives reboot, new add-on, new firmware etc.
I have no reset button for this - yet, so you have to manually reset this value.

Hello,

Kamoj add-on sure makes things easy! Highest temp still says 79c but I've been nowhere near that except the one time right after installing 5.3b15 and starting AdGuard Home. I only had that one "mystery" reboot and since then temps have stayed at 68 to 72c.

I installed 5.3b16 a little while ago...all is working and temps so far are 67 to 70c. All is good!

BL

 

[kamoj]

I'm very happy for your success story and your drive to go to the bottom with your problems.
I'm very glad me and our community can accomplish this kind of investigations.
So thank you for telling your story AND using the kamoj add-on!

The dmesg is "hi-jacked" by my add-on (dumped and reset every 100 ms) if you have the log to USB function on.
It's the only way to not lose data at a sudden reboot. (I have a workaround for it, but it increase cpu-load, so it's not there - now.)
So I advice you to off the logging to USB and then check the dmesg again. Hopefully it will work even for the R9000
To force a new dmesg log entry you can issue command in router : nvram show

@kamoj thanks for this incredible info, if I understood correctly UBIFS should be able to "autocorrect" the issue moving the data out of the bad block to a good one and mark down the bad block, thing is I was still seeing ECC errors in the logs, like UBIFS wasn't doing it.

View attachment 24434

After checking with ubinfo command I've found how the UBI0 is built, short story is I've found 1 bad block in reported and I saw device mtd17 (traffic_meter.bak) having the ECC errors in dmesg log, so I checked the Traffic Meter feature was enabled, as soon I disabled and restarted my router the ECC errors are gone and ECC error counter remains zero.

View attachment 24435

So far dmesg isn't reporting any ECC error in 11 hrs running

View attachment 24436

thank you so much for your valuable help! crossing my fingers this was the issue.

Btw, could you check in your add-on 5.3b15, when you click dmesg log for R9000 it doesn't show anything, just blank. Otherwise it works well with the save log feature you just added.

@kamoj thanks for this incredible info, if I understood correctly UBIFS should be able to "autocorrect" the issue moving the data out of the bad block to a good one and mark down the bad block, thing is I was still seeing ECC errors in the logs, like UBIFS wasn't doing it.

View attachment 24434

After checking with ubinfo command I've found how the UBI0 is built, short story is I've found 1 bad block in reported and I saw device mtd17 (traffic_meter.bak) having the ECC errors in dmesg log, so I checked the Traffic Meter feature was enabled, as soon I disabled and restarted my router the ECC errors are gone and ECC error counter remains zero.

View attachment 24435

So far dmesg isn't reporting any ECC error in 11 hrs running

View attachment 24436

thank you so much for your valuable help! crossing my fingers this was the issue.

Btw, could you check in your add-on 5.3b15, when you click dmesg log for R9000 it doesn't show anything, just blank. Otherwise it works well with the save log feature you just added.

 

[multicast]

I'm very happy for your success story and your drive to go to the bottom with your problems.
I'm very glad me and our community can accomplish this kind of investigations.
So thank you for telling your story AND using the kamoj add-on!

The dmesg is "hi-jacked" by my add-on (dumped and reset every 100 ms) if you have the log to USB function on.
It's the only way to not lose data at a sudden reboot. (I have a workaround for it, but it increase cpu-load, so it's not there - now.)
So I advice you to off the logging to USB and then check the dmesg again. Hopefully it will work even for the R9000
To force a new dmesg log entry you can issue command in router : nvram show

Thanks again @kamoj, and yes I'm very pleased with all the help I've found here. I will take a look at dmesg when I see this whole thing is stable don't want to introduce more noise until then

 

[paddyr]

You have this functionality in the kamoj add-on:
Save the system logs to USB. (or /opt/kamoj/logs if no USB detected)

Hi Kamoj, thanks for the reply, I don't see this as an option maybe it's only in the beta, I'm currently running version Kamoj Add-on:5.00, should it be in that version as well?

Good news is that I've not experienced a random reboot now in 7 days.

 

[kamoj]

Yes, it's only in the beta 5.3+.
PS
5.00 "xmas edition "is also a beta, much less tested than 5.3 beta...

Hi Kamoj, thanks for the reply, I don't see this as an option maybe it's only in the beta, I'm currently running version Kamoj Add-on:5.00, should it be in that version as well?

Good news is that I've not experienced a random reboot now in 7 days.

 

[paddyr]

Thanks for the quick reply, is it possible to get beta 5.3+ or is it a closed beta test?

 

[kamoj]

Thanks for the quick reply, is it possible to get beta 5.3+ or is it a closed beta test?

PM:ed you info about beta testing!

 

[blueliner]

Hello,

Kamoj add-on sure makes things easy! Highest temp still says 79c but I've been nowhere near that except the one time right after installing 5.3b15 and starting AdGuard Home. I only had that one "mystery" reboot and since then temps have stayed at 68 to 72c.

I installed 5.3b16 a little while ago...all is working and temps so far are 67 to 70c. All is good!

BL

Just a quick note. My router is still running well. No more mystery reboots and temps (with OpenVPN, QOS and AdGuard Home running) are typically 67-69 degrees c.

BL.

 

[multicast]

@Voxel and @kamoj, does anyone of you know what is the script/file used to build/load the firewall rules in IPTABLES when a change is done in the configuration? I have some own rules that are overwritten every time so I would like to add them as part of the standard firewall rules.

Thanks in advance!

PS: My router has been stable for a week already, I will try to enable 160Mhz and test if stills reboots, so far I think were the ECC errors.

 

[kamoj]

/opt/scripts/firewall-start.sh
or if you use the kamoj add-on, any file with name matching: /opt/scripts/firewall-start*.sh
PS
Called from
/usr/sbin/net-wall

@Voxel and @kamoj, does anyone of you know what is the script/file used to build/load the firewall rules in IPTABLES when a change is done in the configuration? I have some own rules that are overwritten every time so I would like to add them as part of the standard firewall rules.

Thanks in advance!

PS: My router has been stable for a week already, I will try to enable 160Mhz and test if stills reboots, so far I think were the ECC errors.

 

[multicast]

/opt/scripts/firewall-start.sh
or if you use the kamoj add-on, any file with name matching: /opt/scripts/firewall-start*.sh
PS
Called from
/usr/sbin/net-wall

Thank you @kamoj

 

[VinnyJ]

I've been having mystery reboots recently, not sure what's going on.

Router - Nighthawk X10 R8900
Router Firmware Version - (Voxel) V1.0.4.42HF
Kamoj Add-on V5.3b18

After reading the thread, since people are posting what's under FLASH INFO in Kamoj Add-on Settings page, here's mine.

FLASH TYPE NAND:
mtd0 : u-boot _ _ nand _ worn bad:0 _factory bad:0 _bad allowed:t writable:f ecc errors:0 _ _ _size: _1966080 bytes
mtd1 : u-boot-env nand _ worn bad:0 _factory bad:0 _bad allowed:t writable:f ecc errors:0 _ _ _size: _1179648 bytes
mtd2 : ART _ _ _ _nand _ worn bad:0 _factory bad:0 _bad allowed:t writable:t ecc errors:0 _ _ _size: _1310720 bytes
mtd3 : ART.bak _ _nand _ worn bad:0 _factory bad:0 _bad allowed:t writable:t ecc errors:0 _ _ _size: _1310720 bytes
mtd4 : kernel _ _ nand _ worn bad:0 _factory bad:0 _bad allowed:t writable:t ecc errors:0 _ _ _size: _4456448 bytes
mtd5 : rootfs _ _ nand _ worn bad:0 _factory bad:0 _bad allowed:t writable:t ecc errors:0 _ _ _size: 41943040 bytes
mtd6 : firmware _ nand _ worn bad:0 _factory bad:0 _bad allowed:t writable:t ecc errors:0 _ _ _size: 46399488 bytes
mtd7 : netgear _ _nand _ worn bad:0 _factory bad:0 _bad allowed:t writable:t ecc errors:4327 _ size: 71827456 bytes
mtd8 : plex _ _ _ nand _ worn bad:0 _factory bad:0 _bad allowed:t writable:t ecc errors:0 _ _ _size:314572800 bytes
mtd9 : reserve _ _nand _ worn bad:0 _factory bad:0 _bad allowed:t writable:t ecc errors:0 _ _ _size: 91750400 bytes
mtd10: crashdump _nand _ worn bad:0 _factory bad:0 _bad allowed:t writable:t ecc errors:0 _ _ _size: _ 524288 bytes
mtd11: language _ nand _ worn bad:0 _factory bad:0 _bad allowed:t writable:t ecc errors:0 _ _ _size: _3670016 bytes
mtd12: config _ _ nand _ worn bad:0 _factory bad:0 _bad allowed:t writable:t ecc errors:0 _ _ _size: _1179648 bytes
mtd13: pot _ _ _ _nand _ worn bad:0 _factory bad:0 _bad allowed:t writable:t ecc errors:0 _ _ _size: _1179648 bytes

FLASH TYPE NOR:

FLASH TYPE UBI:
Device: ubi0, ubi1type:ubi _ bad blocks:0 0Allowed bad blocks:38 80Max/mean erase counters:20/16 _size: _ _ _ _0size: bytes 
mtd14: cert _ _ _ _ _ _ _ _ ubi _Volume:0 State:OK _ size: _ 126976 bytes
mtd15: pot.bak _ _ _ _ _ _ _ubi _Volume:1 State:OK _ size: _ 380928 bytes
mtd16: traffic_meter _ _ _ _ubi _Volume:2 State:OK _ size: _1777664 bytes
mtd17: traffic_meter.bak _ _ubi _Volume:3 State:OK _ size: _1777664 bytes
mtd18: dongle _ _ _ _ _ _ _ ubi _Volume:4 State:OK _ size: _1777664 bytes
mtd19: overlay_volume _ _ _ ubi _Volume:5 State:OK _ size: 58408960 bytes
mtd20: plexmediaserver _ _ _ubi _Volume:0 State:OK _ size:294076416 bytes

The only thing that would stand out to me is the ecc error number in mtd7, not sure how to remedy it.

 

[multicast]

I've been having mystery reboots recently, not sure what's going on.

Router - Nighthawk X10 R8900
Router Firmware Version - (Voxel) V1.0.4.42HF
Kamoj Add-on V5.3b18

After reading the thread, since people are posting what's under FLASH INFO in Kamoj Add-on Settings page, here's mine.

FLASH TYPE NAND:
mtd0 : u-boot _ _ nand _ worn bad:0 _factory bad:0 _bad allowed:t writable:f ecc errors:0 _ _ _size: _1966080 bytes
mtd1 : u-boot-env nand _ worn bad:0 _factory bad:0 _bad allowed:t writable:f ecc errors:0 _ _ _size: _1179648 bytes
mtd2 : ART _ _ _ _nand _ worn bad:0 _factory bad:0 _bad allowed:t writable:t ecc errors:0 _ _ _size: _1310720 bytes
mtd3 : ART.bak _ _nand _ worn bad:0 _factory bad:0 _bad allowed:t writable:t ecc errors:0 _ _ _size: _1310720 bytes
mtd4 : kernel _ _ nand _ worn bad:0 _factory bad:0 _bad allowed:t writable:t ecc errors:0 _ _ _size: _4456448 bytes
mtd5 : rootfs _ _ nand _ worn bad:0 _factory bad:0 _bad allowed:t writable:t ecc errors:0 _ _ _size: 41943040 bytes
mtd6 : firmware _ nand _ worn bad:0 _factory bad:0 _bad allowed:t writable:t ecc errors:0 _ _ _size: 46399488 bytes
mtd7 : netgear _ _nand _ worn bad:0 _factory bad:0 _bad allowed:t writable:t ecc errors:4327 _ size: 71827456 bytes
mtd8 : plex _ _ _ nand _ worn bad:0 _factory bad:0 _bad allowed:t writable:t ecc errors:0 _ _ _size:314572800 bytes
mtd9 : reserve _ _nand _ worn bad:0 _factory bad:0 _bad allowed:t writable:t ecc errors:0 _ _ _size: 91750400 bytes
mtd10: crashdump _nand _ worn bad:0 _factory bad:0 _bad allowed:t writable:t ecc errors:0 _ _ _size: _ 524288 bytes
mtd11: language _ nand _ worn bad:0 _factory bad:0 _bad allowed:t writable:t ecc errors:0 _ _ _size: _3670016 bytes
mtd12: config _ _ nand _ worn bad:0 _factory bad:0 _bad allowed:t writable:t ecc errors:0 _ _ _size: _1179648 bytes
mtd13: pot _ _ _ _nand _ worn bad:0 _factory bad:0 _bad allowed:t writable:t ecc errors:0 _ _ _size: _1179648 bytes

FLASH TYPE NOR:

FLASH TYPE UBI:
Device: ubi0, ubi1type:ubi _ bad blocks:0 0Allowed bad blocks:38 80Max/mean erase counters:20/16 _size: _ _ _ _0size: bytes
mtd14: cert _ _ _ _ _ _ _ _ ubi _Volume:0 State:OK _ size: _ 126976 bytes
mtd15: pot.bak _ _ _ _ _ _ _ubi _Volume:1 State:OK _ size: _ 380928 bytes
mtd16: traffic_meter _ _ _ _ubi _Volume:2 State:OK _ size: _1777664 bytes
mtd17: traffic_meter.bak _ _ubi _Volume:3 State:OK _ size: _1777664 bytes
mtd18: dongle _ _ _ _ _ _ _ ubi _Volume:4 State:OK _ size: _1777664 bytes
mtd19: overlay_volume _ _ _ ubi _Volume:5 State:OK _ size: 58408960 bytes
mtd20: plexmediaserver _ _ _ubi _Volume:0 State:OK _ size:294076416 bytes

The only thing that would stand out to me is the ecc error number in mtd7, not sure how to remedy it.

@VinnyJ it seems you have same issue I had, just disable the Traffic Meter and you good.