[R7800] Will no longer save settings after reboot.

[kamoj]

My router is old and I never had a problem despite hundreds of firmware changes, Traffic Meter on, add-on development etc.
Here are my figures:
...
msm_nand_probe: phys addr 0x1ac00000
msm_nand_probe: dmac 0x3
msm_nand_probe: allocated dma buffer at ffdfc000, dma_addr 5f62c000
status: 20
nandid: 1580a12c maker 2c device a1
ONFI probe : Found an ONFI compliant device MT29F1G08ABBEAH4 ,
Found a supported NAND device
NAND Controller ID : 0x4030
NAND Device ID : 0x1580a12c
Buswidth : 8 Bits
Density : 128 MByte
Pagesize : 2048 Bytes
Erasesize: 131072 Bytes
Oobsize : 64 Bytes
CFG0 Init : 0xa8d408c0
CFG1 Init : 0x0004745c
ECCBUFCFG : 0x00000203
...

 

[verbage]

MT29F1G08ABBEAH4

Ok, so that's the Micron one. So far, one R7800 router experiencing the settings loss issue is running the Macronix MX30UF1G18AC variant, and one R7800 without any issues after years of seemingly heavy use is running the Micro MT29F1G08ABBEAH4 variant. Don't worry--I will not be giving a post-by-post tally after each contribution--this is just to start things off

 

[HELLO_wORLD]

How do you find that info?

dmesg output does not show anything like that for me. Is it something you get just after reboot? In that case, I will proceed after getting my replacement router (delivery planned today was delayed to undefined day because of covid), as I avoid by all costs to reboot until then.

 

[verbage]

How do you find that info?

Telnet or ssh into the router, then give the following command:

dmesg | grep -i nand

@HELLO_wORLD, for sure, it would be great if you could get a data point for your failing R7800 before you return it!

 

[NetBytes]

Could not find this exact one here in France, but similar. Switched my order, thank you for the tip!

Good luck with it!

 

[HELLO_wORLD]

Thank you, yes dmesg | grep -i nand is what I did and it returns nothing... My router has been up for more than 4 days now, so I guess I won’t be able to find out until I will be ready to restart.

Telnet or ssh into the router, then give the following command:

dmesg | grep -i nand

@HELLO_wORLD, for sure, it would be great if you could get a data point for your failing R7800 before you return it!

 

[verbage]

Thank you, yes dmesg | grep -i nand is what I did and it returns nothing... My router has been up for more than 4 days now, so I guess I won’t be able to find out until I will be ready to restart.

Hmm, ok, yeah, mine is flashed with OpenWRT so maybe that preserves the dmesg log for a longer period of time? Whatever the case, for sure, do not risk a disruption of family peace at this point!!! No particular rush, once you get the new replacement in, perfect, you will have more flexibility.

 

[kamoj]

For Voxel/Netgear firmware you need to add some code/printout during boot.

 

[microchip]

The fact that some have issues with NVRAM and others don't no matter how much they flash, makes me suspect the former have received a bad batch of the router. I've also done many flashings and my R7800 is 2 years old now. Never had this issue occur so far

 

[verbage]

For Voxel/Netgear firmware you need to add some code/printout during boot.

@kamoj, do you have any suggestions about how to do this if the R7800 is running Netgear/Voxel firmware, and does not save a dmesg log? In your posting, you were able to identify the flash chip in your R7800 somehow--is there any particular tool you might recommend? Thanks!

 

[kamoj]

Just add the above mentioned grep in the dmesg output. But do it early during boot. I did in in the "boot" file itself.

@kamoj, do you have any suggestions about how to do this if the R7800 is running Netgear/Voxel firmware, and does not save a dmesg log? In your posting, you were able to identify the flash chip in your R7800 somehow--is there any particular tool you might recommend? Thanks!

 

[verbage]

Just add the above mentioned grep in the dmesg output. But do it early during boot. I did in in the "boot" file itself.

Thanks for the advice, @kamoj! Just to be clear, are you talking about the /etc/init.d/boot script, right?

 

[verbage]

Another note--I will soon be able to contribute another data point about heat as a potential issue. On my new RMA swap R7800 that lasted only two days before exhibiting the loss of settings issue last Thursday, I mentioned that I had let it sit around unplugged for two days, and then on Saturday evening installed OpenWRT to see how it fared with that firmware. And that's when I recognized this second R7800 started remembering its settings again after being afflicted with the issue just two days prior. As I tested OpenWRT and got it all set up, I did several full resets and power cycles, and settings were preserved. But whatever the case, since we are supposing this is a hardware problem, it should equally affect any firmware*, right? I have not done any full reboots or power cycles on that OpenWRT-flashed R7800 since Sunday, and I am hesistant to try it right now for fear of disturbing family sanity with a lapse in connectivity during business/school hours. But I will try a full power cycle this weekend to see what happens.

Well, I had to unexpectedly do a full power cycle of my OpenWRT-flashed R7800 this morning, and it came back up fine. It became afflicted with the lost settings issue due to a power brownout just a few days after I received it as an RMA swap last week. At the point of the brownout when it started losing settings on a full reboot or power cycle, it was running v1.0.2.75.2SF firmware. Since this lost settings issue seems to be hardware-related, I previously speculated that the issue would affect all firmwares equally (official Netgear, Voxel, OpenWRT, or whatever). But later, I reconsidered the suggestion, and speculated that due to how flash is partitioned and mapped differently between firmwares, it might actually be possible that some alternative firmwares do work correctly (if they avoid the worn out or defective portion of flash used by another firmware). This is all wild speculation, and who knows if that is the case here, but I can say for sure that this R7800 was suffering the settings loss issue last week when running v1.0.2.75.2SF firmware, but now that I am running OpenWRT, it does not seem to be suffering from the same issue.

THIS HAS NOTHING TO DO WITH ONE FIRMWARE BEING BETTER THAN THE OTHER, OR ONE BEING DEFECTIVE OR NOT, SO PLEASE DON'T INTERPRET IT THAT WAY. Instead, this probably has everything to do with the fact that the Macronix flash chip in this particular router is not very robust and wears out with very few write cycles... And it also has to do with the fact that I had enabled the Traffic Monitor under v1.0.2.75.2SF firmware without knowing the implications--that it causes excessive writing to the flash chip to keep its statistics up-to-date. That said, if the flash chip were robust, it should be able to deal with frequent Traffic Monitor writes...

The fact that this router now retains its settings under OpenWRT might also be related to the different flash mappings used by the two firmwares as mentioned above. To really seal the case on this, what I could do is reflash this router back to v1.0.2.75.2SF firmware to see if it retains settings or not. If it does not, one might see this as pretty strong evidence that is it due to the use of different flash maps. I am hesitant to do this "experiment" right now as this is serving as the primary router for the house, and all is working well. But I might be able to do it this weekend...

 

[Droidrat]

@verbage

What basis do you have for believing / suggesting that (any) firmware is storing data in a particular, specified area of NVRAM ? Have you examined the OpenWRT source code?

I don't believe that it is possible to (yet) rule out the possibility that this issue is firmware-based and, in fact, your recent test with OpenWRT suggests that it might be. Much of Voxel's firmware is inherited (w/o change) from Netgear ; so if there is a problem with the inherited NG firmware, it would likely be passed through.

 

[verbage]

What basis do you have for believing / suggesting that (any) firmware is storing data in a particular, specified area of NVRAM ? Have you examined the OpenWRT source code? ... I don't believe that it is possible to (yet) rule out the possibility that this issue is firmware-based and, in fact, your recent test with OpenWRT suggests that it might be. Much of Voxel's firmware is inherited (w/o change) from Netgear ; so if there is a problem with the inherited NG firmware, it would likely be passed through.

Hi @Droidrat, I am pretty sure that official Netgear/Voxel firmwares use a different flash layout than OpenWRT. Here's a comment in a large R7800 thread on the OpenWRT forums that clearly indicates more recent versions of OpenWRT are using a different flash layout than OEM (i.e. Netgear, and hence, Voxel):

https://forum.openwrt.org/t/build-for-netgear-r7800/316/1892

I may be completely wrong--and I have no problem admitting that(!!!)--but I am assuming that the configuration and settings that one expects to persist over a reboot or power cycle are stored some place on the NAND flash chip. But again, I could be completely incorrect--maybe there is a small NOR-style flash chip where the configuration/settings are stored versus the main 128MB NAND-style flash chip, where the ROM, apps, and other things are stored? I am no expert by any means, and somebody in the know can answer this simple question without breaking a sweat. Specifically, are persistent configuration and settings details stored somewhere on the main 128MB NAND chip?

I am not ruling out a firmware-related issue yet. But by getting more data, the picture will hopefully come into focus. Are the R7800's that are experiencing the settings loss issue running only Macronix-brand flash chips, or only Micron-ones, or a mix of the two? If if is just with one brand, for example, this might point towards a hardware problem. But if the issue equally affects routers with both brands of flash chip, then yeah, this might be more suggestive of a firmware-related issue.

 

[spocko]

Another tidbit of background into is that the filesystems used in flash memory, such as ubifs, include a feature called "wear leveling" which spreads writes across the flash memory device to help extend the lifespan. Frequently written files such as traffic monitor data are not always getting written to the same physical location in the flash.

https://en.wikipedia.org/wiki/Wear_leveling

 

[verbage]

@Droidrat, one other thing that might point to a hardware vs. firmware issue is that in the case of at least one of my R7800s, the settings loss issue was triggered by a power brownout. This is not necessarily the case for all those whose routers are experiencing this problem, for sure. Another point is that there are folks that have been running this router for 2-3 years time, that are using the very same firmwares, and that despite tons of flashing, have not run into this problems. Since they are running the very same firmwares, can't we subract that as one of the variables? If so, that would leave just hardware...

Another tidbit of background into is that the filesystems used in flash memory, such as ubifs, include a feature called "wear leveling" which spreads writes across the flash memory device to help extend the lifespan. Frequently written files such as traffic monitor data are not always getting written to the same physical location in the flash. https://en.wikipedia.org/wiki/Wear_leveling

Thanks, @spocko, for bringing this up. Since the ubifs does deal with wear leveling, if the flash chips in these routers are failing with rather short lifespans, for me, this would indicate a hardware issue. But yeah, while it seems to be a hardware-related issue, there is no definitive data to confirm this assertion at this point.

As I mentioned, if we can gather more data points about routers in which this issue has been noted vs. those without it, that would be helpful. Brand of the NAND chip might be a helpful starting point, but for sure, there might also be subtle revisions in R7800 circuitry--most likely to save a few pennies on production costs--that leave the router more susceptible to this issue due to brown-/blackouts. To get at this question, probably more data would be needed.

 

[kokishin]

I doubt this is useful info but WTH.

I am having a similar issue with my 7800 losing config settings and not being able to restore using a saved config file after a reboot or power cycling. Everything gets reset back to out-of-the-box settings except for 74.4SF. I have to manually enter all config info.

I have found that running 74.4SF with Traffic Meter enabled and DST enabled, I cannot pass the Shields Up All Service Ports Test. If I disable Traffic Meter or DST, then I pass the test. Not only that, with DST enabled and Traffic Meter enabled, Traffic Meter does not count up and remains static. Prior to 74.4SF, I am not aware I had this issue. Not blaming 74.4SF, just reporting what I observe.

After reading this thread this morning, I decided to disable Traffic Meter (don't need it) and enable DST. Shields Up All Service Ports Test passes as expected. I did not reboot the router.

I'm in coronavirus government shelter-in-place so I'm nervous to power cycle or reboot my 7800. My 7800 was purchased Dec 2018 so it's out of warranty.

If Netgear has a bad batch of nvrams, they should issue a recall. Wishful thinking I know.

 

[kamoj]

I just added output of flash type and erase counters to the kamoj add-on beta.

 

[verbage]

View attachment 22471
I just added output of flash type and erase counters to the kamoj add-on beta.

Completely awesome--this should make it much simpler for folks to figure it out though they will still at least need to install the beta add-on!!! Thank you, @kamoj!