QOS data storage

[Crazy]

Hello, I’ve just recently upgraded my rt-ac66u to a ac5300. Love all the new feature. But been wondering with all the data Qos like web history and app analysis take up on the router does it kill the nvram over time or does it goes into the ram? Thanks.

 

[jtp10181]

Hello, I’ve just recently upgraded my rt-ac66u to a ac5300. Love all the new feature. But been wondering with all the data Qos like web history and app analysis take up on the router does it kill the nvram over time or does it goes into the ram? Thanks.

It saves the data on the JFFS partition. If you search on the forums there is all sorts of discussion on the life of the nvram and JFFS.


Sent from my iPhone using Tapatalk

 

[RMerlin]

It's fine, the router's flash is rated for handling that level of write activity.

 

[sfx2000]

It's fine, the router's flash is rated for handling that level of write activity.

Sorry to put a clinker in - items like this should probably be written to tmpfs - it's ephermal in nature, so it doesn't need to be kept on the flash... there are ways to reduce impact to flash overall - zram is one, to reduce hits to swap, and then push some of these log items into RAM via tmpfs...

Even with jffs - there's a lot of writes, and eventually it does burn a hole in flash, and jffs can garbage recover to a point where there is insufficient space to run...

It takes a while, but we're starting to see this with older RT-68U devices, and the RT-AC3200 and RT-AC87U's are more impacted due to complexity there.

 

[RMerlin]

it's ephermal in nature, so it doesn't need to be kept on the flash.

We're talking about traffic history here. You want App Analysis data to be persistent, otherwise is pretty useless.

The NAND Asus picked was designed to handle rolling logging according to the datasheet. We're talking tens of thousands of PE cycles supported here (I forgot if it was 10K or 100K).

It takes a while, but we're starting to see this with older RT-68U devices

I haven't seen a single reported case so far.

 

[sfx2000]

We're talking about traffic history here. You want App Analysis data to be persistent, otherwise is pretty useless.

The NAND Asus picked was designed to handle rolling logging according to the datasheet. We're talking tens of thousands of PE cycles supported here (I forgot if it was 10K or 100K).

Pardon me for an english lesson -ephemeral is temp data that does not need to be kept - e.g. not persistent...

In embedded SW - we have stateless things - e.g. code that does not change - we have user data, config items and storage, and we have stuff that just doesn't matter - and most of the writes via logging are just that - so put that over into tempfs in RAM, and keep it off the flash...

The linear/raw NAND with most consumer devices like routers - Asus included - it's cheap stuff, and writing to any given cell in that space has a defined lifespan...

Going to eMMC - things get better - but again, most space there, the more cells - and the longer the lifespan.

I've seen android phones with raw NAND and jffs die after 3 months in use - and that's with quality NAND.

And yes - looking back thru the forums here - there's a fair amount of jffs/nand issues, esp with the more complicated devices with small flash.

 

[Crazy]

Is there any way I can have all logs written on to a usb drive instead? I've searched and it only talks about traffic logs.

 

[RMerlin]

In embedded SW - we have stateless things - e.g. code that does not change - we have user data, config items and storage, and we have stuff that just doesn't matter - and most of the writes via logging are just that - so put that over into tempfs in RAM, and keep it off the flash...

We're not talking about logging, we're talking about the traffic database here. It's actual data people want to track over an extended period of time. Even recently as Asus moved that database to a different location in JFFS following the switch from firmware 380 to 384, at least two users complained on these forums about losing that data (the solution is to manually move the database to the new location).

The linear/raw NAND with most consumer devices like routers - Asus included - it's cheap stuff, and writing to any given cell in that space has a defined lifespan...

Yes, and I gave you that lifespan: 10K or 100K PE cycles. It's not a cheap 1K PE cycle part, the datasheet for it specifically states it's intended for heavy usage. Assuming it's the worst number (10K) on a 64 MB partition, that means you could potentially write 64 MB per day for a period of 27 years before wearing it out...

And yes - looking back thru the forums here - there's a fair amount of jffs/nand issues, esp with the more complicated devices with small flash.

Show me one. I haven't seen a single one so far. The only issues encountered were premapped bad blocks (which are present at purchase time - my own RT-AC86U has two such bad blocks out of the box), which has nothing to do with longevity.

 

[RMerlin]

Is there any way I can have all logs written on to a usb drive instead? I've searched and it only talks about traffic logs.

The system log is written to RAM, and only gets copied back to JFFS once in a while.

 

[sfx2000]

The system log is written to RAM, and only gets copied back to JFFS once in a while.

busybox typically writes syslog to the ringbuffer (tmpfs) - so it's really down to a config...

Hobbisan choices perhaps - but one must admit that some of the older SDK's are not very friendly with regards to flash - whit the RT3200's with tight flash, and the 87U's to a lesser degree...

 

[sfx2000]

Yes, and I gave you that lifespan: 10K or 100K PE cycles. It's not a cheap 1K PE cycle part, the datasheet for it specifically states it's intended for heavy usage. Assuming it's the worst number (10K) on a 64 MB partition, that means you could potentially write 64 MB per day for a period of 27 years before wearing it out...

Actually - with MLC cheap NAND - one can burn enough holes with JFFS to render things inert in a couple of months... Been there with a handset vendor that did a write once a second into a 32MB flash part... and we started see returns within 6 months of that device being launched.

Anyways - the 86U and associated devices on the same SDK have a problem inherited from broadcom's SDK - I'm hoping that newer devices don't have that problem. For most, it's likely not a problem, until one brings third party items into play - and there, writes can and often do, start to be a problem.

 

[RMerlin]

Hobbisan choices perhaps - but one must admit that some of the older SDK's are not very friendly with regards to flash - whit the RT3200's with tight flash, and the 87U's to a lesser degree...

All Asus's high-end models share the same 128 MB NAND space. It's only the nvram space that was problematic for the RT-AC3200 - only 64 KB, recently upgraded to 128 KB.

busybox typically writes syslog to the ringbuffer (tmpfs) - so it's really down to a config...

Asus has BB's syslog pointed to /tmp/syslog.log, with the rc daemon taking care of periodically copying it to /jffs under certain conditions.

Been there with a handset vendor that did a write once a second into a 32MB flash part... and we started see returns within 6 months of that device being launched.

Not surprised. That's 86400 writes per day. Assuming an erase block size of 4 KB, that's a minimum 337 MB written per day. More if what they are writing isn't smaller than 4 KB. Ouch...