Preferred thumbdrive for scripts?

[JaimeZX]

Hey guys... so I'm on my second dead (or dying) thumbdrive now... even when I set it up to be non-journaling;

Two questions:
1) How often do you anticipate replacing these things? I seem to be going on about one per year.
2) Do you have a preferred source for longer-lasting drives? Guess I could get a tiny external HD instead of a thumbdrive... this one is 80GB for $17... https://www.amazon.com/dp/B07L38YGND/?tag=snbforums-20
Of course then I worry about overheating the router. So then I think about a little USB-powered fan on a splitter and then I wonder about the power draw from the router again...

Symptom: internet quits working (wifi AND wired, or often just wifi.) SSH reboot fixes it for a period of time between minutes and hours.

Portion of log follows. No other post content beyond this line.
----------------------------------
Apr 29 21:37:45 kernel: EXT4-fs error (device sdb1): __ext4_get_inode_loc:4908: inode #97: block 279: comm [: unable to read itable block
Apr 29 21:37:56 kernel: EXT4-fs (sdb1): previous I/O error to superblock detected
Apr 29 21:37:56 kernel: EXT4-fs error (device sdb1): __ext4_get_inode_loc:4908: inode #97: block 279: comm [: unable to read itable block
Apr 29 21:38:07 kernel: EXT4-fs (sdb1): previous I/O error to superblock detected
Apr 29 21:38:07 kernel: EXT4-fs error (device sdb1): __ext4_get_inode_loc:4908: inode #97: block 279: comm [: unable to read itable block
Apr 29 21:39:14 kernel: Read-error on swap-device (8:16:282136)
Apr 29 21:39:14 kernel: Read-error on swap-device (8:16:282144)
Apr 29 21:39:14 kernel: Read-error on swap-device (8:16:282152)
Apr 29 21:39:14 kernel: Read-error on swap-device (8:16:282160)
Apr 29 21:39:14 kernel: Read-error on swap-device (8:16:282168)
Apr 29 22:00:00 uiDivStats: Diversion logging not enabled!
Apr 29 22:00:00 uiDivStats: Open Diversion and use option l to enable logging
Apr 29 22:00:02 kernel: EXT4-fs (sdb1): previous I/O error to superblock detected
Apr 29 22:00:07 kernel: ------------[ cut here ]------------
Apr 29 22:00:07 kernel: WARNING: at fs/buffer.c:1079 mark_buffer_dirty+0xac/0xcc()
Apr 29 22:00:07 kernel: module: ip_set_list_set 7f007000 6760
Apr 29 22:00:07 kernel: module: tdts_udbfw 7f0a6000 26092
Apr 29 22:00:07 kernel: module: tdts_udb 7f06c000 180247
Apr 29 22:00:07 kernel: module: tdts 7f465000 416313
Apr 29 22:00:07 kernel: module: ip_set_hash_ip 7f45b000 19239
Apr 29 22:00:07 kernel: module: ip_set_hash_net 7f450000 22930
Apr 29 22:00:07 kernel: module: xt_set 7f449000 7395
Apr 29 22:00:07 kernel: module: ip_set 7f43d000 23785
Apr 29 22:00:07 kernel: module: tun 7f434000 12274
Apr 29 22:00:07 kernel: module: nf_nat_sip 7f42d000 5031
Apr 29 22:00:07 kernel: module: nf_conntrack_sip 7f424000 15713
Apr 29 22:00:07 kernel: module: nf_nat_h323 7f41d000 4761
Apr 29 22:00:07 kernel: module: nf_conntrack_h323 7f40f000 33807
Apr 29 22:00:07 kernel: module: nf_nat_rtsp 7f409000 3202
Apr 29 22:00:07 kernel: module: nf_conntrack_rtsp 7f403000 4067
Apr 29 22:00:07 kernel: module: nf_nat_ftp 7f3fd000 1144
Apr 29 22:00:07 kernel: module: nf_conntrack_ftp 7f3f6000 4909
Apr 29 22:00:07 kernel: module: ip6table_mangle 7f3f0000 934
Apr 29 22:00:07 kernel: module: sr_mod 7f3e8000 10842
Apr 29 22:00:07 kernel: module: cdrom 7f3ce000 30901
Apr 29 22:00:07 kernel: module: usblp 7f3c6000 10321
Apr 29 22:00:07 kernel: module: thfsplus 7f3a9000 85357
Apr 29 22:00:07 kernel: module: tntfs 7f329000 468922
Apr 29 22:00:07 kernel: module: tfat 7f2f0000 191259
Apr 29 22:00:07 kernel: module: ext2 7f2db000 53788
Apr 29 22:00:07 kernel: module: ext4 7f29a000 222306
Apr 29 22:00:07 kernel: module: crc16 7f294000 1007
Apr 29 22:00:07 kernel: module: jbd2 7f280000 49553
Apr 29 22:00:07 kernel: module: ext3 7f25d000 106589
Apr 29 22:00:07 kernel: module: jbd 7f24c000 42799
Apr 29 22:00:07 kernel: module: mbcache 7f245000 4599
Apr 29 22:00:07 kernel: module: usb_storage 7f236000 34410
Apr 29 22:00:07 kernel: module: sg 7f22b000 20039
Apr 29 22:00:07 kernel: module: sd_mod 7f21f000 22167
Apr 29 22:00:07 kernel: module: scsi_wait_scan 7f219000 416
Apr 29 22:00:07 kernel: module: scsi_mod 7f1f4000 108826
Apr 29 22:00:07 kernel: module: ohci_hcd 7f1ea000 17918
Apr 29 22:00:07 kernel: module: ehci_hcd 7f1dd000 31653
Apr 29 22:00:07 kernel: module: xhci_hcd 7f1ca000 51124
Apr 29 22:00:07 kernel: module: cdc_mbim 7f1c4000 3129
Apr 29 22:00:07 kernel: module: qmi_wwan 7f1bd000 5780
Apr 29 22:00:07 kernel: module: cdc_wdm 7f1b6000 7252
Apr 29 22:00:07 kernel: module: cdc_ncm 7f1ae000 8750
Apr 29 22:00:07 kernel: module: rndis_host 7f1a7000 4936
Apr 29 22:00:07 kernel: module: cdc_ether 7f1a1000 3187
Apr 29 22:00:07 kernel: module: asix 7f199000 10832
Apr 29 22:00:07 kernel: module: cdc_acm 7f191000 10357
Apr 29 22:00:07 kernel: module: usbnet 7f188000 11161
Apr 29 22:00:07 kernel: module: usbcore 7f165000 102094
Apr 29 22:00:07 kernel: module: mii 7f15f000 3367
Apr 29 22:00:07 kernel: module: ebt_ip 7f159000 1322
Apr 29 22:00:07 kernel: module: ebtable_broute 7f153000 873
Apr 29 22:00:07 kernel: module: ebtable_filter 7f14d000 1061
Apr 29 22:00:07 kernel: module: ebtables 7f144000 15643
Apr 29 22:00:07 kernel: module: ip6t_LOG 7f13d000 4494
Apr 29 22:00:07 kernel: module: ip6table_filter 7f137000 750
Apr 29 22:00:07 kernel: module: jffs2 7f119000 91538
Apr 29 22:00:07 kernel: module: zlib_deflate 7f10f000 19489
Apr 29 22:00:07 kernel: module: dhd 7f02e000 242573
Apr 29 22:00:07 kernel: module: et 7f01b000 48925
Apr 29 22:00:07 kernel: module: igs 7f013000 11927
Apr 29 22:00:07 kernel: module: emf 7f00a000 15345
Apr 29 22:00:07 kernel: module: ctf 7f000000 19647
Apr 29 22:00:08 kernel: Modules linked in: ip_set_list_set tdts_udbfw tdts_udb(P) tdts(P) ip_set_hash_ip ip_set_hash_net xt_set ip_set tun nf_nat_sip nf_conntrack_sip nf_nat_h323 nf_conntrack_h323 nf_nat_rtsp nf_conntrack_rtsp nf_nat_ftp nf_conntrack_ftp ip6table_mangle sr_mod cdrom usblp thfsplus tntfs(P) tfat(P) ext2 ext4 crc16 jbd2 ext3 jbd mbcache usb_storage sg sd_mod scsi_wait_scan scsi_mod ohci_hcd ehci_hcd xhci_hcd cdc_mbim qmi_wwan cdc_wdm cdc_ncm rndis_host cdc_ether asix cdc_acm usbnet usbcor
Apr 29 22:00:08 kernel: [<80055ff8>] (unwind_backtrace+0x0/0xf8) from [<80075060>] (warn_slowpath_common+0x4c/0x64)
Apr 29 22:00:08 kernel: [<80075060>] (warn_slowpath_common+0x4c/0x64) from [<80075094>] (warn_slowpath_null+0x1c/0x24)
Apr 29 22:00:08 kernel: [<80075094>] (warn_slowpath_null+0x1c/0x24) from [<80100c9c>] (mark_buffer_dirty+0xac/0xcc)
Apr 29 22:00:08 kernel: [<80100c9c>] (mark_buffer_dirty+0xac/0xcc) from [<7f2ba464>] (__ext4_handle_dirty_metadata+0xf8/0x104 [ext4])
Apr 29 22:00:08 kernel: [<7f2ba464>] (__ext4_handle_dirty_metadata+0xf8/0x104 [ext4]) from [<7f2bddac>] (ext4_mb_pa_callback+0x2684/0x346c [ext4])
Apr 29 22:00:08 kernel: [<7f2bddac>] (ext4_mb_pa_callback+0x2684/0x346c [ext4]) from [<7f2c27dc>] (ext4_mb_new_blocks+0x22c/0x470 [ext4])
Apr 29 22:00:08 kernel: [<7f2c27dc>] (ext4_mb_new_blocks+0x22c/0x470 [ext4]) from [<7f2b7ae4>] (ext4_ext_map_blocks+0x418/0x18f0 [ext4])
Apr 29 22:00:08 kernel: [<7f2b7ae4>] (ext4_ext_map_blocks+0x418/0x18f0 [ext4]) from [<7f2a3224>] (ext4_map_blocks+0x19c/0x354 [ext4])
Apr 29 22:00:08 kernel: [<7f2a3224>] (ext4_map_blocks+0x19c/0x354 [ext4]) from [<7f2a5720>] ((((SNIP... plenty of lines like the above)))
Apr 29 22:00:08 kernel: [<800fb560>] (bdi_writeback_thread+0x8c/0x160) from [<8008d9ec>] (kthread+0x88/0x90)
Apr 29 22:00:08 kernel: [<8008d9ec>] (kthread+0x88/0x90) from [<80050b8c>] (kernel_thread_exit+0x0/0x8)
Apr 29 22:00:08 kernel: ---[ end trace 51c7bbbcfa7bccc0 ]---
Apr 29 22:00:08 kernel: EXT4-fs (sdb1): delayed block allocation failed for inode 955 at logical offset 3268 with max blocks 541 with error -5
Apr 29 22:00:08 kernel: EXT4-fs (sdb1): This should not happen!! Data will be lost
Apr 29 22:00:08 kernel: EXT4-fs (sdb1): This should not happen!! Data will be lost
Apr 29 22:00:08 kernel: EXT4-fs (sdb1): delayed block allocation failed for inode 955 at logical offset 3825 with max blocks 3 with error -5
(SNIP.... LOTS of lines like the above)
Apr 29 22:00:10 kernel: EXT4-fs (sdb1): This should not happen!! Data will be lost
Apr 29 22:00:11 kernel: EXT4-fs (sdb1): previous I/O error to superblock detected
Apr 29 22:00:25 kernel: EXT4-fs error (device sdb1): ext4_read_block_bitmap:349: comm sh: Cannot read block bitmap - block_group = 9, block_bitmap = 250
Apr 29 22:00:25 kernel: EXT4-fs error (device sdb1) in ext4_free_blocks:4643: IO failure
Apr 29 22:00:25 kernel: EXT4-fs (sdb1): previous I/O error to superblock detected
Apr 29 22:00:25 kernel: EXT4-fs error (device sdb1): ext4_read_block_bitmap:349: comm sh: Cannot read block bitmap - block_group = 9, block_bitmap = 250
Apr 29 22:00:25 kernel: EXT4-fs (sdb1): delayed block allocation failed for inode 22 at logical offset 0 with max blocks 1 with error -5
Apr 29 22:00:25 kernel: EXT4-fs (sdb1): This should not happen!! Data will be lost
Apr 29 22:00:26 kernel: EXT4-fs (sdb1): previous I/O error to superblock detected

 

[cmkelley]

I'm all for SSDs myself, and I wouldn't go for the least expensive unless you absolutely HAVE to. Kingston is a good brand, personally I'd go with one of these: https://www.amazon.com/dp/B01N6JQS8C/?tag=snbforums-20 attached tothe USB 3.0 port through one of these: https://www.amazon.com/dp/B06XWRRMYX/?tag=snbforums-20 $42 total. Format it ext4 with journaling and never worry again.

 

[JaimeZX]

I like it. Guess I can use the balance of the space on that drive as more NAS. Guess I could also make a whole /Swap partition on there is well.

 

[bbunge]

I have an old PNY 2GB USB2 thumb drive I have used for years. It is on its third Asus router. Has been formated to FAT32, EXT2, EXT3 and EXT4. Has swap partioion + a data partition. As reliable as a Timex!
For scripts, entware and data use USB2 (set the USB3 slot to USB2 and you will have fewer problems). As for storing data, invest in a NAS and use the router as a router. You will be much happier...

 

[JaimeZX]

Ordered the Kingston. Won't arrive until May 4th. I HAVE TO DEAL WITH ADS FOR A WEEK? lol

 

[L&LD]

@JaimeZX amtm works best with a swap file, not a swap partition.

 

[maghuro]

Ordered the Kingston. Won't arrive until May 4th. I HAVE TO DEAL WITH ADS FOR A WEEK? lol

use NextDNS

 

[elorimer]

A normal USB 2.0 thumb drive should do fine for all these scripts. Going all the way to an SSD isn't necessary. I've had some issues with my router USB drives over the years, but only one failed out of 4 or 5. I think one I had to reformat in a PC.

 

[cmkelley]

I like it. Guess I can use the balance of the space on that drive as more NAS. Guess I could also make a whole /Swap partition on there is well.

As L&LD noted, amtm will make a swap file for you, I recommend you use a 2Gb swap, as certain scripts (skynet in particular) need large swap space for certain tasks. I think I recall someone saying swap partitions on the router aren't a good idea, but even if my memory on that is correct, I can't remember the reason why, although I do think it was more than "works better".

Also, routers don't make good NASes. It's relatively easy to overload the router trying to access multiple files on the SSD at the same time, there's a thread around here somewhere where someone demonstrated that. But it can safely be used for low use files, or perhaps a local backup of some files.

 

[cmkelley]

A normal USB 2.0 thumb drive should do fine for all these scripts. Going all the way to an SSD isn't necessary. I've had some issues with my router USB drives over the years, but only one failed out of 4 or 5. I think one I had to reformat in a PC.

It's a matter of convenience. "USB stick going bad" is no longer on my list of troubleshooting steps. The flash memory on a reputable brand SSD is much better quality than that used for USB sticks, an SATA controller is more robust than the circuitry used for thumb drives, and being attached by a cable means no heat is transferred to the drive electronics and I have no issues using USB 3.

The only downside is price, a good quality 128 Gb flash drive is a little less than half what the SSD + controller costs, and of course, no, you don't need all that space. But, eh, you could always use it as a local backup for some of your files.

 

[JaimeZX]

@JaimeZX amtm works best with a swap file, not a swap partition.

SWAP FILE ON SWAP PARTITION?

Also, routers don't make good NASes. It's relatively easy to overload the router trying to access multiple files on the SSD at the same time, there's a thread around here somewhere where someone demonstrated that. But it can safely be used for low use files, or perhaps a local backup of some files.

Pretty light duty. I have 5½TB of storage in a RAID5 array that's really only on the router so that more than one machine can access it; plus the Merlin firmware lets me assign different "logins" different RWX privileges on the thing vs being a Windows share. The heaviest load it really sees besides occasional backups is "serving" a movie back to the computer.

 

[cmkelley]

SWAP FILE ON SWAP PARTITION?

No, a swap file on a regular partition. Make the SSD one single partition and use amtm to create a 2Gb swap file.

Pretty light duty. I have 5½TB of storage in a RAID5 array that's really only on the router so that more than one machine can access it; plus the Merlin firmware lets me assign different "logins" different RWX privileges on the thing vs being a Windows share. The heaviest load it really sees besides occasional backups is "serving" a movie back to the computer.

Wow, certainly not a choice I would make, but if it works for you, that's what matters.

 

[K-2SO]

My choice is SanDisk Ultra Flair USB 3.0 32GB. Many recommend USB 2.0 mode port, but I use USB 3.0 with no ill effects. Runs hot when pushed hard, but it doesn't happen on the router. SSD is better, I agree.

 

[elorimer]

I have 5½TB of storage in a RAID5 array

Wait, what? Piqued my curiosity.

 

[heysoundude]

Wait, what? Piqued my curiosity.

Clearly they’ve rolled their own solution.
I’m guessing the array can more than keep up with network speeds.


Sent from my iPhone using Tapatalk

 

[juched]

Ya, just switched to an SSD I pulled out of my laptop (size upgrade) and stuck it in a USB 3.0 enclosure.

Perhaps I am wrong, but I made sure to overprovision the SSD by leaving 30% non-partitioned, to allow the controller to handle things better as I understand TRIM support is not offered by the router.

 

[L&LD]

@juched with that much capacity, it is tempting to just use 50% or less of an SSD.

 

[cmkelley]

Clearly they’ve rolled their own solution.
I’m guessing the array can more than keep up with network speeds.

Not at all necessarily home-brewed: https://www.amazon.com/dp/B078YQHWYW/?tag=snbforums-20

Correct, the array isn't at all the bottleneck. It's the router's ability to get data from the array and forward it while doing all of its other tasks, on a low-memory device. His stated use case works for him, but in a situation where there are a large amount of read/write activities, it would quickly bog down the router.

 

[cmkelley]

Ya, just switched to an SSD I pulled out of my laptop (size upgrade) and stuck it in a USB 3.0 enclosure.

Perhaps I am wrong, but I made sure to overprovision the SSD by leaving 30% non-partitioned, to allow the controller to handle things better as I understand TRIM support is not offered by the router.

fstrim exists in Entware. Is some kernel support required that's not present in asuswrt?

 

[thelonelycoder]

I like it. Guess I can use the balance of the space on that drive as more NAS. Guess I could also make a whole /Swap partition on there is well.

amtm, Diversion and Skynet use the same method to create, manage and delete a swap file. It does not matter with which script you create one, they all recognise and can manage it.
While all three of them (reluctantly for my two scripts) acknowledge the presence and size of a swap partition, they cannot be created nor managed trough them.