Solved Need help with my usb 3.1 port on my Nas failing.

[DJones]

Hoping someone can help me with this system log to understand what exactly is causing my usb 3.1 port to fail occasionally. I have to note that it only fails when I use both my external drive bay and my 5G qnap external Ethernet port. So I know what likely causes it, but not why.
I'm honestly still learning Linux but debug/trace logs are not my strong suit.

Device in question: 'aqc111 4-1:1.0 enx245ebe655241: 'aqc111' usb-0000:01:00.0-1, QNAP QNA-UC5G1T USB to 5GbE Adapter'

I've redacted hardware name and my smtp mail to root info, but the system runs Proxmox 8.1.11 .

Dmesg also spits a error, but unfortunately I didn't get it. I've setup a script to force restart by pinging my router and reboot if it fails, as rebooting temporarily fixes the pcie usb card disconnecting. But I'd rather that not be the fix, but rather a fail safe.

systemlog

systemlog - Pastebin.com

Pastebin.com is the number one paste tool since 2002. Pastebin is a website where you can store text online for a set period of time.

pastebin.com

Edit: I've added irqpoll to grub will see if that helps any. and installed irqbalance and using irqbalance --oneshot as I'm running a server environment and it was pinning to only one core, so now it's spread evenly. according to cat /proc/interrupts.

 

[TheLostSwede]

Older motherboards weren't known for being able to deliver sufficient power and as USB 3.0 is 900 mA, you might run into an issue where when both ports are used, you don't get 900 mA per port, which means your devices won't work. Could also be that the 5 Gbps dongle draws 900 mA and the hard drive wants 910 mA, which works when the dongle isn't there, but not when it is.

However, with zero system specs, I can't really know what hardware you have, so this is just a guess, but if your NAS truly has an Intel 915 chipset, as your screenshots suggests, it might be high time to upgrade to something far less power hungry and modern, as it's almost 20 years since that chipset was introduced.

 

[DJones]

Older motherboards weren't known for being able to deliver sufficient power and as USB 3.0 is 900 mA, you might run into an issue where when both ports are used, you don't get 900 mA per port, which means your devices won't work. Could also be that the 5 Gbps dongle draws 900 mA and the hard drive wants 910 mA, which works when the dongle isn't there, but not when it is.

However, with zero system specs, I can't really know what hardware you have, so this is just a guess, but if your NAS truly has an Intel 915 chipset, as your screenshots suggests, it might be high time to upgrade to something far less power hungry and modern, as it's almost 20 years since that chipset was introduced.

Runs around 80 watts idle, cpu governor’s set to on demand, but I agree a newer server would be better. System is a lanner inc FW-8758A 16gb ram i7-3770. Added the pcie usb card aftermarket. Servers pretty locked down with mitigations because of its age.

It’s old, maybe 11 years-ish, but very minimal use as a media server / nas. Running proxmox for vm’s. i915 just simply refers to the intel HD graphics, which really isn't used on the server, it's not a Pentium 4.

I do plan on building or buying a new server probably a 1U two blade server probably from supermicro, but this old machine it serves it’s purpose for now.

 

[DJones]

The usb 3.1 card has auxiliary sata power so it’s not dependent on just the pcie power.

 

[TheLostSwede]

The usb 3.1 card has auxiliary sata power so it’s not dependent on just the pcie power.

Could still be an issue with the add-in card. Maybe an old, leaky capacitor that means you're not getting enough power to the devices you plug in. Auxiliary power input doesn't mean anything, at least not when the issue is at the output end. Maybe try a USB power tester?

One time many moons ago I worked for Lanner in Taiwan. Back then they had a basic NAS with a Marvell Kirkwood Arm chip in it. They've gone quite upmarket since then hardware wise.

My own DIY NAS is built around a Core i7-6700K which really is overkill, but same "issue" as yours, not that power efficient.

 

[DJones]

My turn key nas by Asustor AS5202T is my main NAS sits at about 10W, but since the whole qnap,asustor,synology ransomware I don’t let it touch the internet anymore. Glad I keep backups thankfully wasn’t hit by it.

I’ll see what the changes I’ve made do, the issue is unfortunately intermittent usually during a bad boot. I’ve ordered a usb power tester because it seems handy to have in my tool kit. Thanks for the advice.

Edit: as usual I speak too soon.

[  695.302586] xhci_hcd 0000:01:00.0: xHCI host controller not responding, assume dead
[  695.302648] xhci_hcd 0000:01:00.0: HC died; cleaning up
[  695.302726] usb 4-1: USB disconnect, device number 2
[  695.302831] aqc111 4-1:1.0 enx245ebe655241: unregister 'aqc111' usb-0000:01:00.0-1, QNAP QNA-UC5G1T USB to 5GbE Adapter
[  695.303363] aqc111 4-1:1.0 enx245ebe655241: Failed to read(0x1) reg index 0x0002: -19
[  695.303373] aqc111 4-1:1.0 enx245ebe655241: Failed to write(0x1) reg index 0x0002: -19
[  695.303378] aqc111 4-1:1.0 enx245ebe655241: Failed to write(0x1) reg index 0x0002: -19
[  695.303382] aqc111 4-1:1.0 enx245ebe655241: Failed to write(0x61) reg index 0x0000: -19
[  695.303463] vmbr0: port 1(enx245ebe655241) entered disabled state
[  695.311475] aqc111 4-1:1.0 enx245ebe655241 (unregistering): left allmulticast mode
[  695.311484] aqc111 4-1:1.0 enx245ebe655241 (unregistering): left promiscuous mode
[  695.311489] vmbr0: port 1(enx245ebe655241) entered disabled state
[  695.414273] aqc111 4-1:1.0 enx245ebe655241 (unregistered): Failed to write(0x1) reg index 0x0002: -19
[  695.414285] aqc111 4-1:1.0 enx245ebe655241 (unregistered): Failed to write(0x1) reg index 0x0002: -19
[  695.414289] aqc111 4-1:1.0 enx245ebe655241 (unregistered): Failed to write(0x61) reg index 0x0000: -19
[  695.414673] usb 4-2: USB disconnect, device number 3

Interestingly it's not kicking the whole usb pcie card now just the usb Ethernet. Once the tester comes in I can see how much it's pulling.

I've added this also to grub to prevent auto suspend of usb power.

usbcore.autosuspend=-1

and

systemctl mask sleep.target suspend.target hibernate.target hybrid-sleep.target

Annoyingly aqc111 has a driver from Qnap/Marvell for the QNAP QNA-UC5G1T that has a low power 5G mode, but it's not flag able in ethtool because they moved the driver to the linux kernel and it's missing the extra features.

Alternatively a external powered usb hub may also be the answer if it is in fact a power issue.

 

[TheLostSwede]

Alternatively a external powered usb hub may also be the answer if it is in fact a power issue.

That could potentially fix the problem, yes.

 

[DJones]

Short update. A powered usb hub didn’t solve the problem.. plugging the usb hub in with anything actually on the hub causes the server to freak out with hub_ext_port_status failed (err = -71) , no clue what that error means.

I’ve given up lol. USB ports works with the external drives, I just can’t make it go brrr for smb without the 5G nic, will have to settle for LACP. Multichannel doesn’t work in a proxmox vm least I don’t know how to make it work without performing witchcraft and some kind of sacrifice. But for another time.