IoT Botnet, or GUI Bugs?

[HapaxLegomenon]

Hi all,

First of all apologies in advance for what might be an amateurish or potentially annoying post. I swear there's only 2 voices in my head


Here's the backstory:

I replaced an old AC router in a guest house with my RT-AC66U B1 running Merlin (had not been used since 2021). I was connecting IoT devices to the RT-AC66U B1, so I created a 2.4GHz Guest network (index 3) in addition to the main 2.4GHz and 5GHz networks

I set the Merlin 2.4GHz Guest network to the same SSID, but different password, as the old router's 2.4GHz Guest network. Intranet access is disabled for the Merlin Guest network.

The first IoT device I connected was an old Logitech LogiCircle camera that had been connected to the old router since 2016 (so, the LogiCircle has seen the Merlin Guest SSID before). I factory reset the LogiCircle and then connected it to the 2.4GHz Merlin Guest network. LogiCircle connected just fine and footage was showing up in the app. So far so good.

I moved on to connecting the next device, a brand new Google Nest Cam. I also connected this device to the same 2.4GHz Merlin Guest network. Footage showed up in the Google Home app, so I thought everything was in order.

Ten minutes later, the Google Home app reported that my Nest Cam's "live stream was unavailable," but it didn't report that the camera was disconnected from the internet.

So I checked out the list of devices in Merlin, and under "View List [of Clients] -> Interface," my Nest Cam was reported as being connected to my 2.4GHz MAIN NETWORK (not Guest).

At the same time, the LogiCircle had a static IP symbol next to it, despite no static IPs being set under DHCP Server settings


I immediately factory reset the Nest Cam and unplugged the LogiCircle. I haven't connected either device back since.


I was wondering if might be a possibility that the LogiCircle, which hasn't seen a new firmware update in ~5 years, had been infected by a Botnet or other malware while being connected for many years to the old router. If this were the case, I was thinking the supposed Botnet creator might have gotten access to the new Nest Cam via the LogiCircle, as these were the only two devices connected to the Merlin 2.4GHz Guest network. Would it have been possible for said Botnet creator to, via the LogiCircle, then gain access to another IoT device on the same Guest network (the Nest Cam), and then somehow manage to command the Nest Cam to change connection from the 2.4GHz Guest network to the 2.4GHz Main network? I don't know how they would have obtained the password for the 2.4GHz Main network though (potentially from the password manager on my phone).

Another potential clue I noticed, perhaps irrelevant, was that I was using the Cloudflare DNS test website on my phone at around the same time. While connected to WiFi, the test website reported "AS Name" as "Cloudflare." Okay, good. I disconnected my phone from WiFi and refreshed the page. Then "AS Name" showed up as "ER-Telecom," a Russian telecommunications company I've never heard of. I thought this was very odd since I'm based in the United States. I refreshed the page again and then it changed to "Verizon," my carrier. So I thought, either Cloudflare or Verizon rely on "ER-Telecom" sometimes for brief services, or my phone/network/IOT devices have been compromised by a Russian.

Alternatively, in the past couple of days, I noticed some potential GUI bugs in the ASUS Client List. I haven't set any Static IPs, but some of my devices will periodically show up with the Static IP symbol next to them. Before it was my laptop. Now it's my Amazon TV. No static IPs are showing up as being manually set in DHCP Server settings. Are static IPs for IoT devices beneficial to Botnets?

Or could it be the case that my Nest Cam showing up as connected to my 2.4GHz Main network, was simply a GUI bug? Is it even possible for a Merlin-running ASUS router to mistakenly report a device connected to the 2.4GHz Guest network, as being connected to the 2.4GHz Main network?

A potentially relevant fact is that I haven't done a hard reset on the RT-AC66U B1 since 2021, and I directly updated its firmware from 386.2_4 to 386.10_0 without doing a hard reset. Could that be the cause of the purported GUI bugs I'm seeing? Or are bugs of that nature impossible, and is it actually the case that my IoT/Network/Router/Phone have been compromised by a Russian? Anyone have any advice on how to figure out which possibility is correct? I haven't noticed any other network abnormalities like malware or popups or being redirected to fake websites on any of my devices.

Thank you very much!

HL

 

[AndreiV]

I got part way down to "used the same SSID" ...........

Your Guest network needs a different SSID.

 

[drinkingbird]

I got part way down to "used the same SSID" ...........

Your Guest network needs a different SSID.

They said same SSID as the old router's guest network not the main LAN.

 

[drinkingbird]

A potentially relevant fact is that I haven't done a hard reset on the RT-AC66U B1 since 2021, and I directly updated its firmware from 386.2_4 to 386.10_0 without doing a hard reset. Could that be the cause of the purported GUI bugs I'm seeing? Or are bugs of that nature impossible, and is it actually the case that my IoT/Network/Router/Phone have been compromised by a Russian? Anyone have any advice on how to figure out which possibility is correct? I haven't noticed any other network abnormalities like malware or popups or being redirected to fake websites on any of my devices.

Thank you very much!

HL

If you search here you'll see your router can barely run 386.9 and up. And the only way to have it run properly without GUI bugs and other strange behavior is a full WPS factory reset and reconfigure from scratch without many custom client names or DHCP reservations, and even then it will be a close call.

I would stick with 386.7_2 on that router and do the full reset and reconfigure.

Actually I'd recommend doing the reset both before and after the firmware downgrade, just in case the router has been compromised (unlikely, but this is best practice anyway).

If you have WAN access enabled ensure it is HTTPS only and using strong username and password.

For the two devices in question, factory reset and if possible reload the firmware (some will let you reload the same firmware, but if not you can try downgrading then upgrading again, once you're sure your network is secure).

I suspect you probably weren't hacked and this is just related to the limitations of the new firmware on your old router.

 

[HapaxLegomenon]

I got part way down to "used the same SSID" ...........

Your Guest network needs a different SSID.

They said same SSID as the old router's guest network not the main LAN.

Indeed as per my original post, the Merlin Guest network has the same SSID as the old router's Guest network. The new Merlin WLAN networks have different SSIDs from the Merlin Guest network.

 

[HapaxLegomenon]

If you search here you'll see your router can barely run 386.9 and up. And the only way to have it run properly without GUI bugs and other strange behavior is a full WPS factory reset and reconfigure from scratch without many custom client names or DHCP reservations, and even then it will be a close call.

I would stick with 386.7_2 on that router and do the full reset and reconfigure.

Thanks so much for helping out! Yeah I'm strongly considering moving to 386.7_2 until Merlin can circumvent the NVRAM issues, hopefully in 386.11.

If you have WAN access enabled ensure it is HTTPS only and using strong username and password.

WAN remote access was disabled, SSH access was only enabled via LAN (with port changed), and admin access was HTTPS only. The router admin page was set to a strong username/password.

I had a couple more questions I was wondering you'd be able to answer:

1. Is my assumption correct that, if I create a new index 2 Guest network, this will be on a separate VLAN than the index 3 Guest network? My plan is to have the LogiCircle as the only device connected to the new index 2 Guest network to isolate it, as I cannot reflash or downgrade/upgrade its firmware. Will the LogiCircle then be unable to access my other IoT devices, in the slim case it actually has been compromised?
2. Is there a way to explain the temporary "ER-Telecom" (Russian company) AS Name showing up on the Cloudflare DNS test page? Maybe just a Cloudflare test page idiosyncrasy?
3. Not sure if relevant, but "Last Device Seen" by the WAN is reported as having a MAC address starting with "3C:41," corresponding to a Cisco device, despite the Merlin router never having been connected to a Cisco device. Maybe this was just my ISP.

Glad to hear that the GUI bugs are likely caused by the limitations of my RT-AC66U B1. I'm leaning toward that explanation but I'm being extra cautious because I remembered I absentmindedly let my technologically-illiterate aunt join my main network last year for a couple days (on the old router), who might have visited malicious websites or clicked on compromised links. Definitely won't be doing that again. Any tell-tale signs my router/phone/laptop have been compromised apart from the obvious spoofed websites/popups/malware?

Thanks again!

HL

 

[drinkingbird]

Thanks so much for helping out! Yeah I'm strongly considering moving to 386.7_2 until Merlin can circumvent the NVRAM issues, hopefully in 386.11.


WAN remote access was disabled, SSH access was only enabled via LAN (with port changed), and admin access was HTTPS only. The router admin page was set to a strong username/password.

I had a couple more questions I was wondering you'd be able to answer:

1. Is my assumption correct that, if I create a new index 2 Guest network, this will be on a separate VLAN than the index 3 Guest network? My plan is to have the LogiCircle as the only device connected to the new index 2 Guest network to isolate it, as I cannot reflash or downgrade/upgrade its firmware. Will the LogiCircle then be unable to access my other IoT devices, in the slim case it actually has been compromised?

Guest Wireless 1 (on 386 code base) is the only one that gets isolated to a separate VLAN (as long as you disable intranet access). Actually two VLANs, one for 2.4Ghz and one for 5Ghz. Each one gets its own 192.168. subnet separate from the LAN. Guest Wireless 2 and 3 use the old style where they share a subnet with the main LAN and firewall rules block access to/from the main LAN. Unlike GW2 and 3, GW1 is actually accessible from the main lan, but it can't initate any connections to the main LAN. This is useful if you want to manage IOT devices from your trusted LAN etc, while not letting those devices initiate any connections to your trusted LAN. Basically it only allows those devices to reply to packets initiated from the main LAN.

All 3 Guest Wireless networks isolate clients from each other (if you disable intranet access, it also enables AP isolation so the clients can't see each other). So those two devices can't talk to each other even if they're on the same Guest Wireless. I've tested it on my RT-AC1900 (same as RT-AC68U) with 386.7_2 using both types of guest and the devices can't even get an ARP for each other, much less ping etc. So you don't really need to worry about putting them on separate networks. You can, it won't hurt anything, but shouldn't be necessary. You can certainly test the behavior on your setup with a couple laptops or anything that responds to ping. Make sure they can ping each other on the main LAN, then move them both to guest and they should no longer be able to ping. Then put one on GW1 and one on the main LAN. The one on the main LAN should be able to ping the one on GW1, but not the other way around.

I like using GW1 just because the different subnets make it obvious, and I do like having the VLAN separation, even though it really still relies on the firewall. In my case, I extend those VLANs to another AP and a switch (for wired guests/fixing PCs I don't trust) so they come in handy for that too. I do have GW2 enabled for my IOT stuff, technically it could all be on guest wireless 1 and be isolated from each other and the main LAN, but just helps me organize stuff.

1. Is there a way to explain the temporary "ER-Telecom" (Russian company) AS Name showing up on the Cloudflare DNS test page? Maybe just a Cloudflare test page idiosyncrasy?

Not really sure on that, my guess is just a glitch or the test page got confused when you refreshed it from a different source IP. They briefly tried routing out a roundabout path. Not really sure, would need more testing to try and figure it out but I wouldn't worry about that. The AS number could have been loaned out to another company too even though it is registered to ER. Internet AS paths can vary a lot, or maybe their default route when they can't find a route is via ER, or the DNS server it randomly hit routed that way. Again all just guesswork without testing.

1. Not sure if relevant, but "Last Device Seen" by the WAN is reported as having a MAC address starting with "3C:41," corresponding to a Cisco device, despite the Merlin router never having been connected to a Cisco device. Maybe this was just my ISP.

Probably ISP device. Some ISPs use cisco modems/routers at your site and many use them for edge routers too. What is/was the WAN plugged into? Doubt that is anything to be concerned with.

Glad to hear that the GUI bugs are likely caused by the limitations of my RT-AC66U B1. I'm leaning toward that explanation but I'm being extra cautious because I remembered I absentmindedly let my technologically-illiterate aunt join my main network last year for a couple days (on the old router), who might have visited malicious websites or clicked on compromised links. Definitely won't be doing that again. Any tell-tale signs my router/phone/laptop have been compromised apart from the obvious spoofed websites/popups/malware?

On the router you'll see port forwarding rules get created, WAN access get enabled, lots of connections sourcing from the router out to various places, etc. But if you factory reset using the WPS button, update the firmware, reset again, and reconfigure from scratch (not backup) that would all be gone. Unlikely that anything that infected her computer (if it did) would have gotten onto your PC or Phone, or even your router unless you have default passwords or no password etc. You'd have to go out of your way to enable access to your PC or phone in most cases, unless they haven't been updated/patched in quite a while and they exploited some old vulnerability. You can enable one of the guest wireless for actual guests, separate from your main LAN and your cameras, etc (though even if you have a guest on your camera LAN, it would be isolated anyway). Guest devices cannot access the router GUI at all, the only communication allowed to the router IP is DNS and DHCP.

One of the best ways to tell if something is amiss is to look at active network connections. It can be hard to filter through but if you close everything out and see a lot of connections going out to IPs that are not normal or known, something is up. You can look at connections in the router GUI and try to identify if everything is connecting to stuff it is supposed to be. On a PC you can look at netstat also. Just keep in mind there will be several connections related to windows, probably google chrome even if it is closed and other tasks/programs you have installed. Typically they'll resolve to a recognizable name and if not, you can look up who the IP is registered to, tracert to it to see if it is going to another country, etc.

The other things I look for are running processes (in task manager) that look suspect (not always easy to spot as they will call it something that looks legit, but if you look at the path of something that seems to be using CPU regularly and it is going to something odd, that's not a good sign), and go into task scheduler and look for tasks that are rogue (can be hard to spot, usually they'll call it "windows update" or something like that, but you have to look at the actual file it runs, if it is a VBS script or a compiler like aspnet_compiler, that is not good). Often you get lucky and they've spelled something wrong, or haven't capitalized it when most tasks have the first letter capitalized, something like that. Look at startup programs in task manager too. You can build a USB offline virus scanner (lots of free ones out there from Bitdefender, Malwarebytes, etc) from a trusted/known good computer and boot off that to scan as well, though no virus scanner will catch everything, the people making viruses now have gotten really good at working around even heuristic scans. There is one particularly bad one going around called Agent Tesla that gives remote access, keylogging, etc and it is written in such a way that it almost always avoids detection. It was originally built as a "legit" app, or at least that's what it claimed, but the source code was sold off to any hacker willing to pay for it. So in some cases if it is running, the virus scanner may consider it legit. One fairly obvious clue with that one is that some variants prevent your computer from going to sleep, and some even keep the screen on. That is a red flag right away. In investigating that one I saw that in the code instead of doing like c:\script.vbs they were echoing each letter one at a time (as if it was being typed in from the keyboard) so nowhere in the code would a virus scanner see the obvious vbs script being called, and even when it was run, it looked to the virus scanner like you were typing in the command. Honestly I don't know why every virus scanner doesn't just at least warn you when a vbs script is run, even if you run it yourself.

I think you're probably fine, most likely NVRAM issues on that router since you didn't do a factory reset, which can cause all kinds of weirdness. Even when working perfectly fine, the client list can be a tad buggy, though on 386.7_2 mine has been fine.

 

[HapaxLegomenon]

Guest Wireless 1 (on 386 code base) is the only one that gets isolated to a separate VLAN (as long as you disable intranet access). Actually two VLANs, one for 2.4Ghz and one for 5Ghz. Each one gets its own 192.168. subnet separate from the LAN. Guest Wireless 2 and 3 use the old style where they share a subnet with the main LAN and firewall rules block access to/from the main LAN. Unlike GW2 and 3, GW1 is actually accessible from the main lan, but it can't initate any connections to the main LAN. This is useful if you want to manage IOT devices from your trusted LAN etc, while not letting those devices initiate any connections to your trusted LAN. Basically it only allows those devices to reply to packets initiated from the main LAN.

Got it - thanks for clarifying! So it seems like GW1 would be good for a WiFi printer where the main LAN devices would need to access the GW1-connected printer, but any connection requests from the printer to the main LAN would be blocked. And because of firewall rules, GW2 and GW3 are no worse than GW1 for IoT security, despite sharing the same subnet as the main LAN? I'd previously avoided GW1 due to general forum advice that it's "wonky" and "specific to AiMesh," but I'm going to start using it now!

All 3 Guest Wireless networks isolate clients from each other (if you disable intranet access, it also enables AP isolation so the clients can't see each other). So those two devices can't talk to each other even if they're on the same Guest Wireless.

Ah, so in the case of the LogiCircle and the Nest Cam, assuming my router's "Intranet Access Disabled" was indeed effectuated, even if the LogiCircle were compromised it wouldn't even have been able to see the Nest Cam, let alone have it switch from the Guest network to the Main network.

Not really sure on that, my guess is just a glitch or the test page got confused when you refreshed it from a different source IP. They briefly tried routing out a roundabout path. Not really sure, would need more testing to try and figure it out but I wouldn't worry about that.

Okay phew! Glad to hear.

Probably ISP device. Some ISPs use cisco modems/routers at your site and many use them for edge routers too. What is/was the WAN plugged into? Doubt that is anything to be concerned with.

The WAN was/is plugged into a NETGEAR modem with a different MAC address than the Cisco device reported in Merlin.

On the router you'll see port forwarding rules get created, WAN access get enabled, lots of connections sourcing from the router out to various places, etc. But if you factory reset using the WPS button, update the firmware, reset again, and reconfigure from scratch (not backup) that would all be gone. Unlikely that anything that infected her computer (if it did) would have gotten onto your PC or Phone, or even your router unless you have default passwords or no password etc. You'd have to go out of your way to enable access to your PC or phone in most cases, unless they haven't been updated/patched in quite a while and they exploited some old vulnerability. You can enable one of the guest wireless for actual guests, separate from your main LAN and your cameras, etc (though even if you have a guest on your camera LAN, it would be isolated anyway). Guest devices cannot access the router GUI at all, the only communication allowed to the router IP is DNS and DHCP.

One of the best ways to tell if something is amiss is to look at active network connections. It can be hard to filter through but if you close everything out and see a lot of connections going out to IPs that are not normal or known, something is up. You can look at connections in the router GUI and try to identify if everything is connecting to stuff it is supposed to be. On a PC you can look at netstat also. Just keep in mind there will be several connections related to windows, probably google chrome even if it is closed and other tasks/programs you have installed. Typically they'll resolve to a recognizable name and if not, you can look up who the IP is registered to, tracert to it to see if it is going to another country, etc.

The other things I look for are running processes (in task manager) that look suspect (not always easy to spot as they will call it something that looks legit, but if you look at the path of something that seems to be using CPU regularly and it is going to something odd, that's not a good sign), and go into task scheduler and look for tasks that are rogue (can be hard to spot, usually they'll call it "windows update" or something like that, but you have to look at the actual file it runs, if it is a VBS script or a compiler like aspnet_compiler, that is not good). Often you get lucky and they've spelled something wrong, or haven't capitalized it when most tasks have the first letter capitalized, something like that. Look at startup programs in task manager too. You can build a USB offline virus scanner (lots of free ones out there from Bitdefender, Malwarebytes, etc) from a trusted/known good computer and boot off that to scan as well, though no virus scanner will catch everything, the people making viruses now have gotten really good at working around even heuristic scans. There is one particularly bad one going around called Agent Tesla that gives remote access, keylogging, etc and it is written in such a way that it almost always avoids detection. It was originally built as a "legit" app, or at least that's what it claimed, but the source code was sold off to any hacker willing to pay for it. So in some cases if it is running, the virus scanner may consider it legit. One fairly obvious clue with that one is that some variants prevent your computer from going to sleep, and some even keep the screen on. That is a red flag right away. In investigating that one I saw that in the code instead of doing like c:\script.vbs they were echoing each letter one at a time (as if it was being typed in from the keyboard) so nowhere in the code would a virus scanner see the obvious vbs script being called, and even when it was run, it looked to the virus scanner like you were typing in the command. Honestly I don't know why every virus scanner doesn't just at least warn you when a vbs script is run, even if you run it yourself.

Thanks so much for all this! I'm definitely going to be monitoring active network connections, running processes, scheduled tasks, and startup programs from now on. But the best thing is I'm learning a lot and I really appreciate it!

I think you're probably fine, most likely NVRAM issues on that router since you didn't do a factory reset, which can cause all kinds of weirdness.

No more dirty flashing for me!

Best wishes,

HL

 

[drinkingbird]

Got it - thanks for clarifying! So it seems like GW1 would be good for a WiFi printer where the main LAN devices would need to access the GW1-connected printer, but any connection requests from the printer to the main LAN would be blocked. And because of firewall rules, GW2 and GW3 are no worse than GW1 for IoT security, despite sharing the same subnet as the main LAN? I'd previously avoided GW1 due to general forum advice that it's "wonky" and "specific to AiMesh," but I'm going to start using it now!

GW1 had some issues when 386 first came out but it works fine now, have been using it for quite a while. In the scenario you describe, yes, you would be able to print from the main LAN. However guests would not be able to print, and anything you try to initiate from the printer (like a scan to the PC) would not work either. I actually have my printer on the main LAN and added firewall rules to allow guest to print to it. Not too worried about my old Officejet getting compromised.

From what I can see in the firewall rules, there really is no extra security having the VLANs configured, they just did it to allow guest wireless to propagate to the nodes in AiMesh and be able to isolate them at the node as well (previously not possible). I misspoke earlier, it is not using AP isolation, it is using EBTABLES (a layer 2 firewall) to prevent the communication between guests, on both "models" of guest (VLAN Isolated GW1 and non-VLAN GW2 and 3).

It is convenient though as you can use a fairly simple script to extend the VLAN out to another AP, or to switch ports to have wired guest ports, etc. That's what I'm doing (both). In my case my outdoor AP is not an Asus so can't use Aimesh, but accomplishing the same thing by tagging the VLAN out to that AP so I can have both regular and guest outside too. And wired guest ports are handy for certain scenarios too, before that if I was fixing someone's PC or using my "testing/risky stuff" desktop that I keep in a corner, I was putting a USB wifi adapter in it which was slower and sometimes an issue with drivers etc.

Ah, so in the case of the LogiCircle and the Nest Cam, assuming my router's "Intranet Access Disabled" was indeed effectuated, even if the LogiCircle were compromised it wouldn't even have been able to see the Nest Cam, let alone have it switch from the Guest network to the Main network.

Correct, assuming they didn't get into your router and change the firewall rules to allow the communication between devices on Guest. That would have to be one decent hacker to compromise 3 separate devices.

The WAN was/is plugged into a NETGEAR modem with a different MAC address than the Cisco device reported in Merlin.

Is it truly a modem or is it a router too? If just a modem, likely the MAC is of the router beyond that modem (your ISP edge router in their central office).

Thanks so much for all this! I'm definitely going to be monitoring active network connections, running processes, scheduled tasks, and startup programs from now on. But the best thing is I'm learning a lot and I really appreciate it!

In this day and age, getting familiar with how to look for possible compromises is always a good thing. Unfortunately with everything (windows, MS office, web browsers, etc) constantly connecting to various things it can be a bit harder than in the past but once you get an idea of normal stuff vs suspicious stuff (even if you then trace it out and it isn't suspicious after all) it is still one of the best ways to find something malicious. Just about every piece of malware is going to "phone home" these days. In addition to netstat, many 3rd party firewalls on your PC will show you active connections and sometimes the formatting there is easier to sift through. The list in the Asus is similar to netstat but a good central place to look at all connections going out from all devices too.