Diving into ac5300 firmware - aka CFE's reverse engineering and firmware hacking

[darkspr1te]

The reason for this post is so I can cover how I am recovering a asus rt-AC5300 back from the dead, or hope too.
In this thread i will be covering NAND reading, ecc systems, CFE's , bootloaders in general , JTAG , software source code & compiling.
It wont be a easy read to do guide as it's a project in motion and I may incorrectly cover items initially.

So the short story is i have a few bcm47xx arm routers in a bin of electronics I collect and i figured i would give a crack at reverse engineering and possibly reviving these routers, documenting it along the way.
I am in now way a super coder, hacker or what ever just a hobbyist who likes a challenge and this policy has allowed me to recover quite a few devices in the past and come up with usable end-user solutions, and the documentation also allows others to learn and also transfer the methods to other or similar devices.
So onto the meat and potatoes of it.

My two subjects are one Asus rt-ac5300 and one netgear nighthawk x6,
Both contain the same main CPU a Broadcom BCM4709C0KFEBG dual-core @ 1.4 GHz
Both use the Spansion S3ML01G1 Nand device
both have full JTAG points and respond to ID_CODE requests


Now at the time these originally came across my desk two years ago there was no info on the JTAG side and even less about the CFE in use, how it works etc but thats changed a little and it's worth revisiting.
So far I have now removed both NANDS from the system dump via other means, I used a STM32 dev board,
found here VCCGND mini407

Once that was done i ported the NANDO firmware from the original stm32f103 chip to a stm32f407 based chip, found here for the nando original firmware and my port is here
and after some issues, head scratching , reaching out to experts (see that dialog here ) i finally have direct nand dumps (including spare area) to work with.

Those files can be found here , so far online i have not found copies of the ac5300 cfe,nvram table or even nand files so these may be the only copies online as most have given up or replaced the h/w so there is little desire to work to get them.

now out of the two units i will be digging into the ac5300 in more depth and use the nighthawk as example of changes between two providers and how that might possibly allows us to fix the ac5300

attached is a picture comparing the start of both NAND dumps ,

now it seems the nand layout is as follows
0x0-0x3f reset vector table maybe ?
0x3e0-0x3ff ZSIB - magic header for CPU to configure with
0x400-??? - FLSH - magic header for NVRAM config , defined in the source code with #define NVRAM_MAGIC 0x48534C46 /* 'FLSH' */, found the the file bcmnvram.h in merlin source codes. note the little indian cpu loading here

Now here where you will note the first real changes, the left image is the nighthawk and the right is the asus, you will note the asus has no text just garbled letters, well originally most thought it was encrypted , turn outs its not , it is LZMA compressed with a slight altered dictionary table ,

This is the nvram for the rt-ac5300 thats stored there , the tool to decompress it is found here , it does take some patching to compile and at some point i will upload my working source code and guide to my github,

"note, will post link to nvram soon" over char limit


Well now that we dumped NVRAM area and decompressed it we can assume this point of the NAND is good up to this point or the LZMA decompress would fail, something to note.

Now the section we just dumped is only the NVRAM and the CFE follows that ,
the entire cfe actually starts at 0x414 (hex search for 5D 00 00 01) and can be decompressed with this tool LZMA or LZMA_4K found here , this will decompress the routers cfe code, if you wish to dig deeper right now before i write up my own solution the see this chinese write up here , it also covers building the cfe and reverse engineering this ,

here is dump I extracted, it's the uncompressed cfe as loaded into ram, found here ,
So again, we have decompressed the data so again this section of NAND is good, well in this specific case,

if i run binwalk on the extracted nand i get this following

DECIMAL       HEXADECIMAL     DESCRIPTION
--------------------------------------------------------------------------------
2097152       0x200000        TRX firmware header, little endian, image size: 35651584 bytes, CRC32: 0x6C36F8DF, flags: 0x0, version: 1, header size: 28 bytes, loader offset: 0x1C, linux kernel offset: 0x1C2644, rootfs offset: 0x0
2097180       0x20001C        LZMA compressed data, properties: 0x5D, dictionary size: 65536 bytes, uncompressed size: 4486144 bytes
3941956       0x3C2644        Squashfs filesystem, little endian, version 4.0, compression:xz, size: 33801525 bytes, 3204 inodes, blocksize: 131072 bytes, created: 2021-08-06 22:24:14
37840154      0x241651A       xz compressed data
<snip repeats>
45809154      0x2BAFE02       xz compressed data
45836314      0x2BB681A       PNG image, 310 x 277, 8-bit/color RGBA, non-interlaced
45837210      0x2BB6B9A       Zlib compressed data, best compression
45923259      0x2BCBBBB       PNG image, 34 x 34, 8-bit/color RGBA, non-interlaced
45925007      0x2BCC28F       PNG image, 34 x 34, 8-bit/color RGBA, non-interlaced
45926562      0x2BCC8A2       PNG image, 34 x 34, 8-bit/color RGBA, non-interlaced
45927863      0x2BCCDB7       PNG image, 130 x 87, 8-bit/color RGBA, non-interlaced
45927941      0x2BCCE05       Zlib compressed data, best compression
45933139      0x2BCE253       PNG image, 97 x 71, 8-bit/color RGBA, non-interlaced
45933217      0x2BCE2A1       Zlib compressed data, best compression
45937344      0x2BCF2C0       PNG image, 97 x 71, 8-bit/color RGBA, non-interlaced
45937422      0x2BCF30E       Zlib compressed data, best compression
45940388      0x2BCFEA4       PNG image, 128 x 64, 8-bit/color RGBA, non-interlaced
45940466      0x2BCFEF2       Zlib compressed data, best compression
45955653      0x2BD3A45       PNG image, 130 x 87, 8-bit/color RGBA, non-interlaced
45955752      0x2BD3AA8       Zlib compressed data, best compression
45965257      0x2BD5FC9       xz compressed data
46021813      0x2BE3CB5       xz compressed data
46152749      0x2C03C2D       xz compressed data
46276821      0x2C220D5       xz compressed data
46339965      0x2C3177D       xz compressed data
46429682      0x2C475F2       PNG image, 66 x 66, 8-bit/color RGBA, non-interlaced
46429767      0x2C47647       Zlib compressed data, best compression
46432437      0x2C480B5       Zlib compressed data, best compression
46439299      0x2C49B83       PNG image, 90 x 270, 8-bit/color RGBA, non-interlaced
46439340      0x2C49BAC       Zlib compressed data, compressed
46453465      0x2C4D2D9       xz compressed data
46574265      0x2C6AAB9       xz compressed data
46668289      0x2C81A01       xz compressed data
46763137      0x2C98C81       xz compressed data
46858745      0x2CB01F9       xz compressed data
46891813      0x2CB8325       xz compressed data
46901241      0x2CBA7F9       xz compressed data
46954169      0x2CC76B9       GIF image data, version "89a", 580 x 344
47085241      0x2CE76B9       xz compressed data
<snip repeats>
49680373      0x2F60FF5       xz compressed data
73531392      0x4620000       JFFS2 filesystem, little endian

Next will extract the trx side of the firmware (magic code is HDR0 in firmware sources , source code is found here on github )

I will write up the next section soon.


darkspr1te

 

[darkspr1te]

Some side articles on broadcom's CFE ,
Reverse engineer tplink cfe
unpacking huawei router cfe
Broadcom CFE pdf

 

[darkspr1te]

Further links on Broacom JTAG
broadcom MIPS via Openocd/JTAG
another broadcom mips and openocd thread
JTAG a attackers perspective white paper

 

[thiggins]

Apologies. Delete was accidental. Next time contact admin directly. I’ve deleted your other thread complaining about this one being deleted.

 

[sfx2000]

Food for thought...

https://www.kernel.org/doc/Documentation/arm/Booting

Every ARM has a boot vector, a memory address that it first looks for an application - in this case, it's the bootloader for CFE (could be uboot, whatever).

Key thing is - find a working device, and unmount the NAND flash, and burn an image from it - then all the hard stuff is done...

 

[darkspr1te]

Food for thought...

https://www.kernel.org/doc/Documentation/arm/Booting

Every ARM has a boot vector, a memory address that it first looks for an application - in this case, it's the bootloader for CFE (could be uboot, whatever).

Key thing is - find a working device, and unmount the NAND flash, and burn an image from it - then all the hard stuff is done...

Yes the first few bytes of the nand is the NVIC setup and first jump addresses, then it's the DRAM configs, then the NVRAM configs, I have a whole write up on that part which i will post down the line. Most of that info is out there but is either behind the great firewall of china or would violate forums rules if I was to discuss here so I am being careful how i word that write up as I am covering recovery methods and not "out of territory" modifications.
As a few things have come about since my first few posts in that I have confirmed with CRC's and LZMA decompression such that the CFE in this nand and it's NVRAM and the TRX firmware files are intact, So as to why this unit did not boot is a mystery right now.
As of right now I am just finishing up the code to restore a new NAND (yes i like to write my own code even though there are tools there to do it)
this nand will be soldered onto the board "with" the stm32f407 board so i can flash via usb when i want. It will also allow me to confirm that the JTAG commands i have sent to init the NAND & DMA/DDR controller has worked (i will be able to see the CPU ask for the nand ID )
Sadly jtag on ARM is not as swift as say a MIPS cpu, MIPS has EJTAG and it's well documented and works across MIPS in a universal was (eg memory setup routines are common) so that make it often easier to work with them for recovery purpose, ARM is a little harder as a few things happen in arm that MIPS does not do, a example would be on the Broadcoms in ras Pi's , in the bootloader jtag is turned off and handed over to the GPIO and the same goes for asus arm routers, jtag shuts off very early in the boot routine leading to issues writing new NAND data , you can get around this by adding a value to the NVRAM (the one thats compressed and located at 0x400) , there actually a few NVRAM values not listed in NVRAM but can be added to enable features like CFE console access , always boot to tft/web server regardless of flash status and a few more.

to reply to sfx2000, yeah I have looked around locally for a used unit but my location does not have much tech stores and online purchases come with high shipping costs even for air. So next best thing was to spend some time to see if i can come up with a User side recovery method. I could have done it all in the dark and just 'released' a tool or guide but then no one learns from my methods, already a few people have contacted me regarding info i have mentioned that not common knowledge and alters prior thoughts on the boot process , Before i started digging in i though it was a case of boot cfe first then kernel but we have a extra stage now, the cfe decompression and loading into memory, thats a separate program loader and as to why this is there as not on other brand routers?, well feature creep, the CFE has a web server, tftp , serial, memory loading and control routines and more which are not present in the vanilla broadcom cfe sources.

Another thing that I found is to why you cant just copy the NVRAM lzma portion from one device and put on another, in the uncompressed nvram is this value
secret_code=xxxxxxxx and thats checked by the CFE on boot against a code in the cpu, individual to each cpu and you have to know the right value which you can only get from a booted system, how ever another person I am working with has mentioned there is a way of putting a generic code and another value and the system will write the correct code and resave the nvram, we assume this is for "first boot scenario" and it's so that a generic flash can be written and then the system setups up it's own security signed for the flash data (yet to be really proven but the code kinda fits it as of now)

If this is the case and we can get JTAG to function then we will be able to produce a recovery flash file for many devices, already we found how to tell the pre-CFE-loader to use a uncompressed NVRAM +CFE , if you look at the NAND dump at address 0x404 is the CFE size , then at 0x408 are the NVRAM type values, 0x69 is uncompressed and 0x99 is compressed. How ever very early CFE wont support this change as they ONLY use uncompressed NVRAM, then you have the hybrid CFE which support both LZMA and uncompressed CFE and then after that ONLY LZMA compressed NVRAM , I dont have to hand the version numbers yet as I still have to compile those notes.
My fellow developer did also mention that so far in his digging that is the device came with uncompressed LZMA and then later had compressed then it's the same CFE they just enabled the feature to add more NVRAM values and can be switched back , however if the device from first boot had LZMA NVRAM then it cannot use uncompressed NVRAM at all and will cause failure. to fix this you can use the LZMA tool in merlin sources to re-create your NVRAM in compressed form or the braodcom_cfe_tool mentioned above , eagle eyed among you will notice in the compressed NVRAM when uncompressed mentions which territory the device is used in, this is only a place holder and changing it does nothing and you will find it resets back if you change it, worse if you put totally wrong value there it will hang the system , this is because the CFE read this value first then confirms it against another HW value and if the reading of it fails due to too long a value or incorrect formatting then the HW compare wont even happen , instead you have a memory overflow and the system resets putting you into a bootloop you cant get out of as this is all done before you even see the CFE text prompt via a serial port. but we did find removing the value would restore the boot and the cfe would the default to using HW id and write that as it's assigned territory restoring the value. But this took desoldering the NAND and dumping NVRAM to restore the original values, something that made my fellow investigator comment that we so far found a 100 ways to break the device and none to fix it. But that be the way


Hope fully in my next post I will have more info to write on the JTAG side, right now our target is RAM loading a CFE which we know has to be done from the uncompressed file (use LZMA_4K from merlin sources to create this) and we have to stop it initially loading the NVRAM from flash and trying to boot, we have sort of achieved this by shorting out the nand while loading the cfa via jtag and can get the serial prompt but it crashes shortly after , please note this has not been achieved on the ac5300 but another similar asus device with same cpu but from a area that had lesser restrictions on firmware. I am working on the nand side while others dig into the JTAG side currently.


darkspr1te

 

[Tech9]

The device in question is End-of-Life. Is this "diving" for fun? Otherwise there is no point saving an RT-AC5300 in late 2023.

 

[darkspr1te]

Yes, very much for fun and knowledge and a lot of the work and results found here are already applicable to new devices when it comes to the recovery side.

 

[Tech9]

I don't know about that. The basic principles - perhaps yes. Everything else is different on HND platform routers. I had a fun project some time ago converting some popular models in their corresponding "gaming" versions and the amount of time and efforts wasn't worth it. I would rather go buy the "original" version based on the same hardware. Asus made sure things are locked down pretty tight in newer routers.