[384.18_Alpha Builds] Testing all variants

[motox22a]

I agree with EVERYTHING that kernol said...
No need to kill yourself...This stuff is on routers that "should" be used primarily for Home users...
Definitely not needed to have constant updates....Having come from DD-WRT where daily updates are
often the norm (And they usually don't do anything to fix problems and often introduce problems)
I find it refreshing to have a router OS that actually does what it says it will...
You will never make everyone happy....Somebody on they're SukiYaki 50 dollar Smart Phone that's not
happy because his phone is having connection problems (It must be Merlin software..right?)...
Take a break...Enjoy life (As much as you can with Covid and the other problem).
When you feel like it work on Merlin...When you don't feel like it...Don't.
I dare say we will survive on your already EXCELLENT firmware.

In my book - you gave the answer to your production workflow problem - highlighted above!
You have FAR exceeded our expectations of you. Since I joined the Merlin journey [some 30 months ago] you have produced no fewer than 27 firmware releases. There were 10 in 2018 / 10 in 2019 and 7 so far this year in 2020 [in 5 months!!!].
In case anyone is puzzled by the stats - check out the linked PDF listing all the releases I am referring to.
https://onedrive.live.com/?cid=56ABA0E3B79458E5&id=56ABA0E3B79458E5!38939&parId=56ABA0E3B79458E5!131&o=OneUp

So - my best tips for you are: -

1. Set your sights on no more than 4 firmware releases per annum;
2. Don't push Alpha's into a thread on this forum - rather build a list of volunteer Alpha testers across the model range to assist in early development - and provide test code to them only;
3. Start forum thread for BETA testing once ready to release a beta test across the covered model range;
4. Use some of the donated cash to invest in Chill Pills - or Fine Wine ...

There really is little need to constantly feed us "update junkies" . Rather encourage us to enjoy long periods between reboots on your incredibly stable and add-on feature-enabled firmware!

 

[RMerlin]

Set your sights on no more than 4 firmware releases per annum;

Not really a solution, as this does not address the fact that at one given point in time (like right now actually), I may not be able to have code that works on all models at the same time. It would mean that some models might end up getting only one or two updates per year. Not realistic for a router that occasionally requires security updates pushed sooner than later.

Don't push Alpha's into a thread on this forum - rather build a list of volunteer Alpha testers across the model range to assist in early development - and provide test code to them only;

Notice that I do not manage these alpha threads, and I usually only have very limited public interaction with them. They are just me occasionally copying test builds I had made for my own needs to a Onedrive folder, and everything else after that no longer requires my involvement. These are the alternative I chose to having a buildbot issue nightly builds because I don't want people to do like on DD-WRT, with everyone reflashing their router day after day with totally untested builds, occasionally flashing totally broken images that might not even boot, or seeing issues that do not exist by flashing through five different builds, trying to accuse one of these builds of "ruining"t heir wifi even if there might have been zero change to the wifi code. What I upload generally has at least received some basic testing for one of the models (I occasionally compile a few others at the same time just to ensure that the build completes successfully).

So these alpha builds are not a problem at all, they are just a by-the-way, and require very little time investment on my part. That's why they happen when they happen, with no schedule, and not always covering all models.

Start forum thread for BETA testing once ready to release a beta test across the covered model range;

Already the case, these beta cycles are generally planned and managed on my end. I simplified it by no longer publishing them to the update server, so it allows me to have the process more automated (aside from writing the release notes, and announcing on SNB/Twitter)

spacing it out, may make merges harder?

This is another reason indeed. I hate it when Asus takes 4-5 months between GPL releases, as it makes the merge very difficult. And in major gaps like the 6-8 months of piled up changes when Asus released the first 382 release they had been developing in parallel to 380, I ended up restarting my project from zero rather than try to merge it on top of the older 380 code, which would have been near impossible at the time. I took the occasion to streamline my own changes a bit, so at least a few specific portions of following GPL merges were easier (I get much fewer rejects in the httpd code for instance).

 

[raven-au]

GPLs are publicly posted on their support site under the Others OS category. They also upload them on my personal FTP server, so sometimes I get different releases from the website.

Diffing the release won't help you in your case, AiMesh is closed source, so all there is are binary objects.

As I say I got the GPL tars, my bad for even asking.
Ha, and the replies to that dumb question were great, it's a good group of people here, to be sure.

But, what I was actually getting at is, what's your update process, a very vague outline is all I was hoping for.
IOW I'm wondering how I could duplicate the build failures (I think you mentioned) of recent releases to get a look at that.

I get that question might be hard to answer so don't worry about it if that's the case.
I'll just continue to study the git history and try to work it out.

 

[raven-au]

This is another reason indeed. I hate it when Asus takes 4-5 months between GPL releases, as it makes the merge very difficult. And in major gaps like the 6-8 months of piled up changes when Asus released the first 382 release they had been developing in parallel to 380, I ended up restarting my project from zero rather than try to merge it on top of the older 380 code, which would have been near impossible at the time. I took the occasion to streamline my own changes a bit, so at least a few specific portions of following GPL merges were easier (I get much fewer rejects in the httpd code for instance).

It occurs to me that I've seen the situation you have before.

The Plex Media Player for Linux is layered on top of LibreELEC (that might have been renamed by now) and to update the LibreELEC version was a big problem. I gave up playing with it because there was no way to work out what changes Plex had made to turn it into Plex Media Player.

What I thought would have worked well in that case was keeping all the changes separate in some way and applying them on top of new releases, and deal with the inconsistencies when moving to a new release. Pretty hard to do initially but might work ok once processes are established over time.

That might be similar to what you do now but AFAICS there's no way for any one not intimately familiar with those changes to separate the enhancements from the original firmware.

This is a little bit like how the linux-next repo works.

You grab the mainline kernel repo (say remote origin), you add a remote for the linux-next repo (say linux-next) on top of the mainline tree and merge remote linux-next on top of remote origin giving you an up to date linux-next code base to work from (normally you'd also use branches but that's a detail).

So, it might not help you initially and it might not be that simple but, divide and conquer, ways to divide the tasks into parts for others to do might well present themselves as you go.

 

[raven-au]

This is a little bit like how the linux-next repo works.

Another slightly different analogy along these lines is the way package sources are managed in Fedora.

The way that works has evolved partly due to the way the rpm package manager works.

In its simplest terms, you have an upstream release source tar and a bunch of patches.
Those patches are applied to the release and the binary package built.

Once a release occurs it's tagged and the various rpm build definitions and patches become the point in the git history for that release. The source tars are held external to the repo and retrieved as needed.

The point here is the separation of an upstream component (the tar) and a downstream component, the patches allied to the base release. Actual OS releases live in separate branches and when it's time for new OS development the branching occurs and maintenance of previous releases continues on the earlier branch.

I would not say this is easy, there can be significant challenges when a new upstream release comes out so maintainers are encouraged to have there changes accepted upstream as much as possible.

Clearly there's an automated build system surrounding this but there's also a bit of build code around already for Merlin.

Anyway, more food for thought ... maybe an idea or two will materialize along the way ...

 

[RMerlin]

But, what I was actually getting at is, what's your update process, a very vague outline is all I was hoping for.

Basically this, done folder by folder, one at a time, and after a first review of the whole list of changes between both.

diff -Naurp --no-dereference ../asuswrt.384-81116-ac88/release/src/router/one_folder ../asuswrt.385-20490.ac88/release/src/router/one_folder | patch -p2

You need to know which folder to merge this way, and which not to. Manually resolving every patch rejects, taking note at which changes will require further updating, etc... Manually merging some of the files because they are known to frequently get messed up by diff/patch, like rc/firewall.c. Some files require adjusting the fuzz factor. Some files have to be skipped entirely. Manually merging the SDK itself (which requires understanding how the SDK is integrated, and varies between platforms).

And running a script that I wrote which copies all the binary blobs to the appropriate location.

Overall it's a very specialized process, and not something that can be documented as a series of instructions to follow. It requires hands-on experience with the whole project to adjust on the fly.

IOW I'm wondering how I could duplicate the build failures (I think you mentioned) of recent releases to get a look at that.

There's nothing to look at. Building fails for httpd and rc due to function calls that did not exist yet in these older web_hook.o and private.o binary blobs. It's not something that someone can fix, it requires newer binary blobs from Asus with these missing functions.

What I thought would have worked well in that case was keeping all the changes separate in some way and applying them on top of new releases, and deal with the inconsistencies when moving to a new release. Pretty hard to do initially but might work ok once processes are established over time.

This is basically what OpenWRT does, and I find the whole process absolutely horrible as a development workflow. It might work for applying a few minor static patches, but not when you are making full-blown development, which requires you to have a complete view of the final code. Asuswrt-Merlin did start as a patch early on, and the process was dropped once the development became more complex, and I started using an RCS.

You end up wasting a lot of time when trying to debug anything, because every time you have to first unpack the code, apply the patch, and then you can start working. This is a pretty screwed up development process, and only works properly if you are just applying a few specific patches to the original code, and not doing actual active development modifying large sections of it. If Asus makes a structural change, I need to be able to view the flattened code, not just an unpatched copy of it.

And none of this has anything to do with GPL drops being out of sync and/or delayed.

 

[raven-au]

Thanks for taking some time to describe the process and for your thoughts.

Basically this, done folder by folder, one at a time, and after a first review of the whole list of changes between both.

diff -Naurp --no-dereference ../asuswrt.384-81116-ac88/release/src/router/one_folder ../asuswrt.385-20490.ac88/release/src/router/one_folder | patch -p2

You need to know which folder to merge this way, and which not to. Manually resolving every patch rejects, taking note at which changes will require further updating, etc... Manually merging some of the files because they are known to frequently get messed up by diff/patch, like rc/firewall.c. Some files require adjusting the fuzz factor. Some files have to be skipped entirely. Manually merging the SDK itself (which requires understanding how the SDK is integrated, and varies between platforms).

Right, that sounds a lot like what I thought it would be.

That's hard work to be sure, and does require a lot of experience with the code base your working with.
As you've said a few times, a very steep uphill learning curve.

But thanks, at least now I know the process that's being used.

And running a script that I wrote which copies all the binary blobs to the appropriate location.

And another specialized step that doesn't lend itself at all well to git merging capabilities ...

Overall it's a very specialized process, and not something that can be documented as a series of instructions to follow. It requires hands-on experience with the whole project to adjust on the fly.

I knew that already (guessed that would be the case),

There's nothing to look at. Building fails for httpd and rc due to function calls that did not exist yet in these older web_hook.o and private.o binary blobs. It's not something that someone can fix, it requires newer binary blobs from Asus with these missing functions.

Right, got it.

This is basically what OpenWRT does, and I find the whole process absolutely horrible as a development workflow. It might work for applying a few minor static patches, but not when you are making full-blown development, which requires you to have a complete view of the final code. Asuswrt-Merlin did start as a patch early on, and the process was dropped once the development became more complex, and I started using an RCS.

Indeed, I didn't find the OpenWRT environment terribly convenient either.

That type of environment isn't something I think would work well for Merlin development either even though the second example, Fedora development, sounds like that and, to a small extent, is like that.

You end up wasting a lot of time when trying to debug anything, because every time you have to first unpack the code, apply the patch, and then you can start working. This is a pretty screwed up development process, and only works properly if you are just applying a few specific patches to the original code, and not doing actual active development modifying large sections of it. If Asus makes a structural change, I need to be able to view the flattened code, not just an unpatched copy of it.

Yes, the development environment needs to be flat.
That is what you get with the first example I mentioned, the linux-next development environment.
But that doesn't apply cleanly here either, still it might be worth spending a little thought on it.

And none of this has anything to do with GPL drops being out of sync and/or delayed.

Sadly that's out of our control unless there are sympathetic ears to talk to at ASUS.

Ian

 

[SheikhSheikha]

Running my two AC5300's for several days on 384.18 Alpha and they are rock solid! Reading all of the above, my deep respect for how Eric transforms Asus spaghetti firmware in to a 6-course Asus-Merlin firmware meal!

 

[raven-au]

Yes, the development environment needs to be flat.
That is what you get with the first example I mentioned, the linux-next development environment.
But that doesn't apply cleanly here either, still it might be worth spending a little thought on it.

Looking at the repo I'm starting to get an idea of how it hangs together now.
I see that binary blob copy script, those blobs are all over the place, makes any sort of separation of bits of the repo hard.

And it's hard to see how merging overhead could be reduced by any re-organization since merging development kit changes is always going to be a big overhead and, well, getting the GPL tars is not something that can be fixed ...

 

[Simon W]

Yes, because I merged the binary blobs from 385_20490.
Simply that, right now, I can't compile the firmware for both the RT-AX88U and the two other models. Once I merge the newer RT-AX88U code, if I don't have updated code for the other two models, then that release will be missing these two models.

Thanks Merlin. I look forward to this merge/release, as and when, which I guess will make most sense (ie. most bang for buck) when Asus get round to releasing updated binaries and code for the RT-AX56U and RT-AX58U respectively (or at least one of them).. or you get bored of waiting which of course suits RT-AX88U owners like me just fine but you'll never please everyone!

 

[Phil Outram]

I just flashed the second Alpha to my AX88U and the options introduced in the first alpha for 'Supports 2.4G/5GHz uplink OFDMA and 802.11ax MUMIMO' are no longer present in the professional Wi-Fi settings for either band. The only option present now is download OFDMA. The release notes from the gpl of the first alpha were as follows:

Firmware version 3.0.0.4.384.8563
- Fix bandwidth limiter abnormal.
- Fix setup wizard GUI bugs.
- Supports 2.4G/5GHz uplink OFDMA and 802.11ax MUMIMO.
- Security Fix: CVE-2019-15126

 

[Simon W]

I just flashed the second Alpha to my AX88U and the options introduced in the first alpha for 'Supports 2.4G/5GHz uplink OFDMA and 802.11ax MUMIMO' are no longer present in the professional Wi-Fi settings for either band. The only option present now is download OFDMA. The release notes from the gpl of the first alpha were as follows:

Firmware version 3.0.0.4.384.8563
- Fix bandwidth limiter abnormal.
- Fix setup wizard GUI bugs.
- Supports 2.4G/5GHz uplink OFDMA and 802.11ax MUMIMO.
- Security Fix: CVE-2019-15126

The binaries/drivers required for AX hardware to support this aren't included within this build yet (despite now being available from Asus and hence their release note, per mentioned in my post above these will presumably be integrated once compatibility is achieved across the other two AX models) - looking at Merlin's commits he has specifically hidden this option in this alpha (I guess to avoid confusion / unintended consequences as at best it wouldn't have done anything).

 

[Phil Outram]

The binaries/drivers required for AX hardware to support this aren't included within this build yet (despite now being available from Asus and hence their release note, per mentioned in my post above these will presumably be integrated once compatibility is achieved across the other two AX models) - looking at Merlin's commits he has specifically hidden this option in this alpha (I guess to avoid confusion / unintended consequences as at best it wouldn't have done anything).

OK, no worries and thanks for the explanation.

 

[rotareneg]

With 384.18_alpha1-g5f6b535e1a on my RT-AC66U B1, the customized QOS list resets to the default order if you leave the main QOS tab and come back and check the customized list again.

 

[raven-au]

Basically this, done folder by folder, one at a time, and after a first review of the whole list of changes between both.

diff -Naurp --no-dereference ../asuswrt.384-81116-ac88/release/src/router/one_folder ../asuswrt.385-20490.ac88/release/src/router/one_folder | patch -p2

One other thing I was wondering about.
Do you also apply fixes to the development kit directory tree as you go?
For example directory trees like src-rt-5.02hnd or src-rt-5.02axhnd etc.

 

[RMerlin]

One other thing I was wondering about.
Do you also apply fixes to the development kit directory tree as you go?
For example directory trees like src-rt-5.02hnd or src-rt-5.02axhnd etc.

SDKs are part of GPL releases, so they are merged at the same time.

Some SDKs require me to compile a GPL tarball in parallel, and copy a few generated binary blobs to my tree. In the 675x case, I have to copy binary blobs to different locations due to having to handle two separate models within the same SDK.

Once again... It's complicated.

 

[raven-au]

SDKs are part of GPL releases, so they are merged at the same time.

Some SDKs require me to compile a GPL tarball in parallel, and copy a few generated binary blobs to my tree. In the 675x case, I have to copy binary blobs to different locations due to having to handle two separate models within the same SDK.

Once again... It's complicated.

LOL, so the answer is "yes" and "no" ...

But it's not the same as the router sub-directory tree where the bulk of improvements live.
IOW very few, if any, "fixes" or "enhancements" are applied there, is that right?

 

[swejuggalo]

An update regarding Smart Connect and 5Ghz Auto/Manual channel issue. Got a new device. Wifi 6 works. Connects at 1200 with manual channel selection. 286 at auto (same channel).
If there is anything I need to do or try to do to help out let me know.
If this is a Asus or Merlin issue I don't really know. .15 was the last version that worked in auto. My assumption says it's been like this since latest merge on my model.
/AX88

 

[RAH-66]

@swejuggalo use only manual channel and 160Mhz not auto 20/40/80/160

 

[swejuggalo]

@swejuggalo use only manual channel and 160Mhz not auto 20/40/80/160

Well... I already did use manual as a walkaround. But I haven't needed it before. In opinion I should not be forced to since I was not forced to before. Hence the reason why I post here [emoji39]

Sent from my IN2023 using Tapatalk