SquashFS performance booster?

[Fitz Mutch]

Here's the "SquashFS 4K" kernel patch from Linux 3.2, backported to Asuswrt-Merlin. It's supposed to be better for MTD NAND flash devices. Haven't tried any bench-marking yet to know for sure. Large programs seem to load faster.

~/380.66-ARM-squashfs-4K.patch
(see attached file)

How to apply the patch.
cd ~/asuswrt-merlin
patch -p2 -i ~/380.66-ARM-squashfs-4K.patch

 

[RMerlin]

Any benchmark that confirms that this does have any positive impact on performance?

 

[john9527]

Any benchmark that confirms that this does have any positive impact on performance?

It will be a while before I can dig into it, but a quick test on my fork it appears to slow down the boot by about 10-15%.

 

[Fitz Mutch]

It always takes my router exactly 14 seconds to reach init-start. No change in boot time, even with the squashfs-4K enabled.

/jffs/scripts/init-start

#!/bin/sh
/bin/date>/tmp/init-start.txt

/tmp/init-start.txt
Fri Jul 31 20:00:14 DST 2015


Problem with benchmarking this, is there's caching going on. Here's an example (with squashfs-4K) where it takes about 1 second to copy a 3.5MB file from the squashfs compressed firmware into the /tmp folder. Then subsequent attempts are instantaneous because something is caching.

/rom/tor/
-rwxr-xr-x 1 admin root 3539838 Jan 11 09:13 geoip
-rwxr-xr-x 1 admin root 1814484 Jan 11 09:13 geoip6

# date; cp /rom/tor/geoip /tmp; date
Wed Mar 1 15:07:00 EST 2017
Wed Mar 1 15:07:01 EST 2017

# date; cp /rom/tor/geoip /tmp; date
Wed Mar 1 15:07:04 EST 2017
Wed Mar 1 15:07:04 EST 2017

# date; cp /rom/tor/geoip /tmp; date
Wed Mar 1 15:07:06 EST 2017
Wed Mar 1 15:07:06 EST 2017

# date; cp /rom/tor/geoip /tmp; date
Wed Mar 1 15:08:20 EST 2017
Wed Mar 1 15:08:20 EST 2017


Similar thing happens when running a program that very first time. It must load all those dependent libraries. Then subsequent attempts happen immediately with no hiccup. I think my baseline measure (no squashfs-4K) for Cryptsetup loading the very first time wass about 3 seconds, but I could be wrong. With squashfs-4K, it took about 1 second to load Cryptsetup and without squash-4K it takes about 3 seconds. The following example is with squashfs-4K, but it's the 2nd run and something is caching it, therefore the timestamps are completely meaningless.

# date; ldd /entware/sbin/cryptsetup; date

Wed Mar  1 14:59:46 EST 2017
        libcryptsetup.so.4 => /entware/lib/libcryptsetup.so.4 (0x40197000)
        libpopt.so.0 => /entware/lib/libpopt.so.0 (0x40158000)
        libgcc_s.so.1 => /entware/lib/libgcc_s.so.1 (0x400ef000)
        libc.so.6 => /entware/lib/libc.so.6 (0x401c7000)
        libuuid.so.1 => /entware/lib/libuuid.so.1 (0x4006c000)
        libdevmapper.so.1.02 => /entware/lib/libdevmapper.so.1.02 (0x40304000)
        libssl.so.1.0.0 => /entware/lib/libssl.so.1.0.0 (0x40379000)
        libcrypto.so.1.0.0 => /entware/lib/libcrypto.so.1.0.0 (0x403d3000)
        /entware/lib/ld-linux.so.3 (0x40094000)
        libm.so.6 => /entware/lib/libm.so.6 (0x40515000)
        libdl.so.2 => /entware/lib/libdl.so.2 (0x40081000)
Wed Mar  1 14:59:46 EST 2017

 

[john9527]

It always takes my router exactly 14 seconds to reach init-start. No change in boot time, even with the squashfs-4K enabled.

For me....I measure the boot until it issues the first NTP sync. Without the patch it's pretty solid at 1:26
The squashfs patch added 6-10 seconds.

 

[sfx2000]

This one is a little risky, IMHO - depends on the NAND part and how it's organized internally...

Someone would have to test the heck out of it before committing it in... in all sorts of conditions and test/use cases. Worst case is that would could scribble NAND in a big way and corrupt the file system.

@john9527 - I would tend to agree - we don't do just a single read op during bootup - it's many small reads, and the smaller block sizes reduce latency - so your findings make sense to me...

The one use case I could see where this is of benefit would be for large files - e.g. media, but this isn't a realistic use case for internal NAND.

 

[Fitz Mutch]

I'm going to be keeping it turned on because I'm seeing a gain of 5 seconds while booting. Also, my firmware is very lean and probably much different than most. The unwanted apps and features have been removed.

With squashfs-4K:
at 33 seconds ntpd sets the router clock via GPS+PPS

With squashfs-1K:
at 38 seconds ntpd sets the router clock via GPS+PPS

EDIT: Forgot to mention some variables. My router is RT-AC68U overclocked to 1200,666.