Tutorial HOWTO fix stock busybox/sed insert command on your R7800/R9000/etc

[arabesc]

During NG firmware script patching research I've realized that stock busybox/sed tool has a bug - the insert command doesn't work properly.
Here is a test:
# printf "1\n3" | /bin/sed '/3/i 2' | hexdump -C
The buggy output is:
00000000 31 0a 32 01 33 |1.2.3|
And the expected output is:
00000000 31 0a 32 0a 33 |1.2.3|

The issue seems to be not critical for the stock firmware scripts because they don't use the sed insert command.
The bug has been fixed in the commit 826c85f3828c30b772d307c6b60ff3be744cecc2 to the busybox repo but NG firmware uses an older version without the fix.
There is no need to fix the bug if you are not going to use the sed insert command!

I've made a small binary patch for the /bin/busybox that as I hope fixes the issue:
0120a0e3081099e5 -> 0a20a0e3081099e5
It works for a current and a few previous Voxel's firmware versions.

The patch can be applied with the following command sequence:
# hexdump -ve '1/1 "%.2x"' /bin/busybox | sed 's/0120a0e3081099e5/0a20a0e3081099e5/' | sed 's/\([0-9a-f]\{1,60\}\)/\1\n/g' | xxd -r -ps > busybox
I haven't found a way to apply a binary patch in the stock firmware environment without additional dependencies, so the method requires the xxd tool from the Entware.

I've written a more advanced script to patch busybox, it makes some additional checks before and after patching. The script may require some minor adaptation to your environment and then the call to the script may be added to the /mnt/optware/autorun/scripts/post-mount.sh script to automatically patch /bin/busybox after every firmware update.

#!/bin/sh

BASE_PATH="/tmp/mnt/optware/config"
INIT_LOG="$BASE_PATH/init.log"

XXD_SYS="/opt/bin/xxd"
BUSYBOX_SYS="/bin/busybox"
BUSYBOX_BAK="$BASE_PATH/scripts/backup/busybox.bak"
BUSYBOX_NEW="$BASE_PATH/scripts/backup/busybox.new"
SED_SYS="/bin/sed"
SED_TEST="$BASE_PATH/scripts/backup/sed"

/bin/echo "patch-busybox.sh" >> "$INIT_LOG"

if [ ! -x "$XXD_SYS" ]; then
    /bin/echo "There is no $XXD_SYS" >> "$INIT_LOG"
    exit 1
fi

if [ ! -x "$BUSYBOX_SYS" ]; then
    /bin/echo "There is no $BUSYBOX_SYS" >> "$INIT_LOG";
    exit 1
fi

if [ ! -h "$SED_SYS" ]; then
    /bin/echo "There is no $SED_SYS" >> "$INIT_LOG"
    exit 1
fi

result=`/usr/bin/printf "1\n3" | "$SED_SYS" '/3/i 2' | "$XXD_SYS" -p -seek 3 -l 1`
if [ "$result" == "0a" ]; then
    /bin/echo "Busybox is already patched" >> "$INIT_LOG"
    exit 0
fi

if ! /bin/cp -fp "$BUSYBOX_SYS" "$BUSYBOX_BAK"; then
    /bin/echo "Failed to copy $BUSYBOX_SYS to $BUSYBOX_BAK" >> "$INIT_LOG"
    exit 1
fi

if ! /usr/bin/diff "$BUSYBOX_SYS" "$BUSYBOX_BAK" > /dev/null 2>&1; then
    /bin/echo "$BUSYBOX_BAK is corrupted" >> "$INIT_LOG"
    /bin/rm -f "$BUSYBOX_BAK"
    exit 1
fi

/usr/bin/hexdump -ve '1/1 "%.2x"' $BUSYBOX_BAK | $SED_SYS 's/0120a0e3081099e5/0a20a0e3081099e5/' | $SED_SYS 's/\([0-9a-f]\{1,60\}\)/\1\n/g' | $XXD_SYS -r -ps > $BUSYBOX_NEW

if [ ! $? ] || [ ! -f "$BUSYBOX_NEW" ]; then
    /bin/echo "Failed to patch $BUSYBOX_BAK" >> "$INIT_LOG"
    /bin/rm -f "$BUSYBOX_NEW" "$BUSYBOX_BAK"
    exit 1
fi

if ! /bin/chmod 755 "$BUSYBOX_NEW"; then
    /bin/echo "Failed to set access mode to $BUSYBOX_BAK" >> "$INIT_LOG"
    /bin/rm -f "$BUSYBOX_NEW" "$BUSYBOX_BAK"
    exit 1
fi

if ! /bin/ln -sf "$BUSYBOX_NEW" "$SED_TEST"; then
    /bin/echo "Failed to link $BUSYBOX_BAK to $SED_TEST" >> "$INIT_LOG"
    /bin/rm -f "$SED_TEST" "$BUSYBOX_NEW" "$BUSYBOX_BAK"
    exit 1
fi

result=`/usr/bin/printf "1\n3" | "$SED_TEST" '/3/i 2' | "$XXD_SYS" -p -seek 3 -l 1`
if [ "$result" != "0a" ]; then
    /bin/echo "Busybox patch doesn't work" >> "$INIT_LOG"
    /bin/rm -f "$SED_TEST" "$BUSYBOX_NEW" "$BUSYBOX_BAK"
    exit 1
fi

if ! /bin/cp -fp "$BUSYBOX_NEW" "$BUSYBOX_SYS"; then
    /bin/echo "Failed to copy $BUSYBOX_NEW to $BUSYBOX_SYS" >> "$INIT_LOG"
    /bin/rm -f "$SED_TEST" "$BUSYBOX_NEW" "$BUSYBOX_BAK"
    exit 1
fi

/bin/rm -f "$SED_TEST" "$BUSYBOX_NEW"

/bin/echo "Busybox has been successfully patched" >> "$INIT_LOG"

exit 0

 

[kamoj]

Excellent job and tutorial!
I couldn't resist to do the patch with existing tools:

cd /tmp
dd if=/bin/busybox bs=1 skip=0 count=211696 of=busybox.1
echo -n "x" | tr "x" "\x0A" >busybox.2
dd if=/bin/busybox bs=1 skip=211697 count=381940 of=busybox.3
cat busybox.1 busybox.2 busybox.3 >busybox.new
chmod +x busybox.new
\mv -f busybox.new /bin/busybox
cd -

Or a more educational and generic way to work for other busybox versions than the latest:

# Setup
tmpdir="/tmp"
file="/bin/busybox"
filename="$(basename "$file")"
filesize="$(ls -al "$file"|awk '{print $5}')"
offset_to_patch="$(hexdump -ve '1/1 "%.2x"' "$file" | awk -F "0120a0e3081099e5" '{print $1}' | wc -c | awk '{print -1+$1/2}')"

# Check if sed is already working or not:
if printf "1\n3" | "$file" sed '/3/i 2' | hexdump -C | grep "31 0a 32 01 33"; then
   echo "sed command fails, trying to patch $file"
elif printf "1\n3" | "$file" sed '/3/i 2' | hexdump -C | grep "31 0a 32 0a 33"; then
   echo "sed command already working. Restoring original $file, and trying to patch!"
   \cp -fp /rom$file $file
fi

# Do the patching to a copy of busybox in /tmp:
dd if="$file" bs=1 skip=0 count="$offset_to_patch" of="$tmpdir"/"$filename".1
echo -n "x" | tr "x" "\x0A" >"$tmpdir"/"$filename".2
dd if="$file" bs=1 skip="$((offset_to_patch+1))" count="$((filesize-$offset_to_patch-1))" of="$tmpdir"/"$filename".3
cat "$tmpdir"/"$filename".1 "$tmpdir"/"$filename".2 "$tmpdir"/"$filename".3 >"$tmpdir"/"$filename".new

# Make the new busybox executable:
chmod +x "$tmpdir"/"$filename".new

# Test if the new sed is working:
if printf "1\n3" | "$tmpdir"/"$filename".new sed '/3/i 2' | hexdump -C | grep "31 0a 32 0a 33"; then
   echo "Patch succeeded."
   \mv -f "$tmpdir"/"$filename".new "$file"
   echo "To restore original: \cp -p /rom$file $file"
else
   echo "Patch failed. Nothing is changed"
fi

# Clean up:
\rm -f "$tmpdir"/"$filename".*

 

[L&LD]

Great work here! What a way to start in 2021!

Is this applicable to Asus routers too? @RMerlin, @ColinTaylor, @dave14305, @john9527, @Martineau, and any and all others I'm missing mentioning here?

 

[john9527]

Is this applicable to Asus routers too?

No....we have a later version

Also just verified... @arabesc thanks for providing a simple testcase.

# printf "1\n3" | /bin/sed '/3/i 2' | hexdump -C
00000000  31 0a 32 0a 33                                    |1.2.3|

Netgear must be using a prehistoric version of busybox....the commit he referenced went in in 2007!

 

[Voxel]

As I already wrote to @arabesc I am a bit conservative with some from packages used by NG/DNI. The reason is: yeah, there are bugs in some packages. But they are known. If I fix/renew e.g. stone age busybox I have also check a lot of e.g. QCA scripts taken from the stock version... After month or so somebody will write that there are issues in new version, and there will be a headache for me to trace this...

Really, there is very fresh version of busybox from Entware. Everybody is free to use it. Or e.g. "classic" sed https://www.voxel-firmware.com/Downloads/Voxel/Entware/Entware-3x-Voxel/sed_4.8-3_cortex-a15-3x.ipk.

Anyway, thanks to @arabesc and all the rest who wants to improve my build

Voxel.

 

[Voxel]

I am a bit conservative with some from packages

Just to make it a bit more clear. Only some packages. I use for R7800/R9000 the most fresh GCC compiler (10.2.0, for example OpenWRT is using 8.4.0). The most recent versions of e.g. iptables/ipset, etc. So a lot of really newest important packages. So we keep the tracking. Safety is OK.

Voxel.

 

[arabesc]

The busybox binary has been changed in a new firmware, so the patch doesn't work anymore as it was.
The new patch pattern for the R9000-V1.0.4.51HF firmware is: 0120a0e308109ae5 -> 0a20a0e308109ae5.
And a bit relaxed pattern for sed is: 's/0120a0e308109\(.\)e5/0a20a0e308109\1e5/'. I hope the pattern will be more flexible, otherwise it can be relaxed even further.