Looking for feedback: Samba upgrade

[RMerlin]

On my MIPS RT-N66U I have 19MB/read and 12MB/write. As ColinTaylor said, if there is a performance hit we should think if it's worth the encreased security: we risk destroying the NAS-like experience. While I don't think I have severe vulnerabilities to fix in my home network (at least I hope so... ).

Quite frankly, there is nothing NAS-like in 19 MB/s to begin with.

 

[CaptnDanLKW]

I used to be a Linux & Samba nut years ago, compiling every new version of the kernel and Samba in Slackware. I would do extensive performance tests each time, tweaking settings in smb.conf. The two I would constantly tweak were the socket settings (TCP_NODELAY and the buffer sizes) and oplock settings.

Are these 'optimized' for the various SOC platforms? I could imagine that oplocks might provide better throughput but add more load to the already weak CPUs. Just wondering.

IMHO, a small decrease in throughput of something that's not really fast in the first place is worth the trade off for better security.

Dan

 

[RMerlin]

I used to be a Linux & Samba nut years ago, compiling every new version of the kernel and Samba in Slackware. I would do extensive performance tests each time, tweaking settings in smb.conf. The two I would constantly tweak were the socket settings (TCP_NODELAY and the buffer sizes) and oplock settings.

Are these 'optimized' for the various SOC platforms? I could imagine that oplocks might provide better throughput but add more load to the already weak CPUs. Just wondering.

IMHO, a small decrease in throughput of something that's not really fast in the first place is worth the trade off for better security.

Dan

Kernel oplock support is enabled. I would assume this would have a very small performance impact when you are testing with a single client copying a single large file (that probably means only one lock is being obtained at the start of the transfer).

While I tested with and without TCP_NODELAY, I left it disabled for ARM. Asus disables it for ARM, and the comment in the source code state that they did so due to compatibility issues with the switch (they probably mean the Broadcom internal switch used in the router).

I also tested compiling with aio support and enabling it, with no real performance difference. This is probably more useful in a scenario where the bottleneck is the storage subsystem rather than the CPU.

The one thing that did make a difference was compiling with -O3 instead of -Os. Since the size difference was minimal (and I expect to save even more on size once I switch to the multicall binary), I decided to keep it that way for the 3.6 build.

 

[RMerlin]

BTW: Does overclock improve SAMBA performances?

I haven't specifically tested it, but I would wager that it does, since the bottleneck is currently at the CPU level. Also, the 1 GHz RT-AC87U is clearly faster than the 800 MHz RT-AC56.

 

[ColinTaylor]

The one thing that did make a difference was compiling with -O3 instead of -Os. Since the size difference was minimal (and I expect to save even more on size once I switch to the multicall binary), I decided to keep it that way for the 3.6 build.

I know I asked this before (http://www.smallnetbuilder.com/forums/showpost.php?p=118222&postcount=16) but if you're already testing different builds, would you consider changing the MAX_DEBUG_LEVEL from 0 (i.e. none at all) to 2. Obviously it would increase the size of the binary, but it would be very useful to be able to debug connection issues. The current setting makes this practically impossible.

 

[nairn62]

Hello Merlin,
For me Work Stability and Security is First of All. Because of this i allways watch for updates!
Router not NAS, and work speed is second about i think.

My latest FW compilled point on this http://goo.gl/mJjVT1 commit

But look in target.mak file i see, for my MIPS N66U router this feature ( SAMBA36 ) is not present.
Could you tell,new SAMBA version is only for ARM devices or in future you include MIPS too ?

Thanks.

I've been using the following optimizations for compiling C code on the RT-AC68U and the RT-AC87U:

_CFLAGS := -O3 -fPIC -falign-functions -fdata-sections -ffunction-sections -fdata-sections -fomit-frame-pointer -fsigned-char -fsingle-precision-constant -fprefetch-loop-arrays -funroll-loops

I'm not sure if compiling Samba v3.6.24 with these additional options e.g. other than just using -O2 or -O3 will help its speed issues. These options were gleamed from various articles on ARM devices.

 

[RogueSpear]

My vote would be to go with the upgrade. It's been quite a while since I've had the time to follow the development of network protocols, but it would seem to me that more secure and more compatible will inevitably result in less performance if the hardware is equal. This is why dang near everything has exploded with new features, yet seems faster than ever - the hardware we use is always getting faster.

Even though, as someone pointed out, these are routers intended primarily for residential use, security still matters. Automated scans looking for particular vulnerabilities don't really care so much if an IP is home, commercial, non-profit, etc. if the intent is to leverage someone else's hardware/network for their own purposes.

And finally, I just can't imagine someone justifiably getting their shorts in a bunch over the drop in performance that we're looking at here. You can really only expect so much out of these routers to begin with and if you really need serious file serving performance, have a look at any of the top 10 NAS vendors.

 

[keenism]

I throw my vote in the upgrade hat..

 

[RMerlin]

I know I asked this before (http://www.smallnetbuilder.com/forums/showpost.php?p=118222&postcount=16) but if you're already testing different builds, would you consider changing the MAX_DEBUG_LEVEL from 0 (i.e. none at all) to 2. Obviously it would increase the size of the binary, but it would be very useful to be able to debug connection issues. The current setting makes this practically impossible.

I'm not setting any in the Samba 3.6.24 build, so no idea what's the default value.

 

[RMerlin]

I've been using the following optimizations for compiling C code on the RT-AC68U and the RT-AC87U:

_CFLAGS := -O3 -fPIC -falign-functions -fdata-sections -ffunction-sections -fdata-sections -fomit-frame-pointer -fsigned-char -fsingle-precision-constant -fprefetch-loop-arrays -funroll-loops

I'm not sure if compiling Samba v3.6.24 with these additional options e.g. other than just using -O2 or -O3 will help its speed issues. These options were gleamed from various articles on ARM devices.

I believe -O3 would include some of these, such as unroll-loops (while -Os won't do any size-increasing optimization, disabling that one).

 

[nairn62]

>>>While I tested with and without TCP_NODELAY, I left it disabled for ARM. Asus disables it for ARM, and the comment in the source code state that they did so due to compatibility issues with the switch (they probably mean the Broadcom internal switch used in the router).

I would be grateful if you could you point to the Asus source code (your Git branch) that defines TCP_NODELAY (the TCP_NODELAY with Asus's comment)

Thanks

 

[RMerlin]

I would be grateful if you could you point to the Asus source code (your Git branch) that defines TCP_NODELAY (the TCP_NODELAY with Asus's comment)

Thanks

There you go:

https://github.com/RMerl/asuswrt-merlin/blob/master/release/src/router/libdisk/write_smb_conf.c#L291

 

[Nullity]

As an RT-N66U user, I say upgrade.

Power users can optimize if needed, but the default should be security and compatibility.

 

[Shonk]

Id say go with 3.6

It may be worth looking into osstech samba 3.6

buffalo use that on theyr nas's its faster than stock samba 3.6

http://buffalo.jp/php/los.php?to=gpl/storage/ls-x/165/buffalo-samba-3.6.3-31.osstech.tar.bz2

thats the build they are using currently


also when they set read ahead to 1024k/2048k it made quite a difference
for smb read's

tune_blockdev.sh

#!/bin/bash

shopt -s extglob
. /etc/nas_feature

case ${TUNE:-0} in
    1)
        md_ra_value=
        sd_ra_value=1024

        for md in /sys/block/md*
        do
                if [[ $md == /sys/block/md@(0|1|10) ]]; then
                        continue
                fi

                raid_disks=$(cat ${md}/md/raid_disks)
                if [[ raid_disks -ge 3 ]]; then
                        sd_ra_value=2048
                fi
                md_ra_value=$((sd_ra_value * raid_disks))
                echo $md_ra_value > ${md}/queue/read_ahead_kb
        done

        for sd in /sys/block/sd?/queue/read_ahead_kb
        do
                echo $sd_ra_value > $sd
        done

        ;;
    *)

esac

 

[RMerlin]

Id say go with 3.6

It may be worth looking into osstech samba 3.6

buffalo use that on theyr nas's its faster than stock samba 3.6

http://buffalo.jp/php/los.php?to=gpl/storage/ls-x/165/buffalo-samba-3.6.3-31.osstech.tar.bz2

thats the build they are using currently


also when they set read ahead to 1024k/2048k it made quite a difference
for smb read's

tune_blockdev.sh

#!/bin/bash

shopt -s extglob
. /etc/nas_feature

case ${TUNE:-0} in
    1)
        md_ra_value=
        sd_ra_value=1024

        for md in /sys/block/md*
        do
                if [[ $md == /sys/block/md@(0|1|10) ]]; then
                        continue
                fi

                raid_disks=$(cat ${md}/md/raid_disks)
                if [[ raid_disks -ge 3 ]]; then
                        sd_ra_value=2048
                fi
                md_ra_value=$((sd_ra_value * raid_disks))
                echo $md_ra_value > ${md}/queue/read_ahead_kb
        done

        for sd in /sys/block/sd?/queue/read_ahead_kb
        do
                echo $sd_ra_value > $sd
        done

        ;;
    *)

esac

Changing the read ahead value has no impact on USB3 performance, most likely because the problem is currently being CPU bound rather than IO bound.

I wish they provided a patch instead of a pre-patched release. I will have to dig up the original tarball of the same release version to diff it.

 

[AndreaS]

I'd say go ahead with 3.6.
I'm starting to observe some sort of incompatibility with the newer versions of samba. If I access my usb drive from linux I often get a message stating that the selected file or directory does not exist, something that using windows i never get, nor i get with any other combination of linux and windows or linux and linux.