Custom firmware build for R9000 v. 1.0.2.42HF/1.0.2.42HF-HW

[Voxel]

I had to use dnscrypt to get it working properly, and would have kept it on if I had more DNS provider options in Australia.

BTW, are you sure that you should use only Australian dnscrypt servers? E.g. I use servers not from my country.

Just FYI: https://github.com/bitbeans/dnscrypt-measurement

it is some kind of tool to find dnscrypt server(s) which is more fast for you. You can try to run from your Windows PC (if you have PC with Windows).

Voxel.

 

[RMerlin]

Interesting. Kernel version is upgraded too? If I am not mistaken, new Netgear's router with Broadcom has now 4.x version of kernel...

Yes, the new HND 5.x platform uses kernel 4.1. Which goes EOL in... September 2017. Sigh.

Broadcom needs to do like distro makers and assume maintenance of their chosen kernel. Red Hat kept providing updates to their 2.6 kernel throughout the whole life of CentOS 5 for instance, which lasted quite a few years.

 

[Voxel]

Yes, the new HND 5.x platform uses kernel 4.1. Which goes EOL in... September 2017. Sigh.

A lot of time left ;-)

Voxel.

 

[sfx2000]

Well, maybe this requirement from MS will force Netgear to use something more attractive for libc, like musl...

At some point, most will make a jump - not that uclibc is bad, it's really good for small footprint devices, but it's more that musl and glibc have more to offer once memory goes over a certain limit.

What I like about musl is the licensing - it's MIT permissive vs. GPL/lgpl, which makes it more attractive and less entangling for code that links to the libc...

Moving from uclibc/glibc might be a bit of work to ensure that apps still run, but it's worth the effort, IMHO...

 

[sfx2000]

Yes, the new HND 5.x platform uses kernel 4.1. Which goes EOL in... September 2017. Sigh.

Broadcom needs to do like distro makers and assume maintenance of their chosen kernel. Red Hat kept providing updates to their 2.6 kernel throughout the whole life of CentOS 5 for instance, which lasted quite a few years.

I'm sure there will be backports available for the 4.1 kernel for some time... and making the jump forward from 4.1 is not as painful as it seems... it's not like trying to keep 2.6 alive well past it's expiration date..

 

[RMerlin]

Moving from uclibc/glibc might be a bit of work to ensure that apps still run, but it's worth the effort, IMHO...

DD-WRT did the switch to musl a year or two ago. The svn commit log did show quite a bit of changes were necessary to acomodate it vs uclibc.

With routers getting more RAM and flash space these days, I wonder if it's worth the trouble versus a well supported glibc. Broadcom seems to think that glibc is the way to go with their latest platform at least. I believe Asustor also switched to glibc last year for their NAS firmware.

 

[RMerlin]

I'm sure there will be backports available for the 4.1 kernel for some time... and making the jump forward from 4.1 is not as painful as it seems...

Provided a kernel engineer devotes the time to do it. If Red Hat could backport patches to 2.6.xx for RHEL5/RHEL6, then I'm sure someone at Broadcom could do the same for 4.1. Broadcom's never seemed really interested in maintaining their code base unfortunately, leaving it in the hands of the ODM/OEMs instead.

Someone like Leonid is still somehow backporting tons of fixes to 2.6.22.19. And we had someone else backport fq_codel to 2.6.36 (altho in a slightly limited way, due to lack of BQL support in older drivers).

 

[Alazin]

Quick question is there a way to schedule daily restarts like dd wrt?

 

[Voxel]

Quick question is there a way to schedule daily restarts like dd wrt?

Possible if you will use Entware. It has cron where you can schedule your own script running. E.g. with restart command.

Voxel.

 

[andy22]

Cross-compiling Samba is a nightmare, and the Samba devs are providing zero assistance there ("if you get a working WAF build profile, feel free to send it to us" was the conclusion they posted on their mailing list a few years ago). Plenty of folks tried to get 4.x to cross-compile on non-x86, with no real luck with it. Personally I simply gave up both times I tried to.

You basically have to provide your own answers to everything that is normally handled by autoconf, which requires you to be a complete expert in *IX environment.

Samba probably has the worst build system I have ever seen out of an open-source project. So I'll be definitely keeping an eye for a final release from Netgear, and hope they provide the build files for it.

Just a quick info, i have created a working samba 4.8.2 feed for openwrt, that should work for all supported devices (assuming the image size fits). The current version (May 2018) still needs to-be compiled against the master branch of openwrt/packages, since i still wait for the merge request to get through and a related pull request (krb5) just missed the 18.x release.

The compressed packages are around 4.5-5.4 MB for just the fileserver version.

Feedback are welcome via git or in the lede forum, there is a thread about this feed.

PS: I use samba 4.x for over 2 years now on my WRT 1200AC, without major problems.
I also use qemu instead of cross-answer files, to avoid the exact concerns you raised about cross-compiling samba via WAF.

 

[Voxel]

Just a quick info, i have created a working samba 4.8.2 feed for openwrt, that should work for all supported devices (assuming the image size fits). The current version (May 2018) still needs to-be compiled against the master branch of openwrt/packages, since i still wait for the merge request to get through and a related pull request (krb5) just missed the 18.x release.

The compressed packages are around 4.5-5.4 MB for just the fileserver version.

Feedback are welcome via git or in the lede forum, there is a thread about this feed.

PS: I use samba 4.x for over 2 years now on my WRT 1200AC, without major problems.
I also use qemu instead of cross-answer files, to avoid the exact concerns you raised about cross-compiling samba via WAF.

It is interesting.

What is bad in NG version of samba4: almost frozen modified version of Samba 4.6.4 with missed security updates. What is good in samba4 in the stock version for R7800: practically single samba_multicall w/o a lot of additional files /usr/lib

BTW, as I know Samba 4.8.x is available e.g. in Entware e.g.

http://bin.entware.net/armv7sf-k3.2/Packages.html

What is specific in your version? Minimized size? Is in something like single samba_multicall?

Voxel.

 

[andy22]

BTW, as I know Samba 4.8.x is available e.g. in Entware e.g.

Oh, i did not know someone else got a openwrt/entware package ready for samba 4.6+, the last time i researched this i could not find any package/makefile, hence i did this myself
I also still wait to get my pull request approved, so hopefully some day this is a default openwrt package.

http://bin.entware.net/armv7sf-k3.2/Packages.html

Where is the code for this package, i cant find it in the github entware repo? Would really like to compare the makefiles.

What is specific in your version? Minimized size? Is in something like single samba_multicall?

Without the source hard to tell, but from the size of the entware server/lib packages, it looks like a default full compile, without any special handling of WAF's "unique" features. So yes my version is a little smaller and also has experimental support for AD-DC/Winbind/AVAHI. I also suspect it uses the embedded heimdal libs, while my versions uses krb5 libs.

I also have a custom luci ui for it, so you can easily add VFS modules.

PS: Regarding a multi-call binary, that's simply not viable for samba4 and WAF, i tried this a year ago and had no real size improvements.

 

[Voxel]

Would really like to compare the makefiles.

#
# Copyright (C) 2017
#
# This is free software, licensed under the GNU General Public License v2.
# See /LICENSE for more information.
#
# more binaries are added to server and client packages
include $(TOPDIR)/rules.mk
PKG_NAME:=samba4x
PKG_VERSION:=4.7.3
PKG_RELEASE:=1
PKG_SOURCE:=samba-$(PKG_VERSION).tar.gz
PKG_SOURCE_URL:=https://download.samba.org/pub/samba/stable
PKG_HASH:=06e4152ca1cb803f005e92eb6baedb6cc874998b44ee37c2a7819e77a55bfd2c
PKG_BUILD_DIR:=$(BUILD_DIR)/samba-$(PKG_VERSION)
PKG_LICENSE:=GPL-3.0
PKG_LICENSE_FILES:=COPYING
PKG_BUILD_PARALLEL:=1
PKG_INSTALL:=1
PKG_BUILD_DEPENDS:=heimdal
include $(INCLUDE_DIR)/package.mk
# for $(LINUX_VERSION)
include $(INCLUDE_DIR)/kernel.mk
# for $(VERSION_DIST)
include $(INCLUDE_DIR)/version.mk
define Package/samba4x/Default
  SECTION:=net
  CATEGORY:=Network
  TITLE:=Samba $(PKG_VERSION) SMB/CIFS
  URL:=http://www.samba.org/
  MAINTAINER:=Entware team, http://entware.net
endef
define Package/samba4x-libs
  $(call Package/samba4x/Default)
  TITLE+= libraries
  DEPENDS:= +libcap +libarchive +libpopt +zlib +e2fsprogs +jansson +tdb
endef
define Package/samba4x-server
  $(call Package/samba4x/Default)
  TITLE+= server
  DEPENDS:=+samba4x-libs
endef
define Package/samba4x-client
  $(call Package/samba4x/Default)
  TITLE+= client
  DEPENDS:=+samba4x-libs
endef
define Package/samba4x-net
  $(call Package/samba4x/Default)
  TITLE+= net
  DEPENDS:=+samba4x-libs
endef
define Package/samba4x-server/description
  The Samba software suite is a collection of programs that implements the
  SMB protocol for UNIX systems, allowing you to serve files and printers to
  Windows, NT, OS/2 and DOS clients. This protocol is sometimes also referred
  to as the LanManager or Netbios protocol.
endef
CONFIGURE_VARS += \
    CPP="$(TARGET_CROSS)cpp" \
    NCURSES_CONFIG="$(STAGING_DIR)/host/bin/$(NCURSES_CONFIG_SCRIPTS)" \
    PYTHON="$(STAGING_DIR_HOSTPKG)/bin/python" \
    PYTHON_CONFIG="$(STAGING_DIR_HOSTPKG)/bin/python-config" \
    python_LDFLAGS="" \
    python_LIBDIR=""
MAKE_VARS += \
    PRINTCAP_NAME="/opt/etc/printcap"
CONFIGURE_PREFIX = /opt
CONFIGURE_ARGS = \
        --target=$(GNU_TARGET_NAME) \
        --program-prefix="" \
        --prefix=$(CONFIGURE_PREFIX) \
        --exec-prefix=$(CONFIGURE_PREFIX) \
        --bindir=$(CONFIGURE_PREFIX)/bin \
        --sbindir=$(CONFIGURE_PREFIX)/sbin \
        --libexecdir=$(CONFIGURE_PREFIX)/lib \
        --sysconfdir=/opt/etc \
        --datadir=$(CONFIGURE_PREFIX)/share \
        --localstatedir=/opt/var \
        --mandir=$(CONFIGURE_PREFIX)/man \
        --infodir=$(CONFIGURE_PREFIX)/info
CONFIGURE_ARGS += \
        --hostcc="$(shell which gcc)" \
        --cross-compile \
        --cross-answers="$(PKG_BUILD_DIR)/cache.txt" \
        --disable-avahi \
        --disable-cephfs \
        --disable-cups \
        --disable-fault-handling \
        --disable-glusterfs \
        --disable-iprint \
        --disable-python \
        --disable-rpath \
        --disable-rpath-install \
        --enable-fhs \
        --without-acl-support \
        --without-ads \
        --without-ad-dc \
        --without-automount \
        --without-dnsupdate \
        --without-iconv \
        --without-ldap \
        --without-lttng \
        --without-ntvfs-fileserver \
        --without-pam \
        --without-quotas \
        --without-systemd \
        --without-utmp \
        --without-winbind \
        --nopyc \
        --nopyo
       
#--disable-rpath-private-install       
CONFIGURE_ARGS += \
        --with-lockdir=/opt/var/lock/samba \
        --with-logfilebase=/opt/var/log/samba \
        --with-piddir=/opt/var/run/samba \
        --with-privatedir=/opt/etc/samba \
        --without-dmapi \
        --disable-gnutls \
        --with-relro \
        --with-sendfile-support \
        --without-cluster-support \
        --without-fam \
        --without-gettext \
        --without-pie \
        --without-regedit \
        --with-syslog \
        --bundled-libraries='talloc,com_err,!asn1_compile,!compile_et' \
        --enable-auto-reconfigure
       
CONFIGURE_CMD = ./buildtools/bin/waf
define Build/Configure
    $(CP) ./files/cache.txt $(PKG_BUILD_DIR)/cache.txt;
    echo -e "\nChecking uname sysname type: \"$(VERSION_DIST)\" \
    \nChecking uname release type: \"$(LINUX_VERSION)\" \
    \nChecking uname machine type: \"$(ARCH)\" \
    \nChecking uname version type: \"$(VERSION_DIST) Linux-$(LINUX_VERSION) $(shell date +%Y-%m-%d)\"\n" >> $(PKG_BUILD_DIR)/cache.txt;
    $(call Build/Configure/Default,configure)
endef
TARGET_CFLAGS += \
    -fdata-sections \
    -ffunction-sections
TARGET_LDFLAGS += \
    -Wl,--gc-sections
define Build/Compile
    $(call Build/Compile/Default,-j$(shell nproc))
endef
define Package/samba4x-libs/install
    $(INSTALL_DIR) $(1)/opt/lib
    $(CP) $(PKG_INSTALL_DIR)/opt/lib/*.so* $(1)/opt/lib
    $(CP) $(PKG_INSTALL_DIR)/opt/lib/samba/*.so* $(1)/opt/lib
endef
define Package/samba4x-server/install
    $(INSTALL_DIR) $(1)/opt/etc/samba $(1)/opt/etc/init.d
    $(INSTALL_DATA) ./files/smb.conf $(1)/opt/etc/samba
    $(INSTALL_DATA) $(PKG_BUILD_DIR)/examples/smb.conf.default $(1)/opt/etc/samba
    $(INSTALL_BIN) ./files/S91smb $(1)/opt/etc/init.d
    $(INSTALL_DIR) $(1)/opt/bin $(1)/opt/sbin
    $(INSTALL_BIN) $(PKG_INSTALL_DIR)/opt/sbin/{smbd,nmbd} $(1)/opt/sbin
    $(INSTALL_BIN) $(PKG_INSTALL_DIR)/opt/bin/{smbpasswd,smbstatus,pdbedit,eventlogadm,profiles} $(1)/opt/bin
    $(INSTALL_DIR) $(1)/opt/var/{cache,lock,log,run} $(1)/opt/var/lib/samba/private
endef
define Package/samba4x-client/install
    $(INSTALL_DIR) $(1)/opt/bin
    $(INSTALL_BIN) $(PKG_INSTALL_DIR)/opt/bin/smb{client,cacls,cquotas,get,spool,tree} $(1)/opt/bin
    $(INSTALL_BIN) $(PKG_INSTALL_DIR)/opt/bin/nmblookup $(1)/opt/bin
endef
define Package/samba4x-net/install
    $(INSTALL_DIR) $(1)/opt/bin
    $(INSTALL_BIN) $(PKG_INSTALL_DIR)/opt/bin/net $(1)/opt/bin
endef
define Package/samba4x-server/conffiles
/opt/etc/samba/smb.conf
/opt/etc/samba/smbpasswd
endef
$(eval $(call BuildPackage,samba4x-server))
$(eval $(call BuildPackage,samba4x-client))
$(eval $(call BuildPackage,samba4x-libs))
$(eval $(call BuildPackage,samba4x-net))

Note: it is installed to /opt/bin (/opt/sbin) vs /usr/bin (/usr/sbin). I am not sure, but currently it refers to 4.7.3 (4.8.0 in zyxmon's recent release).

Without the source hard to tell, but from the size of the entware server/lib packages, it looks like a default full compile, without any special handling of WAF's "unique" features. So yes my version is a little smaller and also has experimental support for AD-DC/Winbind/AVAHI. I also suspect it uses the embedded heimdal libs, while my versions uses krb5 libs.

I also have a custom luci ui for it, so you can easily add VFS modules.

Maybe it would be interesting for you to check samba from GPL of NG for R7800. Some attempts to cut samba4 too.

PS: Regarding a multi-call binary, that's simply not viable for samba4 and WAF, i tried this a year ago and had no real size improvements.

Well. It is my personal . I just very dislike resulting "single" multicall with a lot of a accompanied files in /usr/lib. It is hard to dig. Does not matter for most of people of course.

Thank you,
Voxel.

 

[andy22]

Thanks for the makefile, looks similar to the samba 4.4 file i based by first version on a year ago. I think i started with Wongs version: https://github.com/wongsyrone/lede-1/tree/master/package/external/samba44related

It uses the "problematic" cache.txt approach, where i question that anyone actually did know/check/test the answers for the arch's. It also does a default full compile and copies all libs, instead of telling WAF which targets it should build/install.

Its still cool that there is a samba4 entware package that works with default firmwares, since i don't supply generic packages. I was hoping to get the pull request done sooner, but had to fix various things. The last info was that someone from openwrt would test it locally and give feedback, so fingers crossed.

 

[RMerlin]

Anyone tried performance tests? I'm curious as to how's the throughput with either SMB2 or SMB3 versus 3.6.25 with a router's limited CPU.

 

[andy22]

Anyone tried performance tests? I'm curious as to how's the throughput with either SMB2 or SMB3 versus 3.6.25 with a router's limited CPU.

From my experience with a somewhat modern router samba 3.6 via smb2 will perform similar to samba4 with smb3 on a local LAN. I did get ~100MB/s from my attached esata ssd with stock firmware (wrt 1200ac) and ddwrt, similar to what i get now under samba4 and openwrt. The only difference i noticed is that i use softether vpn to remotely access my samba shares over wan and in this scenario samba4 with smb3 "felt" much more responsive and snappy.

Samba4 also has improved locking/io behavior, as example i have my PortableApps.com sitting in my samba share and use all tools from it directly. Under samba3.6 programs would often not start or work correctly if you try run them directly from a share. I remember that firefox had serious problems running from a share.

The other reason i wanted samba4 is for the VFS modules, in particular the shadow_copy2 and btrfs.

 

[RMerlin]

From my experience with a somewhat modern router samba 3.6 via smb2 will perform similar to samba4 with smb3 on a local LAN. I did get ~100MB/s from my attached esata ssd with stock firmware (wrt 1200ac) and ddwrt, similar to what i get now under samba4 and openwrt. The only difference i noticed is that i use softether vpn to remotely access my samba shares over wan and in this scenario samba4 with smb3 "felt" much more responsive and snappy.

Samba4 also has improved locking/io behavior, as example i have my PortableApps.com sitting in my samba share and use all tools from it directly. Under samba3.6 programs would often not start or work correctly if you try run them directly from a share. I remember that firefox had serious problems running from a share.

The other reason i wanted samba4 is for the VFS modules, in particular the shadow_copy2 and btrfs.

Do you disable SMB signing? From what I heard from another dev, it had a pretty significant impact on the performance on a low-powered router.

 

[andy22]

Do you disable SMB signing? From what I heard from another dev, it had a pretty significant impact on the performance on a low-powered router.

No, but than again the wrt 1200ac has a 1.3GHz dual-core cpu, so its not really "low-powered".

 

[RMerlin]

No, but than again the wrt 1200ac has a 1.3GHz dual-core cpu, so its not really "low-powered".

Compared to a 3 GHz x86 it kinda is Don't forget MIPS/ARM CPUs are not as advanced as an x86 part, so performance will be lower at equal clock rates.

The x86 will also benefit from AES-NI, which the router lacks.

 

[Voxel]

Anyone tried performance tests? I'm curious as to how's the throughput with either SMB2 or SMB3 versus 3.6.25 with a router's limited CPU.

Very brief testing of performance using SMB2. 3.6.25 vs 4.6.x (the last was compiled using GPL of R7800). Results are 87-91MB/s vs 68-70MB/s. 3.6.25 wins. R7800.

Samba was forced to use second core, and USB driver to use first core for 3.6.25 and 4.6.x


Voxel.