Entware [DEVEL] Cross-Compile python3-onionbalance Entware Package via Debian Live DVD

[garycnew]

All:

I'm in the process of trying to build an Entware python3-onionbalance package. The process is a little more complicated with no existing OpenWRT Makefile, built using Python3, and missing Entware Python3 Dependencies. I've started by modifying a basic, existing OpenWRT Makefile (i.e., python3-bottle) and have been able to get it to build with readily available and newly built dependencies. It's coming along better than first expected.

[NOTE: I am attempting to use an SRPM python-onionbalance.spec file as a loose guide to create the Entware Makefile.]

The following onionbalance dependencies with their appropriate OpenWRT/Entware package names:

python3-setuptools (AVAILABLE IN ENTWARE REPO)
python3-stem >=1.8 (BUILD FROM OPENWRT REPO)
python3-pyyaml >=4.2b1 is python3-yaml (AVAILABLE IN ENTWARE REPO)
python3-cryptography >=3.2 (AVAILABLE IN ENTWARE REPO)
python3-pycryptodomex is python3-cryptodomex (BUILD FROM OPENWRT REPO)
python3-future >=0.14.3 is python3-future (CREATE)
python3-setproctitle >=1.1.9 (AVAILABLE IN ENTWARE REPO)

Respectfully,


Gary

# opkg install python3-setuptools
Installing python3-setuptools (49.2.1-1) to root...
Downloading http://bin.entware.net/armv7sf-k2.6/python3-setuptools_49.2.1-1_armv7-2.6.ipk
Installing python3-pkg-resources (49.2.1-1) to root...
Downloading http://bin.entware.net/armv7sf-k2.6/python3-pkg-resources_49.2.1-1_armv7-2.6.ipk
Configuring python3-pkg-resources.
Configuring python3-setuptools.

# opkg install ./python3-stem_1.8.0-2_armv7-2.6.ipk
Installing python3-stem (1.8.0-2) to root...
Configuring python3-stem.

# opkg install python3-yaml
Installing python3-yaml (5.3.1-2) to root...
Downloading http://bin.entware.net/armv7sf-k2.6/python3-yaml_5.3.1-2_armv7-2.6.ipk
Installing libyaml (0.2.5-1) to root...
Downloading http://bin.entware.net/armv7sf-k2.6/libyaml_0.2.5-1_armv7-2.6.ipk
Configuring libyaml.
Configuring python3-yaml.

# opkg install python3-cryptography
Installing python3-cryptography (3.3.2-1) to root...
Downloading http://bin.entware.net/armv7sf-k2.6/python3-cryptography_3.3.2-1_armv7-2.6.ipk
Installing python3-ply (3.11-2) to root...
Downloading http://bin.entware.net/armv7sf-k2.6/python3-ply_3.11-2_armv7-2.6.ipk
Installing python3-pycparser (2.20-5) to root...
Downloading http://bin.entware.net/armv7sf-k2.6/python3-pycparser_2.20-5_armv7-2.6.ipk
Installing python3-cffi (1.14.5-1) to root...
Downloading http://bin.entware.net/armv7sf-k2.6/python3-cffi_1.14.5-1_armv7-2.6.ipk
Installing python3-six (1.15.0-1) to root...
Downloading http://bin.entware.net/armv7sf-k2.6/python3-six_1.15.0-1_armv7-2.6.ipk
Configuring python3-ply.
Configuring python3-pycparser.
Configuring python3-cffi.
Configuring python3-six.
Configuring python3-cryptography.

# opkg install ./python3-cryptodomex_3.9.7-2_armv7-2.6.ipk
Installing python3-cryptodomex (3.9.7-2) to root...
Installing libgmp (6.2.1-1a) to root...
Downloading http://bin.entware.net/armv7sf-k2.6/libgmp_6.2.1-1a_armv7-2.6.ipk
Configuring libgmp.
Configuring python3-cryptodomex.

# opkg install ./python3-future_0.18.2-1_armv7-2.6.ipk
Installing python3-future (0.18.2-1) to root...
Configuring python3-future.

# opkg install python3-setproctitle
Installing python3-setproctitle (1.2.2-1) to root...
Downloading http://bin.entware.net/armv7sf-k2.6/python3-setproctitle_1.2.2-1_armv7-2.6.ipk
Configuring python3-setproctitle.

# opkg install ./python3-onionbalance_0.2.1-1_armv7-2.6.ipk
Installing python3-onionbalance (0.2.1-1) to root...
Installing python3-base (3.9.4-1) to root...
Installing libffi (3.3-2) to root...
Installing libbz2 (1.0.8-1) to root...
Installing python3-light (3.9.4-1) to root...
Installing python3-unittest (3.9.4-1) to root...
Installing python3-ncurses (3.9.4-1) to root...
Installing python3-ctypes (3.9.4-1) to root...
Installing python3-pydoc (3.9.4-1) to root...
Installing python3-logging (3.9.4-1) to root...
Installing python3-decimal (3.9.4-1) to root...
Installing python3-multiprocessing (3.9.4-1) to root...
Installing python3-codecs (3.9.4-1) to root...
Installing libgdbm (1.19-1) to root...
Installing python3-gdbm (3.9.4-1) to root...
Installing libsqlite3 (3330000-2) to root...
Installing python3-sqlite3 (3.9.4-1) to root...
Installing python3-email (3.9.4-1) to root...
Installing python3-urllib (3.9.4-1) to root...
Installing libexpat (2.2.10-1) to root...
Installing python3-xml (3.9.4-1) to root...
Installing python3-distutils (3.9.4-1) to root...
Installing python3-openssl (3.9.4-1) to root...
Installing python3-cgi (3.9.4-1) to root...
Installing python3-cgitb (3.9.4-1) to root...
Installing libdb47 (4.7.25.4.NC-7) to root...
Installing python3-dbm (3.9.4-1) to root...
Installing liblzma (5.2.5-3) to root...
Installing python3-lzma (3.9.4-1) to root...
Installing python3-asyncio (3.9.4-1) to root...
Installing python3 (3.9.4-1) to root...
Configuring python3-base.
Configuring libffi.
Configuring libbz2.
Configuring python3-light.
Configuring python3-email.
Configuring python3-urllib.
Configuring python3-pydoc.
Configuring liblzma.
Configuring python3-unittest.
Configuring python3-ncurses.
Configuring python3-ctypes.
Configuring python3-logging.
Configuring python3-decimal.
Configuring python3-multiprocessing.
Configuring python3-codecs.
Configuring libgdbm.
Configuring python3-gdbm.
Configuring libsqlite3.
Configuring python3-sqlite3.
Configuring libexpat.
Configuring python3-xml.
Configuring python3-distutils.
Configuring python3-openssl.
Configuring python3-cgi.
Configuring python3-cgitb.
Configuring libdb47.
Configuring python3-dbm.
Configuring python3-lzma.
Configuring python3-asyncio.
Configuring python3.
Configuring python3-onionbalance.

# opkg list-installed|grep -i onion
python3-onionbalance - 0.2.1-1

# opkg files python3-onionbalance
Package python3-onionbalance (0.2.1-1) is installed on root and has the following files:
/opt/lib/python3.9/site-packages/OnionBalance-0.2.1-py3.9.egg-info/requires.txt
/opt/bin/onionbalance-config
/opt/lib/python3.9/site-packages/onionbalance/hs_v2/service.pyc
/opt/lib/python3.9/site-packages/onionbalance/hs_v2/descriptor.pyc
/opt/bin/onionbalance
<TRUNCATED>

# opkg info python3-onionbalance
Package: python3-onionbalance
Version: 0.2.1-1
Depends: libc, libssp, librt, libpthread, python3, python3-setuptools, python3-stem, python3-yaml, python3-cryptography, python3-cryptodomex, python3-future, python3-setproctitle, tor
Status: install user installed
Architecture: armv7-2.6
Installed-Time: 1624697206

# /opt/bin/onionbalance
2021-06-26 08:48:33,860 [WARNING]: Initializing onionbalance (version: 0.2.1)...
2021-06-26 08:48:33,949 [ERROR]: The specified config file '~/config.yaml' does not exist. The onionbalance-config tool can generate the required keys and config files.

# opkg remove python3-onionbalance python3-future python3-cryptodomex libgmp python3-cryptography python3-six python3-cffi python3-yaml libyaml python3-stem python3-setuptools python3-pkg-resources python3-setproctitle python3 python3-decimal python3-lzma python3-asyncio python3-codecs python3-dbm libdb47 python3-cgitb python3-cgi python3-openssl python3-distutils python3-xml libexpat python3-sqlite3 libsqlite3 python3-gdbm libgdbm python3-readline python3-multiprocessing python3-logging python3-ctypes python3-ncurses python3-unittest liblzma python3-pydoc python3-urllib python3-email python3-pycparser python3-ply python3-light libbz2 libffi python3-base libpython3

 

[heysoundude]

cryptodome and cryptodomex - there IS a slight difference IIRC. maybe to do with processor of host? It could be that domex is for earlier ARM and dome is for newer? (I could have that bass ackwards)
It's a dependency for the Netflix app I use on my kodi box...I'd have to dig back into my research from over a year ago when I first looked it up to to try and get it refreshed..
but good on you moving forward with py3

 

[garycnew]

All:

I have to say, my hat goes off to the OpenWRT, Entware, & PYPI Developers whom have seemlessly integrated the building of Python packages. I was introduced to the process a week ago and it has been a joy to work with. It could have been a completely different experience. Thank you for all your hard work!

Had to build several package dependency to get onionbalance to successfully install and work.

Some already had Makefiles and others did not:

python3-stem_1.8.0-2_armv7-2.6.ipk (Used Existing Makefile)
python3-cryptodomex_3.9.7-2_armv7-2.6.ipk (Used Existing Makefile)
python3-future_0.18.2-1_armv7-2.6.ipk (Created New Makefile)
python3-onionbalance_0.2.1-1_armv7-2.6.ipk (Created New Makefile)

### New python3-future Makefile ###

$ cat python3-future/Makefile
#
# Copyright (C) 2015 OpenWrt.org
#
# This is free software, licensed under the GNU General Public License v2.
# See /LICENSE for more information.
#

include $(TOPDIR)/rules.mk

PKG_NAME:=python3-future
PKG_VERSION:=0.18.2
PKG_RELEASE:=1

PYPI_NAME:=future
PKG_HASH:=b1bead90b70cf6ec3f0710ae53a525360fa360d306a86583adc6bf83a4db537d

PKG_MAINTAINER:=Ed Schofield <ed@pythoncharmers.com>
PKG_LICENSE:=MIT
PKG_LICENSE_FILES:=LICENSE

include ../pypi.mk
include $(INCLUDE_DIR)/package.mk
include ../python3-package.mk

define Package/python3-future
    SECTION:=lang
    CATEGORY:=Languages
    SUBMENU:=Python
    TITLE:=future
    URL:=https://python-future.org/
    DEPENDS:=+python3
endef

define Package/python3-future/description
future is the missing compatibility layer between Python 2 and Python 3. It allows you to use a single, clean Python 3.x-compatible codebase to support both Python 2 and Python 3 with minimal overhead.
endef

$(eval $(call Py3Package,python3-future))
$(eval $(call BuildPackage,python3-future))
$(eval $(call BuildPackage,python3-future-src))

### New python3-onionbalance Makefile ###

$ cat python3-onionbalance/Makefile
#
# Copyright (C) 2015 OpenWrt.org
#
# This is free software, licensed under the GNU General Public License v2.
# See /LICENSE for more information.
#

include $(TOPDIR)/rules.mk

PKG_NAME:=python3-onionbalance
PKG_VERSION:=0.2.1
PKG_RELEASE:=1

PYPI_NAME:=OnionBalance
PKG_HASH:=3421e5fdef24674432021c605164c790ff7f66ec2c111f3e610b212c058b33ea

PKG_MAINTAINER:=Donncha O'Cearbhaill <donncha@donncha.is>
PKG_LICENSE:=GPLv3
PKG_LICENSE_FILES:=LICENSE

include ../pypi.mk
include $(INCLUDE_DIR)/package.mk
include ../python3-package.mk

define Package/python3-onionbalance
    SECTION:=lang
    CATEGORY:=Languages
    SUBMENU:=Python
    TITLE:=onionbalance
    URL:=https://onionbalance.readthedocs.io
    DEPENDS:=+python3 +python3-setuptools +python3-stem +python3-yaml +python3-cryptography +python3-cryptodomex +python3-future +python3-setproctitle +tor
endef

define Package/python3-onionbalance/description
OnionBalance provides load-balancing and redundancy for Tor onion services by distributing requests to multiple back-end Tor instances.
endef

$(eval $(call Py3Package,python3-onionbalance))
$(eval $(call BuildPackage,python3-onionbalance))
$(eval $(call BuildPackage,python3-onionbalance-src))

After installing these and other readily available dependencies, I was able to get the onionbalance binary to successfully report its version at the command-line. The python3-onionbalance package could still use some custom directories and unslung scripts, but we'll save those for another day.

Next step is to create an onionbalance config and tor config the AiMesh... I mean TorNodes. ;-)

Attached to this post are the newly built armv7-2.6 packages (download and rename them from .txt to .ipk to install). Please feel free to help test them and report your results back to this post.

Respectfully,


Gary

References:

https://onionbalance.readthedocs.io/_/downloads/en/stable/pdf/
https://pypi.org/project/OnionBalance/
https://pypi.org/project/OnionBalance/0.1.2/
https://pypi.org/project/future/
https://openwrt.org/docs/guide-developer/helloworld/chapter3
https://openwrt.org/faq/include_tailored_custom_configuration_files
https://openwrt.org/docs/guide-developer/packages
https://openwrt.org/docs/guide-developer/helloworld/chapter3
https://openwrt.org/docs/guide-developer/build-system/use-buildsystem#custom_files
https://github.com/Entware/entware-packages/blob/master/lang/python/python3-bottle/Makefile
https://src.fedoraproject.org/rpms/python-onionbalance/blob/rawhide/f/python-onionbalance.spec
https://rpmfind.net/linux/RPM/fedora/devel/rawhide/s390x/p/python3-onionbalance-0.2.0-6.fc35.noarch.html