Tor on ASUS RT-AC66U after FW upgrade to 380.67

[Fitz Mutch]

Watchdog only creates a backup of the database if there isn't already one. Doesn't that mean that this database will become stale after a while?

Yes, it would become stale. You could probably get away with removing that entire block of code from watchdog. So every time you reboot the router, Tor would re-cache the certs and descriptors.

 

[Fitz Mutch]

I can't say that I know what I'm doing but I executed rm -rf /jffs/.tordb and ...

It's exactly what Merlin said, just "rm -rf /jffs/.tordb" and reboot the router.

 

[RMerlin]

Yes, it would become stale. You could probably get away with removing that entire block of code from watchdog. So every time you reboot the router, Tor would re-cache the certs and descriptors.

I wonder how useful that whole cache is anyway. First, every times I started Tor today during my tests, it never created the log entry watchdog was looking for, therefore it would never cache anything for me. The referred string does exist in Tor's sources tho, so I don't know what triggers it. Makes me wonder if it shouldn't simply be deleted - maybe caching was more critical in Tor's earlier days? It also never took more than a few seconds before it was able to establish a working circuit, and my routed VM was able to go through the Tor network.

 

[Fitz Mutch]

... maybe caching was more critical in Tor's earlier days?

The Tor data directory is always cached in RAM anyway until the router is rebooted. Plus, backing up the Tor data directory while Tor is running could cause file corruption issues.

Been using Tor on the router for a long time without the cache. I haven't seen Tor take longer than 60 seconds to start, after a reboot. Most of the time it's really fast. Every once in a while, you'll encounter a malicious or misconfigured guard node that prevents you from connecting to the Tor network. In this case, maybe having the cache would help, but it's rare. The solution is simply, service restart_tor.

 

[RMerlin]

Thanks. I'll consider just removing the cache backup code.

 

[kuzmwitch]

Hi
Thanks ALL for support and help
Now after
rm -rf /jffs/.tordb
everything fine with Tor
Also check after router reboot

Now I on FW 380.67

Best regards
Sergey

 

[RMerlin]

The Tor data directory is always cached in RAM anyway until the router is rebooted. Plus, backing up the Tor data directory while Tor is running could cause file corruption issues.

Just thought about it... How about, instead of using wanduck to do one single backup (and never update it), what if we always backed up the database when tor is being stopped (as part of stop_tor())? This way, it would ensure a few things:

1) Database would be refreshed every time, so no more stale database
2) Backing it up after Tor is shut down would ensure that the data isn't being written to, and therefore is in a more consistent state

 

[Fitz Mutch]

... what if we always backed up the database when tor is being stopped ...

Yes, it's a good idea. If the backup and restore logic is exactly as follows, then it would update only when the files have changed, to reduce wear on the flash memory. Also, it could allow the source and destination directories to be symbolic links.

Backup

/bin/mkdir -p /jffs/.tordb
/bin/cp -afu /tmp/.tordb/* /jffs/.tordb/.

Restore

/bin/mkdir -p /tmp/.tordb
/bin/cp -afu /jffs/.tordb/* /tmp/.tordb/.

 

[RMerlin]

I'll go ahead with backing up the database at shutdown time. I will also only overwrite the backup if it's more than a few days old, to limit the amount of flash writes, and also ensure the database does not get too stale.

 

[RMerlin]

Here's the new mechanism:

https://github.com/RMerl/asuswrt-merlin/commit/66fff3b550540c3238c47d82555a2e18caad6131

 

[Fitz Mutch]

Here's the new mechanism ...

I like your idea of nuking the /jffs/.tordb directory, when Tor hasn't been used in a while.

Also, I noticed a big file in /tmp/.tordb hasn't changed in 24 hours. I don't know if these two files are static. The others are definitely updated more frequently.

-rwx------    1 tor      tor          18209 Aug  5 13:51 cached-certs
-rwx------    1 tor      tor        3531757 Aug  5 14:21 cached-microdescs

So, when backing up the files to /jffs, you could use the -u option with cp, so it will check the file times and only copy the newer files. The big file shown above doesn't need to be copied to /jffs every time, since it's already there. That's why my cp command is:
cp -afu /tmp/.tordb/* /jffs/.tordb/.

 

[RMerlin]

So, when backing up the files to /jffs, you could use the -u option with cp, so it will check the file times and only copy the newer files. The big file shown above doesn't need to be copied to /jffs every time, since it's already there. That's why my cp command is:
cp -afu /tmp/.tordb/* /jffs/.tordb/.

I didn't want to get too fancy in determining what to update and what to keep, so
I wipe the whole backup if it's older than 7 days, so there's no file left to potentially update.

 

[Fitz Mutch]

I didn't want to get too fancy ...

Here's my Tor backup and restore code, as implemented in services.c. It minimizes writes to /jffs, and supports symbolic linking of the .tordb directory. It doesn't purge the cache every 7 days.

~/asuswrt-merlin/release/src/router/rc/services.c

#if (defined(RTCONFIG_JFFS2)||defined(RTCONFIG_BRCM_NAND_JFFS2))
        //logmessage("Tor", "Backing up database");
        eval("/bin/sh", "-c", "if [ -d /tmp/.tordb ] ; then "
                              "  mkdir -p /jffs/.tordb ; "
                              "  cp -afu /tmp/.tordb/* /jffs/.tordb/ ;"
                              "fi");
#endif

# ...

#if (defined(RTCONFIG_JFFS2)||defined(RTCONFIG_BRCM_NAND_JFFS2))
        //logmessage("Tor", "Restoring database");
        eval("/bin/sh", "-c", "mkdir -p /tmp/.tordb ; "
                              "cp -afu /jffs/.tordb/* /tmp/.tordb/ ; "
                              "chown tor:tor /tmp/.tordb ; "
                              "chown -h tor:tor /tmp/.tordb ; "
                              "chown -R tor:tor /tmp/.tordb/* ; "
                              "chmod u=rwx,g-rwx,o-rwx /tmp/.tordb ; "
                              "chmod -R u+rw,g-rwx,o-rwx /tmp/.tordb/*");
        sleep(1);
#endif

 

[Fitz Mutch]

His firmware was updated with this change. Just testing it now.

https://github.com/blackfuel/asuswrt-merlin/commit/447e213224f2a5a13341c23c25f9f2bcd9164a5b