These tweaked settings have been working really well on my RT-AC86U - no crashes and impressive speeds.

RAM consumption runs up to 95% and stays there with small use of Swap file.
Just wish we didn't lose these conf settings and trash the cache to start all over again every time it is necessary to run "u" followed by "i" .
Code:
# no threads and no memory slabs for threads
num-threads: 2
msg-cache-slabs: 4
rrset-cache-slabs: 4
infra-cache-slabs: 4
key-cache-slabs: 4
# tiny memory cache
key-cache-size: 16m
msg-cache-size: 16m
rrset-cache-size: 32m
cache-max-ttl: 21600
cache-min-ttl: 0
prefetch: yes
prefetch-key: yes
serve-expired: yes
serve-expired-ttl: 3600
incoming-num-tcp: 1024
outgoing-num-tcp: 128
ip-ratelimit: 0
edns-buffer-size: 4096