Slow rsync

[BassKozz]

I am running an rsync (downloaded via entware) from a remote server to my USB Mounted Hard Drive. I am only seeing throughput at 250KB/s. To test if the issue was the USB connection I rsynced /usr to the USB HD, and I was able to achieve 2.4 MB/s. To Really ensure the USB HD wasn't the culprit I tried the rsync directly to /tmp (tmpfs) and still I was only getting 251KB/s. Lastly I tried pushing from the remote server to my router (after opening up WAN SSH which is not something I want to do for a long period of time), and still only getting 226KB/s.

Also testing pulling directly from remote server to my local machine I was able to get 2.25 MB/s.

My Internet connection is a 50Mb/s down FiOS connection.
The USB HD I am using is: WD Elements SE 750 GB USB 2.0 (formatted ext3)

I realize rsync delta algorithm uses _some_ CPU, but watching top while running the rsync didn't show much activity CPU wise (avg 80~90% idle). It's almost as if my connection is being throttled when using the Router directly (p.s. QoS is disabled).
So what might be causing the rsync directly on the router to run this slowly

-BassKozz

 

[jvande]

I too have very slow rsync transfer speeds. I was able to get my SSH keys working after a lot of reading. (Thanks for the link - that's the one I used too) Transfers start at 15MB/s and degrade quickly such that within 60 secs they have "stabilized" at about 250kb/s - much slower than my 20MB connection. I have also found that transfers tend to time out or stop after a certain time with a .c error:


ASUSWRT RT-N66U_3.0.0.4 Sun Mar 17 19:23:55 UTC 2013
xxx@SERVER:/tmp/home/root# /jffs/scripts/backup.sh
receiving incremental file list
monthly/
monthly/backup-2.1.2013_03-32-26_file1.tar.gz
1768878234 28% 219.91kB/s 5:32:09
rsync: connection unexpectedly closed (886533393 bytes received so far) [receiver]
rsync error: error in rsync protocol data stream (code 12) at io.c(605) [receiver=3.0.9]
rsync: connection unexpectedly closed (8808 bytes received so far) [generator]
rsync error: unexplained error (code 255) at io.c(605) [generator=3.0.9]

I'm exploring using wget instead to see if the transfer speeds are any better.

 

[BassKozz]

I ran a wget test and downloaded a 100MB file in 2.8s at about 3.2MB/s while SSHed into my router:

wget -O /dev/null http://proof.ovh.net/files/100Mb.dat

So it doesn't look like this is related to any sort of throttling, simply rsync running really slowly....

 

[BassKozz]

I figured it out !
Dropbear SSH was the culprit !

I installed openssh-client:

opkg install openssh-client

Now my rsync calls that ssh client instead:

rsnyc -e "/tmp/mnt/<usbhdname>/entware/bin/ssh -i <identityfile>" <source> <destination>

BINGO !

Though I would like to know why Dropbear ssh is so shirtty?

 

[jvande]

This is good to know - now I just have to take the time to figure out (again) how to set up ssh keys again - or can we use the same key?

 

[BassKozz]

This is good to know - now I just have to take the time to figure out (again) how to set up ssh keys again - or can we use the same key?

You can use the same key, but it's not difficult to generate new ones:

ssh-keygen -t rsa -b 2048 -f /path/to/key

See also: http://linux.die.net/man/1/ssh-keygen

 

[whycomputers]

I figured it out !
Dropbear SSH was the culprit !

I installed openssh-client:

opkg install openssh-client

Now my rsync calls that ssh client instead:

rsnyc -e "/tmp/mnt/<usbhdname>/entware/bin/ssh -i <identityfile>" <source> <destination>

BINGO !

Though I would like to know why Dropbear ssh is so shirtty?

What speed did you get after using the different ssh?
I am thinking to use that to sync media files and then share them with DLNA server.
Few years ago I tried ssh over rsync with a WL0500g p2 and performance was terrible, not sure if it was the old CPU or again a shirtty ssh.

Thank you,