Looks like it's possible for rsync to get into a state where it stops transferring data and when this happens the connection will stay live forever.
So pass --timeout with a configurable default (probably 30 seconds) to rsync and catch the return value (30). Should probably abort the transfer when this happens instead of retrying.
Commit 49d21168 fixes this issue