[ILUG] rsync over ssh problem
brian.omahony at curamsoftware.com
Wed Aug 18 11:25:38 IST 2010
My issue is 0 bytes are transferred (just checked) in the first sync at this time. However other things are getting sync'd on the hour. This is just CVS folders. I have a feeling as the folders are so large the reading file lists bit is timing out. However if I do these manually iot works fine, and all the rest (about 15 sync's) work fine after the first fails during the night. Which I find weird.
I have implemented the change Marcus recommended and will see how it goes tonight.
From: ilug-bounces at linux.ie [mailto:ilug-bounces at linux.ie] On Behalf Of kevin
Sent: Wednesday, August 18, 2010 11:18 AM
To: Irish Linux Users Group
Subject: RE: [ILUG] rsync over ssh problem
On Wed, 2010-08-18 at 11:04 +0100, Brian O'Mahony wrote:
> FYI I have been having this issue sync'ing data over an MPLS between Dublin & Bangalore on a nightly basis. It only happens on the first data transfer of the night, and things are quiet (ie its 5am BLR time, and 1am Irish time, so there should be no one in the office.)
> I would be very interested if you find a solution. To be honest I have ignored this for a good few months (I just scheduled an extra data transfer a minute before all the rest kick off that I know will fail).
> -----Original Message-----
> From: ilug-bounces at linux.ie [mailto:ilug-bounces at linux.ie] On Behalf Of kevin
> Sent: Wednesday, August 18, 2010 10:00 AM
> To: Irish Linux Users Group
> Subject: [ILUG] rsync over ssh problem
> Part of our backup routine is a cron job that runs rsync over ssh to an
> offsite computer every night. Recently I have noticed that it fails
> about 1/2 of the time, not always and a failed rysnc one night might
> work fine the next night.
> The error we get is:
> Write failed: Broken pipe
> rsync: connection unexpectedly closed (13729658 bytes received so far)
> rsync error: error in rsync protocol data stream (code 12) at io.c(601)
> rsync: connection unexpectedly closed (783 bytes received so far)
> rsync error: unexplained error (code 255) at io.c(601) [generator=3.0.7]
> I have the "KeepAlive Yes" option in the sshd_config file
> rsync options used in the batch file are:
> rsync -e ssh -avzxHS --numeric-ids --del --delete-excluded
> --exclude-from "/home/cybercolloids/exclusions.txt" --exclude ".gvfs"
> Looking at the log file the only thing that looks odd is that the
> connection fails when transferring large image files - a few Mb
> Any ideas?
An interesting point - I actually have a batch file that runs through a
series of rsync commands - one for each partition on the raid array.
However the first is the /home directory which is the one that changes
most often and is the most important.
I will also swap the commands over so that rsync /home is further down
the list and see if that helps.
Irish Linux Users' Group mailing list
About this list : http://mail.linux.ie/mailman/listinfo/ilug
Who we are : http://www.linux.ie/
Where we are : http://www.linux.ie/map/
The information in this email is confidential and may be legally privileged.
It is intended solely for the addressee. Access to this email by anyone else
is unauthorized. If you are not the intended recipient, any disclosure,
copying, distribution or any action taken or omitted to be taken in reliance
on it, is prohibited and may be unlawful. If you are not the intended
addressee please contact the sender and dispose of this e-mail. Thank you.
More information about the ILUG