Monday, January 08, 2007

Rsync & SSH on a Windows 2003 server

In my previous post about using ssh & rsync for backups, I wrote about backing up from my Fedora UNIX virtual machine to my UNIX backup server.

That was a fairly simple case (other than the need for the enveloping ssh sessions) because both systems had the requisite tools installed & configured. However, in a windows environment, those tools aren't there out of the box and need to be installed & configured as well as getting the script setup to do the backup.

This post addresses the steps necessary to setup a Windows 2003 server as an rsync server for backups from Windows clients. I'll address the client side in a future article. While I set this up on a Windows 2003 server, the steps most likely apply equally to any Windows server OS. \

Since the OS does not come with the necessary tools, I turned to the cygwin distribution of UNIX utilities for Windows.

Server Setup

  1. Download cygwin's setup.exe, which you then use to download and install cygwin components. I installed the complete package. I'm sure that you could get by with a lesser install, but given the cheap cost of disk space, why bother.
  2. Initialize sshd on your server:
    cpcahil(mercury,505): ssh-host-config
    Generating /etc/ssh_host_key
    Generating /etc/ssh_host_rsa_key
    Generating /etc/ssh_host_dsa_key
    Generating /etc/ssh_config file
    Privilege separation is set to yes by default since OpenSSH 3.3.
    However, this requires a non-privileged account called 'sshd'.
    For more info on privilege separation read /usr/share/doc/openssh/README.privsep.
    Should privilege separation be used? (yes/no) yes
    Warning: The following function requires administrator privileges!
    Should this script create a local user 'sshd' on this machine? (yes/no) yes
    Generating /etc/sshd_config file
    Added ssh to C:\WINDOWS\system32\drivers\etc\services
    Added ssh to /etc/inetd.conf
    Warning: The following functions require administrator privileges!
    Do you want to install sshd as service?
    (Say "no" if it's already installed as service) (yes/no) yes
    Which value should the environment variable CYGWIN have when
    sshd starts? It's recommended to set at least "ntsec" to be
    able to change user context without password.
    Default is "ntsec".  CYGWIN=ntsec
    The service has been installed under LocalSystem account.
    To start the service, call `net start sshd' or `cygrunsrv -S sshd'.
    Host configuration finished. Have fun!
  3. Configure sshd as appropriate

    For my server, I edited /etc/sshd_config to make the following changes:

    *** sshd_config.old     Sun Jan 16 07:31:24 2005
    --- sshd_config Sun Jan 16 16:38:38 2005
    *** 28,34 ****
      # Logging
      #obsoletes QuietMode and FascistLogging
      #SyslogFacility AUTH
    ! #LogLevel INFO
      # Authentication:
    --- 28,34 ----
      # Logging
      #obsoletes QuietMode and FascistLogging
      #SyslogFacility AUTH
    ! LogLevel INFO
      # Authentication:
    *** 39,45 ****
      #RSAAuthentication yes
      #PubkeyAuthentication yes
    ! #AuthorizedKeysFile   .ssh/authorized_keys
      # For this to work you will also need host keys in /etc/ssh_known_hosts
      #RhostsRSAAuthentication no
    --- 39,45 ----
      #RSAAuthentication yes
      #PubkeyAuthentication yes
    ! AuthorizedKeysFile    /etc/authorized_keys
      # For this to work you will also need host keys in /etc/ssh_known_hosts
      #RhostsRSAAuthentication no

    The key change there being the specification of the /etc/authorized_keys file as I use private keys to authenticate my clients to the server.

    This isn't necessary if you want to use username/password authentication as the out-of-the-box sshd implementation will use the windows active directory for authentication.

  4. Open port 22 on your firewall

    If you are running a firewall on your server, you need to open up incoming connections to port 22 (the port used by sshd) so that your clients can communicate with the sshd daemon (service) on your server.

  5. Start the "CYGWIN sshd" service. This can be done using the Windows Control Panel->Administrator's Tools->Services or via the following command:
    net start sshd
  6. Test sshd availability

    Run the command: telnet localhost 22

    The output should look like:

    bash-3.1$ telnet localhost 22
    Connected to
    Escape character is  '^]'.

    the key being the last line. Once you see that, sshd is fine and you can abort the telnet (<ctrl>-] followed by quit).

  7. Configure Rsyncd for your system

    The rsyncd server was installed with the rest of CYGWIN in the first step above, so all we need to do is configure it as necessary. First part of the configuration is the /etc/rsyncd.conf file (if you installed CYGWIN into c:\cygwin, then the windows path for the file is c:\cygwin\etc\rsyncd.conf). If the file doesn't exist already create it. A portion of my rsyncd.conf:

    use chroot = false
    strict modes = false
    hosts allow = *
    uid = administrator
    secrets file = /etc/rsync.secrets.txt
            path = e:/data/angie
            read only = no
            auth users = angie

    The important lines in this file:

    • strict modes = false

      This disables strict access mode checking on the rsync.secrets.txt file (which rsyncd normally requires to have no access to anyone other than the owner).
    • secrets file = /etc/rsync.secrets.txt

      This tells rsync where the secrets file is located. The secrets file contains a list of id:password combinations. My secrets file looks like:

      This defines a secret for my wife (Angie) and I.
    • [angie]
              path = e:/data/angie
              read only = no
              auth users = angie

      This defines an rsync "module" (which you could look at as the equivalent of a rsync shared drive entry point). In this case the the module "angie" equates to the windows path "e:/data/angie", is not read-only and is only available to an rsync client that can present angie's secret.

  8. Install rsyncd as a service

    Run the command (in a cygwin bash shell window):

    cygrunsrv.exe -I "Rsync" -p /cygdrive/c/cygwin/bin/rsync.exe -a "--daemon --no-detach" -f "Rsync daemon service" -u Administrator -w admin_passwd

    That is all on one line (in case it wrapped in your browser).

  9. Start the rsync service (yeah, it's the rsyncd daemon (by UNIX naming conventions), but called the rsync service in Windows). This can be done using the Windows Control Panel->Administrator's Tools->Services or via the following command:
    net start rsync
  10. I did not have to open a firewall port for rsyncd since I would only be accessing rsyncd via an ssh tunnel (so the connection to rsyncd is a local connection from the sshd process).
  11. Test rsyncd availability

    Run the command: telnet localhost port
    Where port is the port chosen for rsyncd to listen on (the default 873).

    The output should look like:

    bash-3.1$ telnet localhost 873
    Connected to
    Escape character is  '^]'.
    @RSYNCD: 29

    the key being the last line. Once you see that, rsyncd is fine and you can abort the telnet (<ctrl>-] followed by quit).


The system works very well for backups. One problem that we have experienced is that permissions set on files and folders created on the server are messed up. They are owned by the user id that rsyncd is being run as (in my case, administrator) and read, write and execute permissions are turned off. This means that when I add new photos to our photo collection on our server, my wife can't view them until I get onto the server and reset the permissions.

I have tried a few things to fix this to no avail. Others on the net have claimed that setting the CYGWIN environment variable to "nontsec" (as opposed to the default of "ntsec"). I tried that and it had an extremely negative impact on performance (the backup of my photo directory -- with no changes -- takes about 40 seconds normally, but with "nontsec" I killed it after 40 minutes with no perceived progress.

So, for now I live with the permissions problem.

Tags : / / / / / /


Anonymous said...

Many thanks for the tutorial. Your rsyncd.conf example certainly saved me much detailed reading of the man page.

I didn't experience your problem with file ownership. I did have to start the rsyncd service as the Administrator user, but my files are owned by the user specified in my rsync commands. The only places I strayed from your settings is that I didn't set uid, secrets file or auth users in the rsyncd.conf file. I did try setting uid to administrator to see if I could reproduce the problem, but it made no difference in my results. The only other difference I could think of is that I'm running XP Pro on both machines.

Thanks Again,

Brad Wosmek

Anonymous said...

Hey Conor, great information.

Question: Did you ever figure out the issue with permissions on the destination server.

I too need to figure out a way to make it so the rsync user doesnt take ownership over the files?


Conor P. Cahill said...

Sadly I have to admit that I have not found a solution and have just lived with the problem. I tried numerous ways to solve the problem to no avail (though I am sure it's just one little tweak here/there and it will all magically work).

For now, I just periodically use VNC to get on the server and manually set the permissions on the files.

Neville said...

You can do this for sure. You can download ROBOCOPY (comes with the Windows Resource Kit) and use ==

ROBOCOPY source dest /E /COPY:ATSOU ==

This will set the permissions on the destination folder to be the same as the source folder. I hate to say I have yet to figure out how the permissions work while doing Rsync over SSH.

Boogerman said...

Hello. Thanks for this great article. About your authorizations problem, I solved it by using CYGWIN=nontsec on the SERVER side:

c:\cygwin\bin\cygrunsrv.exe -I "rsync" -p /cygdrive/c/cygwin/bin/rsync.exe -a "--daemon --no-detach" -f "rsync daemon service" -e CYGWIN=nontsec -u Administrator

With this parameters, the created files on the server side will simply inherit the ACLs of their containing folder.

I hope it helps. Best regards,

Kev said...

The CYGWIN=nontsec is no longer used in Cygwin.

The correct way to ensure that Cywin doesn muck around with permissions is to add the noacl option to your mount point options. For example in /etc/fstab we have:

# none /cygdrive cygdrive binary,posix=0,user 0 0

Uncomment this line and change to:

none /cygdrive cygdrive binary,posix=0,user,noacl 0 0

Once you've saved the file do a:

mount -a

Then check that this has taken effect by doing a mount, you should see something like:

C: on /cygdrive/c type ntfs (binary,noacl,posix=0,user,noumount,auto)

Hope that helps.