RSS2.0

Rsync & Solaris Problems

Saturday, May 24, 2008

In my last blog, I posted some rsync test results from the BackupBouncer Mac metadata test suite, running against both a Mac-based ZFS file system and a ZFS-based Solaris server. Today, I'm going to discuss both my testing methodology and some pretty big problems I've ran into with rsync on Solaris. First off, I've compiled rsync 3.0.3 (pre2) with the fileflags and crtimes patches on both my Mac and Solaris. On Solaris, the crtimes patch wouldn't compile, so it was left out. With the same version of rsync running on both systems, I began my tests.


According to Mike Bombich (Mac backup expert and developer of CarbonCopyCloner) it is necessary to use a few specific flags to successfully backup Mac metadata: -X for extended attributes; -A for ACLs; -H for hard-links; and --fileflags for Mac file flags. Additionally, Bombich threw in a few additional flags in his recommended script: -N (unknown); --protect-args; --force-change; -a (archive mode); and -x ("don't cross filesystem boundaries". I'm not sure the utility of some of these commands, but I will include them because Bombich suggests it. The surprising result: numourous flags seem incompatable with the Solaris-compiled rsync. 

The results (and rsync error message) are below:
-X won’t work (xtended attributes are not supported on this server)
-H will work
-A won’t work (recv_acl_access: value out of range: 8000)
-x will work
--fileflags won’t work (on remote machine: --fileflags: unknown option)
--protect-args works
--force-change won’t work (on remote machine: --force-change: unknown option)
-N (??) won’t work (on remote machine: -svnlHogDtNprxze.iL: unknown option)

What does this mean? Well, it seems the flags necessary to make rsync backup Mac data properly (specifically -X, --fileflags, & -A) are not supported on Solaris. Thus, when running rsync without these flags, rsync doesn't backup Mac metadata and BackupBouncer reports its very poor score. This leaves the utility of a Solaris-based NAS for a Mac backup server is serious question.

I then ran a rsync --version on both the Mac and Solaris versions of rsync 3.0.3 (pre2). Here are the capabilities reported, which seem to echo the above results. 

Mac: rsync version 3.0.3pre2 protocol version 30
Capabilities:
64-bit files, 32-bit inums, 32-bit timestamps, 64-bit long ints,
socketpairs, hardlinks, symlinks, IPv6, batchfiles, inplace,
append, ACLs, xattrs, iconv, symtimes, file-flags


Solaris: rsync version 3.0.3pre2 protocol version 30
Capabilities:
64-bit files, 64-bit inums, 32-bit timestamps, 64-bit long ints,
socketpairs, hardlinks, symlinks, no IPv6, batchfiles, inplace,
append, ACLs, no xattrs, iconv, no symtimes, no file-flags


Unfortunately, the necessary flags are seemingly just not supported in Solaris rsync. However, this doesn't explain the results I received when running Mac rsync to a SMB share of the Solaris server (never calling the incompatible Solaris rsync). These numbers are almost as poor as running through the Solaris rsync. Me 

I will note, however, that the -A (ACL) flag may actually be supported by rsync, but reports a "out of range" error message. I'm going to research what might be going on here, as the Mac ACL numbers may be incompatible with Solaris (thus a Solaris issue and not an rsync issue).

Finally, I'd like to comment on the ZFS filesystem in general. It seems ZFS, at least on the Mac, has relatively good support for Mac metadata. I really have to applaud both the Sun ZFS and Apple ZFS teams here. According to BackupBouncer, old historical Mac-only data, such as resource-forks and Finder locks are some of the few things that fail the Mac-based ZFS test; and these things arn't typically used anymore on OS X. Furthermore, Mac ZFS fails the "devices," "xattrs + rsrc forks" and "lots of metadata" tests.

Mac Metadata, ZFS, and Solaris

Ok, this does not look so good! If you've been following my blog, I've begun a project to build and explore a Solaris-based ZFS NAS server to serve as both a file server and backup server for my Mac network. After exploring various backup solutions (in a previous post), I finally decided on rsync 3 to perform my backups due to its perfect support of Mac metadata as tested through the BackupBouncer test suite. You can view the BackupBouncer results for rsync, rdiff-backup, ditto, and a few other apps. to a Mac OS X 10.5-based host and a HFS+ filesystem in a previous post. This time, however, I decided to test rsync 3 to both an OS X-based ZFS filesystem (currently in beta) as well as to my Solaris ZFS test server via both SSH and a mounted SMB share. The results are interesting, and understandably a little concerning!


Rsync 3.0.3 pre2 & Mac-Based ZFS; Direct Local Copy, Full Args:
Verifying: basic-permissions ... ok
Verifying: timestamps ...
Sub-test: modification time ... ok
ok
Verifying: symlinks ... ok
Verifying: symlink-ownership ... ok
Verifying: hardlinks ... ok
Verifying: resource-forks ... grep: some-file/rsrc: Not a directory
FAIL
Verifying: finder-flags ... ok
Verifying: finder-locks ... FAIL
Verifying: creation-date ... FAIL
Verifying: bsd-flags ... ok
Verifying: extended-attrs ...
Sub-test: on files ... ok
Sub-test: on directories ... ok
Sub-test: on symlinks ... ok
ok
Verifying: access-control-lists ...
Sub-test: on files ... ok
Sub-test: on dirs ... ok
ok
Verifying: fifo ... ok
Verifying: devices ... FAIL
Verifying: combo-tests ...
Sub-test: xattrs + rsrc forks ... FAIL
Sub-test: lots of metadata ... FAIL
FAIL

Rsync 3.0.3 pre2 & Mac-Based ZFS using SSH and Full Args:
Verifying: basic-permissions ... ok
Verifying: timestamps ...
Sub-test: modification time ... ok
ok
Verifying: symlinks ... ok
Verifying: symlink-ownership ... ok
Verifying: hardlinks ... ok
Verifying: resource-forks ... grep: some-file/rsrc: Not a directory
FAIL

Verifying: finder-flags ... ok
Verifying: finder-locks ... FAIL
Verifying: creation-date ... FAIL
Verifying: bsd-flags ... ok
Verifying: extended-attrs ...
Sub-test: on files ... FAIL
Sub-test: on directories ... ok
Sub-test: on symlinks ... ok
FAIL
Verifying: access-control-lists ...
Sub-test: on files ... ok
Sub-test: on dirs ... ok
ok
Verifying: fifo ... ok
Verifying: devices ... FAIL
Verifying: combo-tests ...
Sub-test: xattrs + rsrc forks ... FAIL
Sub-test: lots of metadata ... FAIL
FAIL

Rsync 3.0.3 pre2 & Mac-Based ZFS using SSH and Limited Args (Solaris/rsync compatible args):
Verifying: basic-permissions ... ok
Verifying: timestamps ...
Sub-test: modification time ... ok
ok
Verifying: symlinks ... ok
Verifying: symlink-ownership ... ok
Verifying: hardlinks ... ok
Verifying: resource-forks ... grep: some-file/rsrc: Not a directory
FAIL

Verifying: finder-flags ... FAIL
Verifying: finder-locks ... FAIL
Verifying: creation-date ... FAIL
Verifying: bsd-flags ... ok
Verifying: extended-attrs ...
Sub-test: on files ... FAIL
Sub-test: on directories ... FAIL
Sub-test: on symlinks ... FAIL
FAIL
Verifying: access-control-lists ...
Sub-test: on files ... FAIL
Sub-test: on dirs ... FAIL
FAIL
Verifying: fifo ... ok
Verifying: devices ... FAIL
Verifying: combo-tests ...
Sub-test: xattrs + rsrc forks ... FAIL
Sub-test: lots of metadata ... FAIL
FAIL

Rsync 3.0.3 pre2 & Solaris ZFS via Direct Copy to Mounted SMB Share, Full Args:
Verifying: basic-permissions ... FAIL
Verifying: timestamps ...
stat: ./some-file: stat: No such file or directory

Sub-test: modification time ... FAIL
FAIL
Verifying: symlinks ... ok
Verifying: symlink-ownership ... FAIL
Verifying: hardlinks ... stat: link1: stat: No such file or directory
FAIL

Verifying: resource-forks ... grep: some-file/rsrc: Not a directory
FAIL

Verifying: finder-flags ... ok
Verifying: finder-locks ... FAIL
Verifying: creation-date ... ok
Verifying: bsd-flags ... ok
Verifying: extended-attrs ...
Sub-test: on files ... ok
Sub-test: on directories ... ok
Sub-test: on symlinks ... ok
ok
Verifying: access-control-lists ...
Sub-test: on files ... FAIL
Sub-test: on dirs ... FAIL
FAIL
Verifying: fifo ... FAIL
Verifying: devices ... FAIL
Verifying: combo-tests ...
cmp: EOF on /Volumes/Src/99-combo-tests/xattr-with-rfork/..namedfork/rsrc

Sub-test: xattrs + rsrc forks ... FAIL
cmp: EOF on /Volumes/Src/99-combo-tests/many-metadata/..namedfork/rsrc
Sub-test: lots of metadata ... FAIL
FAIL

Rsync 3.0.3 pre2 & Solaris ZFS via SSH; Limited Solaris/rsync Compatible Args:
Verifying: basic-permissions ... FAIL
Verifying: timestamps ...
Sub-test: modification time ... ok
ok
Verifying: symlinks ... FAIL
Verifying: symlink-ownership ... FAIL
Verifying: hardlinks ... FAIL
Verifying: resource-forks ... grep: some-file/rsrc: Not a directory
FAIL

Verifying: finder-flags ... FAIL
Verifying: finder-locks ... FAIL
Verifying: creation-date ... FAIL
Verifying: bsd-flags ... ok
Verifying: extended-attrs ...
Sub-test: on files ... FAIL
Sub-test: on directories ... FAIL
Sub-test: on symlinks ... FAIL
FAIL
Verifying: access-control-lists ...
Sub-test: on files ... FAIL
Sub-test: on dirs ... FAIL
FAIL
Verifying: fifo ... FAIL
Verifying: devices ... FAIL
Verifying: combo-tests ...
Sub-test: xattrs + rsrc forks ... FAIL
Sub-test: lots of metadata ... FAIL
FAIL

I'll explain the different rsync commands and arguments I've used for these tests in my next post, as I explain some rsync and metadata problems I've been experiencing on Solaris.