RSS2.0

Possible Homeserver & Backup Solutions

Sunday, April 20, 2008

When I built my latest computer last December, I had the intention of using it as a backup and home file server for other computers on my network, mainly my PowerBook. Once built, however, I installed Windows on it (because I don't have an Intel-based Mac) and began playing with different operating systems. The file server idea got delayed, and I'm just about ready to pick it up again. The problem is, I will periodically need to use Windows, which won't be able to access my ZFS pool from my server. I can't get around this until ZFS for Windows is available, so I'm thinking about the following solution (barring any compatibility conflicts).


Current hardware:
  • 1 500GB SATA HD
  • 1 150GB PATA HD
  • 1 100GB PATA HD
Partition Idea:
  • Use the 100GB HD as the OS drive, partitioned into the following:
  • 10GB Solaris Partition - (File-server OS - either Solaris Nevada or OpenSolaris Indiana)
  • 5-10GB *nix Partition  - (for Indiana or Ubuntu)
  • 10GB Windows Vista Partition
  • 20-25GB Test OS Partition
  • 45-50GB (aprox.) Windows XP Partition
  • Use the 150GB drive as Windows compatible storage (NTFS)
  • Use the 500GB SATA drive and additional to-be-purchased drives for ZFS storage pool
Backup Strategy:
I have a couple options when it comes to a potential backup strategy for my new fileserver, which I will explore below. Assume the backup sources are running either the HFS+ filesystem or NTFS filesystems, and the backup target is running ZFS.
  • rsync: Use rsync to make delta backups to the file-server (no incrementals or old data would be kept)
  • rsync + ZFS Snapshots: Use rsync to make delta backups to the file-server, and automatic ZFS snapshots to keep incrementals
  • rsyncsnapshot: Script uses rsync to make delta backups to the file-server, which keeps incrementals through rotated hard-links.
  • rdiff-backup: Makes delta backups to file-server, and keeps incrementals as diffs
  • ZFS send/receive: Use built in ZFS commands to send delta ZFS file-system snapshots from a ZFS source to either a ZFS target or an archive file (incrementals kept via snapshot on ZFS target; no incrementals kept in archive file?)
I would obviously like to take advantage of aspects of the ZFS filesystem, specifically snapshots, so this sways my choice a little bit, but here are some pros and cons for the differing solutions:
  • rsync: Pros (uses rsync engine, fast) Cons (no incrementals)
  • rsync + ZFS: Pros (uses rsync engine, fast, ZFS snapshots for incrementals) Cons (???)
  • rsyncsnapshot: Pros (uses rsync engine) Cons (slow?, hard-link incrementals?)
  • rdiff-backup: Pros (incrementals as diffs) Cons (slow, bad Mac meta-data support, must use rdiff-backup to browse/restore diffs)
  • ZFS Send/Receive: Pros (all ZFS) Cons (source filesystems not running ZFS)
So, I like the rsync engine. It performs very well, and has perfect Mac meta-data support. However, plain rsync won't work, since it doesn't keep incrementals. I still need to do some more testing with rsyncsnapshot, but my limited testing indicates it is dog slow. The most cons seem to be with rdiff-backup, and I can't use plain ZFS send/receive because my source to be backup is not yet running ZFS. 

This leaves me with one very promising option: rsync + ZFS. I get to use rsync's delta copies combined with ZFS's efficient snapshot feature. A seemingly win-win! Here's the only problem I can forsee: I have to find some sort of script to either automatically create a ZFS snapshot on the target upon completion of a rsync backup, or have the target automatically take snapshots at certain times (and name them after the time). Furthermore, I will need a script that can manage my snapshots automatically, keeping a given number of snapshots (such as 24 hourlies, 7 dailies, and 4 weeklies) and delete snapshots that exceed this. It is too complicated and time-consuming to manually manage the barrage of ZFS snapshots that would be created.

0 comments: