Richard> I need to replicate data between a FAS250 and non-NetApp Richard> storage - I can't warrant another Filer at the moment, so Richard> SnapMirror is out sadly.
Richard> The amount of data will be just under 1TB and will contain Richard> Maildirs, so the level of change will be quite high for about Richard> 300GB of that. The rest will be fairly static.
Doesn't sound like too bad a number at all.
Richard> The connection used for replication will be 100Mbit/sec. I'd Richard> like to find a method that can get me as close to immediate Richard> replication as possible, however a lag of about 3 hours is Richard> probably acceptable for most of the content.
How far apart with the two systems be? Next to each other in the rack, or across the country? It will make a difference due to the way TCP works with longer RTTs (Round Trip Times).
What you can probably get away with is just a simple 'rsync', but to get the speed, you'll want to run multiple copies in parallel, since the bottleneck is the time is takes to scan each and every file in the volume over the past three hours, and then compare it to the destination volume as well.
We're running rsync cross country one file systems with 1tb of data and 7 million files, and it can take 8 hours to run, along with a bunch of RAM on the servers to hold the rsync data.
Good luck, John