Hello,
I need to replicate data between a FAS250 and non-NetApp storage - I can't warrant another Filer at the moment, so SnapMirror is out sadly.
The amount of data will be just under 1TB and will contain Maildirs, so the level of change will be quite high for about 300GB of that. The rest will be fairly static.
The connection used for replication will be 100Mbit/sec. I'd like to find a method that can get me as close to immediate replication as possible, however a lag of about 3 hours is probably acceptable for most of the content.
I've been looking at the file server extension to NDMPv4 and various third-party software, as well as things like rsync, but wondered if anyone else had any ideas of things I should be looking at?
Thanks in advance,
i use rsync regularly for this sort of action.
-- michael
On Mon, Feb 20, 2006 at 03:37:30PM +0000, Richard Seabrook wrote:
Hello,
I need to replicate data between a FAS250 and non-NetApp storage - I can't warrant another Filer at the moment, so SnapMirror is out sadly.
The amount of data will be just under 1TB and will contain Maildirs, so the level of change will be quite high for about 300GB of that. The rest will be fairly static.
The connection used for replication will be 100Mbit/sec. I'd like to find a method that can get me as close to immediate replication as possible, however a lag of about 3 hours is probably acceptable for most of the content.
I've been looking at the file server extension to NDMPv4 and various third-party software, as well as things like rsync, but wondered if anyone else had any ideas of things I should be looking at?
Thanks in advance,
Richard Seabrook Areti Internet Ltd., http://www.areti.net/
-- This message has been scanned for viruses and dangerous content by the Areti Internet Ltd email scanning service, and was found to be clean. http://www.areti.net/
Le 20/02/2006 15:37, Richard Seabrook a �crit:
Hello,
I need to replicate data between a FAS250 and non-NetApp storage - I can't warrant another Filer at the moment, so SnapMirror is out sadly.
If you are tight on your budget, get an old F760 with 4 FC9 shelves on ebay and use snapmirror. If you find the correct resseller you can get it around 7000$, sometimes with a basic warranty.
The connection used for replication will be 100Mbit/sec. I'd like to find a method that can get me as close to immediate replication as possible, however a lag of about 3 hours is probably acceptable for most of the content.
I've been looking at the file server extension to NDMPv4 and various third-party software, as well as things like rsync, but wondered if anyone else had any ideas of things I should be looking at?
Rsync is probably your best bet.
Regards, Max
It reall depends on the number of files and the size for rsync. On a decent box, I can run through 400 files in not even a second. Figure out your rate of change by doing some sample rsync's of X mail dirs and multiple it out.
100 mbit is kinda slow, gig-e at least between the rsync box and the filer might be obtained even cheaply by a crossover cable on a private ip network for just the rsync's.
--- Richard Seabrook richard.seabrook@areti.net wrote:
Hello,
I need to replicate data between a FAS250 and non-NetApp storage - I can't warrant another Filer at the moment, so SnapMirror is out sadly.
The amount of data will be just under 1TB and will contain Maildirs, so the level of change will be quite high for about 300GB of that. The rest will be fairly static.
The connection used for replication will be 100Mbit/sec. I'd like to find a method that can get me as close to immediate replication as possible, however a lag of about 3 hours is probably acceptable for most of the content.
I've been looking at the file server extension to NDMPv4 and various third-party software, as well as things like rsync, but wondered if anyone else had any ideas of things I should be looking at?
Thanks in advance,
Richard Seabrook Areti Internet Ltd., http://www.areti.net/
-- This message has been scanned for viruses and dangerous content by the Areti Internet Ltd email scanning service, and was found to be clean. http://www.areti.net/
__________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com
Richard> I need to replicate data between a FAS250 and non-NetApp Richard> storage - I can't warrant another Filer at the moment, so Richard> SnapMirror is out sadly.
Richard> The amount of data will be just under 1TB and will contain Richard> Maildirs, so the level of change will be quite high for about Richard> 300GB of that. The rest will be fairly static.
Doesn't sound like too bad a number at all.
Richard> The connection used for replication will be 100Mbit/sec. I'd Richard> like to find a method that can get me as close to immediate Richard> replication as possible, however a lag of about 3 hours is Richard> probably acceptable for most of the content.
How far apart with the two systems be? Next to each other in the rack, or across the country? It will make a difference due to the way TCP works with longer RTTs (Round Trip Times).
What you can probably get away with is just a simple 'rsync', but to get the speed, you'll want to run multiple copies in parallel, since the bottleneck is the time is takes to scan each and every file in the volume over the past three hours, and then compare it to the destination volume as well.
We're running rsync cross country one file systems with 1tb of data and 7 million files, and it can take 8 hours to run, along with a bunch of RAM on the servers to hold the rsync data.
Good luck, John
-
Herve Boulouis -
Jerry -
John Stoffel -
Max -
Michael Galloway -
Richard Seabrook