I have done Snapshots with EVMS (Enterprise Volume Management Software) under LINUX. It does basic snapshots and only has a command line interface (you do your own crontab). It is slower then NetApp snapshots (even more so the more snapshots you get), but it does work. You still have to figure out how to quiesce your filesystem during the snap (umount/mount works, but can be a challenge).
The only thing close to SnapMirror I have found is to do RAID-1 via iSCSI or FCIP. Big investment in switches, though. Just have one local disk and more remote disk striped together under the "md" driver.
The RAID striping isn't done in the shelf, the enclosure has a SES/CEMI card that gives the Filer information on temperatures, voltages, what power supplies are working, what the health of the local drive bus is, etc. This is either a SCSI target or a Fibre Channel port/WWN that the Filer talks to. The shelves are just JBODs and you can hook them up to any host you want and use them. It's just that when you hook them into the filer, the filer expects certain behaviors.
You are on the right track on the IDE (I prefer the term ATA) drives, but the following issues present themselves:
1. LINUX has a 2TB file system limit. Solaris has a 1TB limit. NetApp has something like a 6TB limit. 96Tbytes is the most number of disk shelves (24) that can be hooked into that filer. I would much rather have 16 file systems with NetApp than 48 file systems with LINUX.
2. Getting LOTS of drives working all at once is a real challenge and starts to cost $$$ for more hardware (enclosures, SCSI/Fibre Channel interfaces, etc). The 7TB R200 is high on a $/MB basis ($.01 per MB), but a fully loaded R200 LISTS for half that per MB ($.006 per MB) and any reseller will give you a discount (maybe 20%) off that. The old R100/R150 have been discontinued (real pain to get those SCSI drive shelfs installed in the field). The R200 is easier to install/manage than the old R1x0. Plus MAXTOR doesn't make the old drive shelves anymore.
I'm not saying that NetApp is always better, but they have their place in the overall scheme. Small NAS boxes are getting really good and NetApp is responding to the market that they choose to, which I would characterize as the "more fully featured solution" that has a higher return to NetApp.
You might also want to look at http://www.openfiler.org. I have tried it and it has promise, but it is still a challenge. I still stick with RedHat 8 (yes, 8, not 9, cause 8 doesn't ever crash/hang) and just add in SAMBA 3.x for a really cheap NAS box.
-----Original Message----- From: owner-toasters@mathworks.com on behalf of Joe Schmoe Sent: Mon 6/14/2004 12:26 PM To: toasters@mathworks.com Subject: Piecing together a netapp on the cheap - PART II
Thanks for your help so far. Let me give some more details and answer some of the points you all have raised.
First, I am looking for cheap and large - and am willing to sacrifice speed to get it.
Second, I am very intrigued by SnapMirror (replication over WAN) and SnapShot copies (point in time backups). The reason I don't just build my own little unix fileservers with linux or FreeBSD is that I don't see how GNU tools and scripting can replicate the functionality of snapmirror and snapshot copies. Maybe I'm wrong ?
So, I see that my initial plan is ill advised - you have pointed out that the disk shelves that NetApp ships are not just JABODs, but have intelligence in them - I did not know that the RAID striping was done in the shelf - I thought all RAID and logic was housed in the head unit and it just accessed big stacks of disks.
So at this point, my questions are as follows:
1. Is there any reasonable implementation of snapmirror / snapshot copies in free UNIX OSs ? That is, can I get netapp like features out of a normal NFS fileserver ?
2. When I see netapp quote a certain capacity for its filers, like 96TB for the NearStore R200, does that simply mean that they can get to 96TB using the largest currently available IDE drives, and that once larger drives are available that number will naturally go up ? OR does it mean there is some logical limitation in place on the unit that will keep it from going above 96TB ?
3. What is the absolute cheapest IDE based NAS that netapp has put out ? Is it the NearStore R100 ? What if I bought the head unit and shelves from netapp (or netapp branded from ebay), but used my own 400gig hitachi IDE drives ? - I see a 7TB R100 on ebay right now for $78,500 - which is completely absurd. My 4TB freeBSD fileserver at home cost about $5k ...
Again, many thanks.
--------------------------------- Do you Yahoo!? Friends. Fun. Try the all-new Yahoo! Messenger