Hi Bryan !
Please take into account these issues, which favor the appliance solution over the Linux box:
1. Proven stability of filesystem - NetApp's WAFL has real-life proven stability, which is consistent thru shutdown and powerdowns - even though ext3 is using journaling, WAFL's Consistency Points using the NVRAM should cover stability much better (and faster). 2. Crash recovery - filer's fsck (wack) is much much faster than other vendors' solutions - ~600GB/hr and more. 3. Ease of use - adding capacity is that easy - no extra config. Also - you're wasting time on choosing the right CPU arch. and motherboard bus chipsets - leave that to NetApp's engineering experts, and you can save time (and money, and get anyway better performance). 4. Extra NetApp-specific "bonus-features" - snapshot can lower your restore needs dramatically, and can help for some recovery cases. 5. Enterprise-level features which you can leverage in the future (which Linux doesn't have YET, not that I've seen, others - correct me if I'm wrong) - SnapMirror/SnapRestore, which complement the disaster-recovery issues. 6. Future growth - one filer can grow (now) to 1.4TB, which I don't think many Linux storage solution go that high, and NetApp will upgrade to 3TB and higher for one filer very soon. 7. Performance - CIFS performance should be much higher, NFS also much higher. 8. NFS/CIFS securuty - Linux can only share to PCs with Samba, which is fine, but only a user-level application (slower that Data ONTAP's kernel level implementation), and also the filesystem on Linux will still be Unix-based, and will not be able to maintain NT ACLs etc.
I hope it sums it up somehow, Eyal.
--- Bryan -TheBS- Smith thebs@theseus.com wrote:
Linux-NFS list recommended I post here -- RE: NetApp versus Linux
All --
The Linux-NFS list recommended I post here. I am looking at either a $35K NetApp F720 or a $18K Linux (w/Ext3+NFS3 patches) solution. I'd greatly appreciate any input anyone has.
My boss favors the NetApp solution on paper (never used one himself, nor does he do any admin-level stuff). But IMHO, unless we spend $70K to buy two (2) NetApp F720s and take advantage of the NetApp's superior failover/load-balancing capability, I'm not sure it's the right move. We still have to have a Linux server around for CVS (and NIS?) anyway, which would put a NetApp at the mercy of the Linux box anyway (any way to get CVS on NetApp?).
I break down my limited knowledge below. I currently have a 100GB Linux 2.2 server (Ext2+NFS2) that I didn't setup (but have heavily reconfigured because the consulting firm didn't know jack crap, but some things like using 1KB block sizes in the /home filesystem that take forever to fsck is something I can't change right now).
We're looking at going to 250GB (possibly 500-1000GB in the next 6 months). Most of our clients are Solaris 2.6 (about a dozen serious production boxes with dual to 6-way processor, 1 to 6GB of RAM, etc...). We are starting to add more and Linux workstations (both dedicated and dual-booting with NT).
Stability:
+++ NetApp (F720) Boots in under 2 minutes due to the use of NVRAM even after an improper shutdown. Buying two (2) units is an extremely viable failover/load-balancing solution. [ Unfortunately, my boss doesn't want to spend more than $30K).
--- Linux (Ext2) Although our Linux systems have averaged >100 days uptime, a routine power outtage without proper shutdown, unable to umount NFS clients, etc... causes an fsck. We figure 30 minutes per 100GB for Ext2 fscks on 8-16KB blocks (currently takes 90 minutes of 1KB blocks, something the consultants before me did -- dumb!). Never had a problem with lost data though.
--- Linux (ReiserFS) Meta-data journaling, but has NFS locking issues (except for a SuSE patch to the old knfsd?). Eliminated from consideration
??? Linux (Ext3) Full data journaling means sync I/O??? What about current stability (I'm using Trond+Higgen 0.21.3 on non-production systems with Solaris 2.6 clients and I'm happy)? Locking with Ext3 (never used Ext3, but many others are now)? I know VA Linux now has Linux 2.2 kernels with Ext3 and NFS3 that are supposively quite stable.
Performance:
??? NetApp (F720 + 1000Base-SX NIC) While the NVRAM in the NetApp allows almost instant sync acknoledgement to clients, the F720 only comes with 1/4th the NVRAM of the F740/760 (8MB versus 32MB). The result is that the F720 can only handle 1/3-1/4th the NFS requests and throughput of the F740/760. I have seen good marks on the later, but cannot find much on the former which makes me wonder if we will run into a performance issue? Transmeta also mentioned that they have had performance issues with the 700-series on high-loads because the on-baord CPU, a single Alpha (usually ~600MHz) cannot even keep up with the number of XORs for just the RAID-4 volume. Of course I only have a dozen NFS clients and Transmeta has many more. Hard drives are of older, 36GB, 7200rpm, Ultra2 (aka Ultra80) SCSI type.
+++ Linux (dual-cpu/memory ServerSet IIIHE chipset + dual-733MHz Pentium III + 233MHz StrongARM-powered, 4-channel, Ultra160 RAID controller on its own PCI 64-bit channel + 1000Base-SX NIC card on it's own 66MHz PCI 64-bit channel) These ServerSet III-powered mainboards seem to best both the older 4-way i450HX EDO mainboards, and the 2-way i840 in just about every memory benchmark. Going to use 2GB of RAM (which is 8x as much as in the F720), and can expand to 8GB later. The 3-independent PCI channels (of which, 2 are 66MHz PCI 64-bit) allows me to put the disk controller and network controller on their own, 64-bit channels. They Mylex 233MHz StrongARM-powered RAID controllers are the fastest you can get for just about anything (including Linux, especially when compared to vendors who still use a 100MHz i960 ASIC) and I'm going to pair it with 73GB, 10000rpm, Ultra160 disks with two disks per channel spread over all 4 channels. On the software side, I've had no issues running async with Linux. I'm more worried about Ext3's full-data journaling being the slouch here. Also wondering how long it takes, exactly, to recover after an improper shutdown with full-data journaling Ext3 (I'm assuming <5 minutes for 250GB?).
[ Further insight wanted: Just wondering if the F720 will be a slouch compared to this? Or will the NVRAM really kick Ext3's butt, even if there is only 8MB of it? ]
Other Considerations:
NetApp (Failover Option) Features "ready-to-go" fail-over option via proprietary gigabit links between units. Another unit can even use the disk controller and disks on a failed unit CPU unit. Very nice. Also lowers the admin requirements (like having to keep kernel/apps up-to-date, etc...). Someone on the Linux-NFS list actually mentioned administration wasn't so easy with NetApp (no web-based admin???) and would rather have Linux (I personally couldn't believe this, but she was a seasoned Linux admin like myself).
Linux (Software Support) I still need a Linux box to server as the CVS server, Mail server (including various Sendmail/Mail-based programs like AV scanner, HP OpenMail, Majordomo, GNATS, etc...), NIS server (or does NetApp have NIS server capabilities?), etc... This makes me wonder if I really will "save on TOC" with the NetApp since my Linux admin outside of this has only been about 40-80 hours over the past year (compiling new kernels, installing new NFS, etc...).
Cost:
--- NetApp F720 (7 x 36GB disk = 252GB, 216GB usable RAID-4, 180GB if using hot spare) $35K for the configuration (F720 = 600MHz? Alpha, 256MB RAM, 8MB NVRAM) with both SMB and NFS plus the 1000Base-SX upgrade.
+++ Linux (5 x 73GB disk = 365GB, 292GB usable RAID-5, 219GB if using hot spare) $18K for the configuration as above (dual-733MHz, 2GB RAM, RAID controller, 1000Base-SX NIC, etc...) with SAF-TE rackmount disk chassis, ATX rackmount chassis, cabling and redundant power on both the SAF-TE and ATX case.
Thanx in advance ...
-- Bryan "TheBS" Smith "Lead Computer Geek" Theseus Logic, Inc.
-- Bryan "TheBS" Smith CONTACT INFO
Chat: thebs413 @ AOL/MSN/Yahoo (see http://Everybuddy.com) Email: mailto:thebs@theseus.com,b.j.smith@ieee.org Home: http://www.SmithConcepts.com
===== Yours, Eyal Traitel eTraitel@yahoo.com, Home: 972-3-5290415 (Tel Aviv) *** eTraitel - it's the new eBuzzword ! ***
__________________________________________________ Do You Yahoo!? Get Yahoo! Mail � Free email you can access from anywhere! http://mail.yahoo.com/