You're right about one thing... all things are not equal. You raised some good points along the way... see below for my annotations.
One thing to remember in regards to caching... when you request a block on disk, your local OS (with direct attach storage) will read that block and read ahead several more blocks. Those read-ahead blocks will be transferred via your fibre channel, even if they aren't used by your application. The filer on the other hand will do the same read-ahead, but it will only transmit the block that was requested. If the application asks for the next block, then both have that block in memory. Now, in a database environment where blocks are very randomly placed within a filesystem, this read-ahead via direct attach storage is going to kill the efficiency of the bandwidth to your storage, and that's your OS's fault. Whereas with a filer, you can turn off read-ahead within the filer to minimize the work it does.
This is an apples to oranges test.
Sure, streaming to/from RAID0 will always kick ass. However, who really runs RAID0 these days??? (I'm sure there's about <.1% of applications where RAID0 is suitable, because the data is not critical) RAID 0+1 might be a slightly better comparison since the filer doesn't have a RAID0 mode.
Ok, this is apples to apples.
Ok, given the environment you just described, you could enable a undocumented feature within the solaris mount_nfs command, 'llock'. This tells the NFS client that he shouldn't use NLM to do file locking, I'll just do it locally. This will essentially give you in-kernel locking. The caveat here is that you can't share the filesystem with other clients, but you can't do that with today's direct attach storage either. I usually recommend using the llock option in a Solaris/Oracle/filer environment.
Aaron
On Thu, 10 Aug 2000, Sims, Aaron wrote:
This is an apples to oranges test.
Yep, but it is interesting that even in such an unbalanced comparison, the Netapp still manages to fare quite admirably.
Sure, streaming to/from RAID0 will always kick ass. However, who really runs RAID0 these days???
I lied when I said this biller application was the only thing we ran on local disk. Our news servers also run on local disk. In that situation, we're not concerned enough about reliability to spend the premium on Netapps. We just wanted fast, cheap multi-terabyte storage.
Ok, given the environment you just described, you could enable a undocumented feature within the solaris mount_nfs command, 'llock'.
Ahhhhhh... *that's* what it is! I had a vague memory that this was possible under Solaris, but never could find a reference to it. I'm re-running the biller database benchmark now, and I am seeing about 16% *better* performance than on the Veritas RAID-0 filesystem. Sir, you may just have sold another filer with that. I'll be sure to get my sales rep to take you out to the most expensive restaurant in Toronto, should you ever come up this way. ;-)
If you do a man on "nfsstat" from your solaris box, you will see that, although "llock" is not "documented" in the normal fashion, sun continues to show it is "available" as a mount option on the nfsstat man page.
Look in the section referring to mount flags and you will find:
llock Local locking being used (no lock manager).
--tmac
Brian Tao wrote:
-- ******All New Numbers!!!****** ************* *************
Timothy A. McCarthy --> System Engineer, Eastern Region Network Appliance http://www.netapp.com 240-268-2034 Office \ / Page Me at: 240-268-2001 Fax / 888-971-4468
have
a RAID0 mode.
Of course, with RAID 0+1, you're paying for twice as many disks. For the same money, you could buy a second filer, and spread the load, and get better performance for the same dollars. (And a better MTBF).
Bruce
+-- "Bruce Sterling Woodcock" sirbruce@ix.netcom.com once said: | Of course, with RAID 0+1, you're paying for twice as many disks. | For the same money, you could buy a second filer, and spread the | load, and get better performance for the same dollars. (And a better | MTBF).
Say huh? For the same money? Even having to buy twice the disks, you'd still most likely end up cheaper than a filer. If this isn't the case for you, I want in on that discount.
There's no way you'd get two filers with X diskspace between them for anywhere near the price of some cheaper DAS with X diskspace, unless that DAS you're talking about is an EMC or something like that.
Oz
----- Original Message ----- From: "Ozzie Sabina" ors@cimedia.com To: "Bruce Sterling Woodcock" sirbruce@ix.netcom.com; "Sims, Aaron" Aaron.Sims@netapp.com; toasters@mathworks.com Sent: Monday, August 14, 2000 5:06 PM Subject: Re: Filer storage for databases, seriously? (Was: Re: NetApp ques tions)
Depends on the amount of disk you are talking. Note that I didn't say you could buy a second duplicate filer, just that you could get a second one. Do the math and you'll come out on top.
That's because the "some cheaper DAS with X diskspace" would not have compareable performance and RAID protection. If you cost out your DAS of comparable performance to a filer, but with twice the disk (for RAID 0+1), it will be more expensive.
Bruce
+-- "Bruce Sterling Woodcock" sirbruce@ix.netcom.com once said: | Depends on the amount of disk you are talking. Note that I didn't | say you could buy a second duplicate filer, just that you could get a | second one. Do the math and you'll come out on top.
I agree, certainly if you're talking about 2TB or something like that, the cost of the additional disk gets very high. | | That's because the "some cheaper DAS with X diskspace" would | not have compareable performance and RAID protection. If you | cost out your DAS of comparable performance to a filer, but with | twice the disk (for RAID 0+1), it will be more expensive.
I'll argue that you get just fine "RAID protection" from 0+1 on any reasonable array. Performance is another matter, but I didn't think we were trying to compare apples-to-apples there. Performance and features are why you'd spend the extra money on a Netapp in the first place.
Oz
In message 20000815000631.18668.qmail@trail.cimedia.comyou write:
..Depends a lot on who you're buying your disk from and what features you want, I think..
At my place of business, we were recently deciding between EMC and NetApp.
We wanted: Protected Data (RAID) Snapshots Off-line Backups Remote Mirroring
In EMC Land, where the only way to get more features is to buy more mirrors, we had: Copy 1 - The Original Data Copy 2 - A mirror of the data Copy 3 - A BCV of the data for rapid recovery Copy 4 - A BCV of the data for offline backups Copy 5 - A BCV for SRDF mirroring to a remote site
Copy 6 - A BCV for receiving SRDF from the primary site Copy 7 - The live data Copy 8 - A mirror of the data
That's 8 TBs of disk on the floor for 1 usable TB of storage.
Okay, we were also deciding between putting up another NT fileserver with ~200 gigs usable and an F740 with ~200gigs usable.
The head unit for the F740 was about $10k more than a Compaq DL380, however the disk with shelves was about $1,000 _cheaper_ than the dumb shelves Compaq was trying to sell us.
So it's a tad more expensive, but the extra features of the NetApp pushed it over the top.
-James
-
Brian Tao -
Bruce Sterling Woodcock -
James Neal - HandiCAT -
Ozzie Sabina -
Sims, Aaron -
Timothy A. McCarthy