----- Original Message ----- From: "Todd C. Merrill" tmerrill@mathworks.com To: toasters@mathworks.com Sent: Tuesday, November 07, 2000 11:01 AM Subject: Re: 250K NFS ops/sec, eh? (warning...long and ranting)
What I was trying to emphasize were the "CPU" units that actually push data onto the network. For NetApp, these are filer heads; for EMC, these are datamovers.
The EMC Channel Directors also actually push data.
Actually, I didn't view "all results" at the SPEC page; I see NetApp has published some TCP numbers. Hoo-ray!
With that F760 config? I hadn't looked recently.
Let's compare F840 TCP NFS v3 (7,783 ops/s) with Celerra 507 DataMover 2 CPU (they don't list a single node) at 15,723 ops/s. Giving NetApp the benefit of the doubt of perfect linearity, a cluster of F840s would then be 15,556 ops/s. Damn, that's close. And, that's one main point I was trying to make.
Except it isn't close... the Celerra 507 probably has Symmetrix on the back end, with more CPUs, more disks, and probably lacking RAID. I'm just guessing here; I haven't looked at the configuration in question but that is typical for EMC.
Depending on discounts from each vendor (does anyone pay list price?!), you can imagine a situation in which the line for each vendor *crosses*, for some number of datamovers/filers and the same amount of disk, that is, the incremental cost in adding another pair of clustered F840s and some disk will become more expensive overall than just adding another datamover and the same amount of disk.
Yes, but when you calculate in the up front cost of the Celerra, plus the total cost of ownership, the Netapp solution will always be cheaper. Especially if you are comparing against a Celerra with actual RAID protection.
These "other" issues touch on the complexity of the architecture and the specific application, for us and for each customer. Performance is "about the same" so concentrate on the other issues. Another main point I was trying to make.
And the point is wrong. Performance is far superior in the Netapp case for the same $. If "other" issues overrride that, so be it.
Providing more configurations would allow the customer to find the configuration closest to what they need, and hence get an accurate indication of how well either box would do for them.
I agree, but providing every possible configuration is probably cost prohibitive. While altruism is a nice virtue, I think from a corporate perspective if configurations A and B are sufficient for most customers to make a sale, testing and publishing configuration C might not be cost-effective.
And, would it not require less work to grab a filer, install ONTAP, keep the defaults, and run the SPEC program, rather than tweaking it to juice every last op out of the box? (More work, sure, in running the SPEC program a *second* time...)
Right, you'd have to run it a second time. So, if from this point forward, you *always* run it with the default config, sure you save a little work, but presumably your reduce numbers can cost you sales when all the other vendors *are* tweaking. Having the best number *must* mean something, or else corporations wouldn't spend hard-earned dollars on it.
There is also a benefit for internal evaluation. You want to measure how much better a given software release is from a previous one, or a hardware platform from another, for your own evaluation purposes. If the new code has new features that are not turned off, then you will not have a fair 1:1 comparison to see if you *actually* managed to squeeze more performance out of your new software version. By using a stable baseline that doesn't change as new features are added, you can more accurately measure incremental improvements in performance.
EMC would certainly always quote the numbers that made NTAP look the worst,
And, NTAP has fired the last volley with the 16-node non-failover "cluster." All I'm saying is let's stop the dick waving to see whose is the biggest.
If NTAP and EMC stop, you don't think someone like SUN wouldn't step in? Sure, maybe you're savvy enough not to buy SUN if they do that, but many customers are not.
I think the best solution is to have future benchmarks provide ratios of ops/$ and ops/disk and so on. Of course, these numbers will not scale linearly, so I suppose there is a risk where each company will "push" the numbers of their low-end configuration more in order to make them look better, and this can also lead to unrealistic numbers. But at least the vendors can't just keep adding more and more servers to make themselves look better.
So, what did you finally decide with Mathworks?
Bruce