Tom Yates says:
before i deliver myself of my opinions on upgradability, would anyone from netapp care to comment on whether that's a hardware or "supported configuration" limitation?
Ah -- a request for input! I assure you that all of us at NetApp watch discussions like this with great interest. :-)
I'm going to answer the question in general, since Chris didn't say what model he's got, and because I can't remember the details for all our models anyway.
First some "philosophy", and then the hardware issues.
Philosophy:
When we very first started, the "appliance" analogy drove us to strongly restrict configurations. Sun old-timers have a saying: "We sell rope." The implication being: "Don't come whining to us if you accidentally hang yourself."
We wanted to avoid accidental hangings. We argued that NetApp should simply disallow configurations that might burn people -- say because RAID reconstruct would take forever and the probability of a second drive failure gets high. I suspect that this position is very reasonable for some markets -- maybe branch-offices or low-end environments with little technical sophistication. But in sophisticated, technical, high-end environments things are different.
We have received the message loud-and-clear from our customers that they want the hard-coded capacity limits removed. That will definitely change, along with some new features that allow higher capacities. (Please bear with me as I delicately avoid any pre-announcements here.)
Hardware Issues
There are a variety of nasty hardware issues that one can encounter in large configurations, and since we've avoided -- up until now -- allowing "dangerous" configs, we haven't done as much testing in those areas as we might.
One issue is that EISA and PCI can both run into trouble with too many different cards operating at high bandwidths. A configuration that causes no problem at low load might get weird at very high loads. This is something that we'll need to invest more in as we remove hard-coded limits. The cabling length limits on SCSI are also annoying and can lead to trouble.
The way that some companies handle this is to specify a maximum "supported" configuration, that they have actually tested, but let customers know that there's nothing to stop them from doing their own experiments beyond this point. Other companies announce maximum "supported" configurations that are larger than anything they've actually tested. The first order becomes an instant beta site. I prefer the former approach to the later.
I'd rather not get real specific right at the moment, but I hope that I've said enough to give you a sense of our thinking moving forward.
Keep the feedback coming. We really do appreciate the input, even if it sometimes seems that it takes a long time to respond.
Dave