We are considering moving our F740 and F760 toasters from quad 100-Mbit ethernet EtherChannel'ed on a Cisco 5500 switch to the gigabit NICs on a new Cisco switch, a 6509. We're doing this 'cause:
- We need more ports. Period.
- The 6509 has a higher density of ports and gig ports.
- The gigabit NICs are supposedly much faster, and you can offload
the checksum processing from the main CPU to the gigabit card. (We're not even close to saturating the quad link now; I'm just talking latencies. Plus, we beat the crap out of our toasters; if the CPU can do less overhead and more data serving, I want it.)
We're running to F760s with gig ethernet into a Foundry TurboIron switch. In a write intensive environment, the gig cards have not proven to be useful. When doing streaming writes (Oracle hot backup to disk, ndmpcopy from one head to another, database load, etc) we peg the CPU of the F760 at 100% while only getting around 200mbs throughput. We could accomplish the same performance with quadethernet. Hopefully the 800series netapp will improve this write performance bottleneck. However, in a mixed read/write environment, the gig network connection might prove more useful (we haven't done any testing with streaming reads). If you're hoping to switch over to gig to decrease CPU load *and* if your usage is write intensive, don't expect a huge improvement.
Just my $.02.