Hello Toasters.
I've done some testing recently with using NetApp shelfs (FC8/FC9) attached directly to a Linux box. It's worked very well, and I'm happy with the setup. There are some rough points, but the implimentation is pretty straight forward. I had started writting an article on how to do things end-to-end, but somehow half way thru just didn't see the point. Anyway, here are some interesting tidbits to wet the lips of anyone with shelfs sitting unused.
Single FC8 shelf with 18G disks: [root@nexus /]# df -h Filesystem Size Used Avail Use% Mounted on /dev/evms/fc8_volume 118G 15M 118G 1% /filer_shelf [benr@nexus benr]$ bonnie++ -d /filer_shelf/
Version 1.03 ------Sequential Output------ --Sequential Input- --Random- -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP nexus.homeste 1256M 5604 81 19477 18 15546 17 6861 98 59964 37 677.7 5 ------Sequential Create------ --------Random Create-------- -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 16 998 7 +++++ +++ 376 2 422 5 +++++ +++ 401 3
And this is 2 FC9 shelfs with 36G disks: Filesystem Size Used Avail Use% Mounted on /dev/evms/fc9_vol 474G 60M 474G 1% /filer_shelf
[benr@nexus benr]$ bonnie++ -d /filer_shelf/ Version 1.03 ------Sequential Output------ --Sequential Input- --Random- -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP nexus.homeste 1256M 5636 82 33744 33 23145 28 6306 93 60607 37 712.0 7 ------Sequential Create------ --------Random Create-------- -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 16 775 5 +++++ +++ 345 2 326 4 +++++ +++ 358 3
In both cases, the volumes that were build were striped across all avalible disks, using a stwidth of 16k. LVM was used under EVMS control to all volumes.
As you can see from the benchmarks, the block read performance was excellent. For the tests I used my standard Linux workstation, a very slow Dell PIII 550Mhz with 512M RAM. JFS was used as the filesystem.
The following are some notes for anyone who might wanna try this: a) DO NOT USE THE KERNELS QLOGIC DRIVER. It sucks! And don't think that you can just push the dev count to 255 and get past it, that only works for 3-4 devices. If you use a QLA2100 grab the QLA2200 driver from Qlogic.com, hack the makefile to build you a QLA2100.o module, the code is there but they don't advertise it. b) I tests all this with a custom 2.4.20 kernel, works great, added EVMS patches. c) EVMS rox, much more fun than LVM raw, but takes much longer to set things up due to the abstration layers added d) You'll have to wipe out the 2 partitions on the disk, which fdisk can't do. You'll need to dd over the first 1k to manually wipe the partitioning scheme. e) If your not already using DevFS I highly suggest it before piling shelfs on.
If anyone wants more information or benchmarks, mail me directly and I can send my notes over.
benr.