RE: SUMMARY: A-SIS (NetApp De-Duplication)

-- Michael W. Sphar - IS&T - Lead Systems Administrator SMBU Engineering Support Services, BMC Software -----Original Message----- From: owner-toasters@mathworks.com [mailto:owner-toasters@mathworks.com] On Behalf Of Peter W. Osel Sent: Wednesday, August 01, 2007 10:24 AM To: toasters@mathworks.com Subject: SUMMARY: A-SIS (NetApp De-Duplication) Hello, here my summary on A-SIS, based on all the responses I received. * at this point in time you need both, the nearstore license as well as the a_sis license. * installing the nearstore license enables the nearstore personality, biggest impact is that the number of allowed parallel replication streams is increased (e.g. for an FAS30xx from 8 to 64). * performance impact is negligible as long as you do not use the additional data streams (some scheduling magic might be necessary). * while A-SIS seems to be fairly low overhead, a maxed out filer is probably not the best system to use for A-SIS ;-) * for data volumes with mostly read access, A-SIS will add less overhead than for data volumes where data constantly changes. * when using Volume SnapMirror (VSM) to replicate a de-duped volume, the update traffic does benefit from the de-duplication (you might want to coordinate when A-SIS de-dupes the volume and when SnapMirror transfers it ;-). It is recommended to have a-sis licenses on the source and the destination filer, e.g. in case you have to reverse the SnapMirror relationship. * you can also qtree SnapMirror form a non-de-duped volume and just de-duplicate the target volume (obviously source volume and transfer volume will not be reduced in this case) * The Data ONTAP 7.2 Data Protection Online Backup and Recovery Guide has a chapter on A-SIS. http://now.netapp.com/NOW/knowledge/docs/ontap/rel722/pdfs/ontap/online bk.pdf (now account needed) * There is an excellent A-SIS De-duplication FAQ, and a Technical Report (TR-3505) in the making (not yet available on the public library). Ask your NetApp SE, they can also help with estimating the savings that A-SIS would achieve on your data volumes. Work with your NetApp SE, they have access to more information and tools that really helps to plan your A-SIS deployment. I will definitely give A-SIS a try, and can hopefully share some experiences later this year. A big thank you to all who responded: Bill Holland, Larry Freeman, M. Vaughn Stewart, Michael Barrow, Reinoud Reynders, and Sels Roger. Cheers --pwo For reference, here my original posting: > *** Peter W. Osel pwo@qimonda.com > *** [2007-07-27 18:00:52]: > Hello *, > > we are in the process of consolidating several NetApp filers and > NearStore systems onto a single FAS30x0HA system. The system's main > function is to serve primary data (part from FCAL disks, part from SATA > disks), it's secondary function is to act as the NearStore system for a > second site, storing the copy of the other site's data on SATA disks. > > Now we would like to try A-SIS to reduce the required physical > storage. > > It looks as if in order to add A-SIS (a_sis license), I also need to > install the NearStore license. Now my concern is, that by giving the > system a NearStore personality I might impact the system's primary role > - serving primary storage data to users. > > Has anyone experience with how a system changes when the NearStore > licenses is enabled? Any negative impact for serving data to users? number of allowed parallel data streams increases. impact negligible as long as you limit the number of parallel data streams. > Is the NearStore personality activated by installing the nearstore > license, or could we install the nearstore and a_sis license, but keep > the nearstore personality deactivated? yes, installing the nearstore license activates the nearstore personality, biggest change: number of parallel data streams is increased. > For data that we volume SnapMirror, would we need an A-SIS license on > both sides - source and target system? strictly needed only for source, recommended is source and target, e.g to be prepared when you want to revert the direction of the SnapMirror. > Will the volume SnapMirror transfer less data (assuming that A-SIS > de-dupe of data saved some of the required data blocks)? > yes?!) definitely yes. > Any other experience (good or bad) or concerns when using A-SIS? > > Any reason to not use it on all (home directory type data) volumes? A-SIS makes more sense for volumes where not all data changes all the time, so if you have a volume that is very write intensive, with all data changing all the time, it might be a bad candidate for any de-duplication technology. > Any limitation encountered? right now (Data ONTAP 7.2.x) there are limits on how big a volume can be, depending on the platform. Ask your SE for current limitations. They might change or be completely lifted in a future version of ONTAP. > Any experience how much savings can be achieved (e.g. for home > directories volumes, volume storing tons of application software > packages)? Heavily depends on the data, 20% to 85% have been seen for regular data volumes, even more for highly redundant volumes (e.g. when storing weekly full backup images you might - over time - see 20:1 reduction). Your mileage may vary! Try it out yourself. > > > Cheers > --pwo > > -- > Peter W. Osel eMail: pwo@Qimonda.COM > Principal Development Systems Phone: +1.919.677.6333 > Qimonda North America Corp. Cell : +1.408.348.6735 > 3000 CentreGreen Way eFax : +1.804.952.3905 > Cary, NC 27513-5760, USA www : http://pwo.de/ >