toasters December 2002

toasters@lists.teaparty.net

80 participants
87 discussions

Defragmentation
by Edward Hibbert 20 Dec '02

20 Dec '02

I think it's general knowledge that once a NetApp gets above 90% full, performance starts to degrade quite noticeably. I've been testing an application recently and observed that this is the case on ONTAP 5.2: * When ~95% full, my performance test came up with a number of ~4 seconds. * Freeing up space to get to around 60% full, the number was ~3.5 seconds. * Copying off my data files and copying them back to defragment them, the number was 2.8s. The only thing I've been able to find about fragmentation is: * Some statements that WAFL tries hard to avoid fragmentation (which I'm sure it does) but can't always (which I'm sure of too). * Copying off data and copying it back again is the only way to resolve it. The latter solution doesn't seem practical if the data in question is supposed to be available 24*7. Some applications have a method of doing this at an application level, but does anyone know whether there are methods/future plans for background non-intrusive defragmentation at a filesystem level? Or is it just me who thinks this would be a good idea? Edward Hibbert Internet Applications Group Data Connection Ltd Tel: +44 131 662 1212 Fax: +44 131 662 1345 Email: eh(a)dataconnection.com Web: http://www.dataconnection.com

4 4

RE: Jukeboxes for filers
by Ed Sanborn 19 Dec '02

19 Dec '02

Hi folks, I too am using QuickRestore and backing up my F720 via a SpectraLogic Ait-2 Bullfrog tape library direct-attached via HVD SCSI to the filer. I am see'ing about 16GB's transfer rate over SCSI. However, all for all of my other clients (PC's, Sun's, etc.) I am see'ing only about 1.7 GB transfer rate. All of these other clients are being backed up over a 100Mb switched network. 2 of my clients take about 6+ hours to backup as they have about 13GB of data on each of them. I was contemplating purchasing local tape drives for these 2 clients however more tape libraries equals more tape changing headaches. Have folks found that migrating clients like these to GigE REALLY does take care of the I/O issue? It certainly makes sense to me but I want to be sure before purchasing GiGE NIC cards. Thanks, Ed -----Original Message----- From: Quentin Fennessy [mailto:quentin.fennessy@amd.com] Sent: Thursday, December 19, 2002 1:58 PM To: Ed Sanborn Cc: Geoff Hardin; Barry Lustig; toasters(a)mathworks.com Subject: Re: Jukeboxes for filers >>>>> Ed Sanborn writes: > That's 16MB/sec over GigE? Yes. > -----Original Message----- > From: Quentin Fennessy [mailto:quentin.fennessy@amd.com] > Sent: Thursday, December 19, 2002 1:02 PM > To: Geoff Hardin > Cc: Barry Lustig; toasters(a)mathworks.com > Subject: Re: Jukeboxes for filers >>>>> Geoff Hardin writes: >> Barry, >> We use an ATL P3000 library with 16 DLT7000 drives. We use >> Workstation Solutions QuickRestore (now Reliaty Backup) to back >> everything up. We have 15 filers being backed up and the library is >> starting to get overwhelmed. We are considering the new SDLT drives, >> since they are backwards read compatible with DLT. It's been a great >> solution for over two years now; only change I would have made would >> have been to wait for the P3000 GbE library. > We've installed a P14000 with 24 SDLT220 Gigethernet NDMP tape drives, also > using Quickrestore. > First: Quickrestore is an excellent product, with excellent vendor support. > The Reliaty/Quickrestore staff are very impressive. > But: The Quantum P14000 with ethernet/ndmp sdlt tape drives has been a major > hassle. We may have finally reached stability, but not after too much work > with the vendor. Quantum support is not in the same class as Quickrestore or > Netapp. And I question our decision to go with ethernet/ndmp tape drives. > Our integration might have gone easier if we'd used FC drives. > We use this library to support an environment with >30TB of data, >15 filers, > over a Cisco dedicated gigethernet backup network. Our best backup throughput > for one filer is 16MB/sec--a good number! (F880) >> Barry Lustig wrote: >>> >>> What jukeboxes are people running on their filers, DLT, AIT-2, etc? >>> Are you happy with what you've chosen? The last group of folks I >>> worked with had a Spectralogic juke with AIT-2 and Netvault. It took >>> way too long to get the problems with firmware on the juke and the >>> drives working with the software. > -- > Quentin Fennessy Quentin.Fennessy(a)amd.com > Office: 512.602.3873 > Cell: 512.694.7489 -- Quentin Fennessy Quentin.Fennessy(a)amd.com Office: 512.602.3873 Cell: 512.694.7489

3 2

RE: Jukeboxes for filers
by Ed Sanborn 19 Dec '02

19 Dec '02

That's 16MB/sec over GigE? -----Original Message----- From: Quentin Fennessy [mailto:quentin.fennessy@amd.com] Sent: Thursday, December 19, 2002 1:02 PM To: Geoff Hardin Cc: Barry Lustig; toasters(a)mathworks.com Subject: Re: Jukeboxes for filers >>>>> Geoff Hardin writes: > Barry, > We use an ATL P3000 library with 16 DLT7000 drives. We use > Workstation Solutions QuickRestore (now Reliaty Backup) to back > everything up. We have 15 filers being backed up and the library is > starting to get overwhelmed. We are considering the new SDLT drives, > since they are backwards read compatible with DLT. It's been a great > solution for over two years now; only change I would have made would > have been to wait for the P3000 GbE library. We've installed a P14000 with 24 SDLT220 Gigethernet NDMP tape drives, also using Quickrestore. First: Quickrestore is an excellent product, with excellent vendor support. The Reliaty/Quickrestore staff are very impressive. But: The Quantum P14000 with ethernet/ndmp sdlt tape drives has been a major hassle. We may have finally reached stability, but not after too much work with the vendor. Quantum support is not in the same class as Quickrestore or Netapp. And I question our decision to go with ethernet/ndmp tape drives. Our integration might have gone easier if we'd used FC drives. We use this library to support an environment with >30TB of data, >15 filers, over a Cisco dedicated gigethernet backup network. Our best backup throughput for one filer is 16MB/sec--a good number! (F880) > Barry Lustig wrote: >> >> What jukeboxes are people running on their filers, DLT, AIT-2, etc? >> Are you happy with what you've chosen? The last group of folks I >> worked with had a Spectralogic juke with AIT-2 and Netvault. It took >> way too long to get the problems with firmware on the juke and the >> drives working with the software. -- Quentin Fennessy Quentin.Fennessy(a)amd.com Office: 512.602.3873 Cell: 512.694.7489

2 1

RE: Jukeboxes for filers
by Greg Libert 19 Dec '02

19 Dec '02

We are using 4 - ATL M1500 dual SDLT modular units with LVD SCSI interfaces. We backup three filers via NDMP and about 40 W2K and UNIX servers via a NT media server using Reliaty backup software. The adminhost is a Sun V120 that also manages the robotics. Nice units as they are a scalable solution and can be stacked on-demand to increase capacity. In the next phase of our implementation we intend to convert the SCSI interfaces to a fiber SAN interface (M1500 has swappable interfaces) to increase performance and flexibility. Install was relatively painless (aside from a DOA unit with the wrong "mushware") and was up and running pretty quickly. -----Original Message----- From: John Clear [mailto:jclear@ati.com] Sent: Thursday, December 19, 2002 11:29 AM To: 'Geoff Hardin'; Barry Lustig Cc: toasters(a)mathworks.com Subject: RE: Jukeboxes for filers We have an ATL dual P4000 hybrid with 6 SDLT NDMP drives and 4 SDLT SCSI drives, and are mostly happy with it (an ATL Field Engineer is onsite right now dealing with a hardware issue :( ). The NDMP drives speak gigabit, so the library isn't connected directly to the filer at all, just via the network. The NetApps just see the NDMP drives as a remote device. We previously had a <mumble> jukebox with DLT7000s connected directly to the filers, and had a few occasions where we had to reboot a filer to fix a hung drive. The SCSI side is used for the other Unix/NT hosts. Software is Legato Networker. John -----Original Message----- From: Geoff Hardin [ mailto:geoff.hardin@dalsemi.com <mailto:geoff.hardin@dalsemi.com> ] Sent: Thursday, December 19, 2002 8:52 AM To: Barry Lustig Cc: toasters(a)mathworks.com Subject: Re: Jukeboxes for filers Barry, We use an ATL P3000 library with 16 DLT7000 drives. We use Workstation Solutions QuickRestore (now Reliaty Backup) to back everything up. We have 15 filers being backed up and the library is starting to get overwhelmed. We are considering the new SDLT drives, since they are backwards read compatible with DLT. It's been a great solution for over two years now; only change I would have made would have been to wait for the P3000 GbE library.

1 0

Jukeboxes for filers
by Barry Lustig 19 Dec '02

19 Dec '02

What jukeboxes are people running on their filers, DLT, AIT-2, etc? Are you happy with what you've chosen? The last group of folks I worked with had a Spectralogic juke with AIT-2 and Netvault. It took way too long to get the problems with firmware on the juke and the drives working with the software. barry

3 2

RE: Jukeboxes for filers
by John Clear 19 Dec '02

19 Dec '02

We have an ATL dual P4000 hybrid with 6 SDLT NDMP drives and 4 SDLT SCSI drives, and are mostly happy with it (an ATL Field Engineer is onsite right now dealing with a hardware issue :( ). The NDMP drives speak gigabit, so the library isn't connected directly to the filer at all, just via the network. The NetApps just see the NDMP drives as a remote device. We previously had a <mumble> jukebox with DLT7000s connected directly to the filers, and had a few occasions where we had to reboot a filer to fix a hung drive. The SCSI side is used for the other Unix/NT hosts. Software is Legato Networker. John -----Original Message----- From: Geoff Hardin [mailto:geoff.hardin@dalsemi.com] Sent: Thursday, December 19, 2002 8:52 AM To: Barry Lustig Cc: toasters(a)mathworks.com Subject: Re: Jukeboxes for filers Barry, We use an ATL P3000 library with 16 DLT7000 drives. We use Workstation Solutions QuickRestore (now Reliaty Backup) to back everything up. We have 15 filers being backed up and the library is starting to get overwhelmed. We are considering the new SDLT drives, since they are backwards read compatible with DLT. It's been a great solution for over two years now; only change I would have made would have been to wait for the P3000 GbE library.

1 0

RE: Jukeboxes for filers
by Clawson, Simon 19 Dec '02

19 Dec '02

We have just got a Storage Tek L80, with SDLT drives. I have to say it is excellent so far. I was using an aging DLT 4000 autoloader, and compared to that the STK is lightyears ahead. We use it in conjunction with Netbackup 4.5 and all in all it is working well, with extreamly good performanc both using NDMP and using cross network backups for other systems. The main swaying point for a unit like the STK L80 was that each drive can be used independantly, meaning we can get more from our investment, backing up NT systems, UNIX boxes and the NetApp. Simon -----Original Message----- From: Barry Lustig [mailto:barry-dated-1040712031.6fa001@lustig.com] Sent: 19 December 2002 06:41 To: toasters(a)mathworks.com Subject: Jukeboxes for filers What jukeboxes are people running on their filers, DLT, AIT-2, etc? Are you happy with what you've chosen? The last group of folks I worked with had a Spectralogic juke with AIT-2 and Netvault. It took way too long to get the problems with firmware on the juke and the drives working with the software. barry

1 0

RE: Jukeboxes for filers
by Alan McLachlan 19 Dec '02

19 Dec '02

LANL are running with 8 x F800 series filers to Spectra Logic 64000 libraries with 16 x AIT-3 drives in each (can go to 32 drives in each), no problems. There were firmware issues with AIT2, particularly if you source additional drives direct from Sony to put into an existing Spectra or Overland library. AIT-3 has been relatively trouble free. What version of OnTap are you running? -----Original Message----- From: Barry Lustig [mailto:barry-dated-1040712031.6fa001@lustig.com] Sent: Thursday, 19 December 2002 5:41 PM To: toasters(a)mathworks.com Subject: Jukeboxes for filers What jukeboxes are people running on their filers, DLT, AIT-2, etc? Are you happy with what you've chosen? The last group of folks I worked with had a Spectralogic juke with AIT-2 and Netvault. It took way too long to get the problems with firmware on the juke and the drives working with the software. barry **** ASI Solutions Disclaimer **** The material transmitted may contain confidential and/or privileged material and is intended only for the addressee. If you receive this in error, please notify the sender and destroy any copies of the material immediately. ASI will protect your Privacy according to the 10 Privacy Principles outlined under the new Privacy Act, Dec 2001. This email is also subject to copyright. Any use of or reliance upon this material by persons or entities other than the addressee is prohibited. E-mails may be interfered with, may contain computer viruses or other defects. Under no circumstances do we accept liability for any loss or damage which may result from your receipt of this message or any attachments. **** END OF MESSAGE ****

2 1

RE: Defragmentation
by Alan McLachlan 19 Dec '02

19 Dec '02

Hi Edward, It's a case of whether or not it makes sense... Firstly, since WAFL always writes to new locations (a basic reason for NetApp's outstanding write performance - avoidance of seeks), any defragmentation effort would quickly be wasted. Filers tend to stay in a state of "partially fragmented" as NVRAM write gathering keeps files in large contiguous "chunks". Secondly, since blocks in snapshots are shared with the active filesystem and several versions of a file are sharing blocks, defragmenting becomes an impossibly complex task. Even if you treated the active filesystem as the base for gathering file segments you would need to rearrange the pointers in the snapshot metadata inodes. Defragmentation would be VERY slow. Doing what you have done (copy off then back on again) may improve read transfer rate for some large files for a short time. But it would do little to nothing for transactional I/O, so for most applications there's no point. Also, this means that you have created new versions of the files that don't share blocks with any of your snapshots, so you lose huge amounts of disk space. Not knocking your intentions, it's just that this discussion has been coming up for years know, and the answer hasn't changed. If it had (or if it really mattered) NetApp would have done somthing about it long ago. regards, Alan McLachlan Senior Systems Engineer Storage Management Solutions ASI Solutions www.asi.com.au Ph +61 2 6230 1566 Fax +61 2 6230 5343 Mobile +61 428 655644 e-mail amclachlan(a)asi.com.au -----Original Message----- From: Edward Hibbert [mailto:EH@dataconnection.com] Sent: Wednesday, 18 December 2002 10:36 PM To: 'toasters(a)mathworks.com' Subject: Defragmentation I think it's general knowledge that once a NetApp gets above 90% full, performance starts to degrade quite noticeably. I've been testing an application recently and observed that this is the case on ONTAP 5.2: * When ~95% full, my performance test came up with a number of ~4 seconds. * Freeing up space to get to around 60% full, the number was ~3.5 seconds. * Copying off my data files and copying them back to defragment them, the number was 2.8s. The only thing I've been able to find about fragmentation is: * Some statements that WAFL tries hard to avoid fragmentation (which I'm sure it does) but can't always (which I'm sure of too). * Copying off data and copying it back again is the only way to resolve it. The latter solution doesn't seem practical if the data in question is supposed to be available 24*7. Some applications have a method of doing this at an application level, but does anyone know whether there are methods/future plans for background non-intrusive defragmentation at a filesystem level? Or is it just me who thinks this would be a good idea? Edward Hibbert Internet Applications Group Data Connection Ltd Tel: +44 131 662 1212 Fax: +44 131 662 1345 Email: eh(a)dataconnection.com Web: http://www.dataconnection.com **** ASI Solutions Disclaimer **** The material transmitted may contain confidential and/or privileged material and is intended only for the addressee. If you receive this in error, please notify the sender and destroy any copies of the material immediately. ASI will protect your Privacy according to the 10 Privacy Principles outlined under the new Privacy Act, Dec 2001. This email is also subject to copyright. Any use of or reliance upon this material by persons or entities other than the addressee is prohibited. E-mails may be interfered with, may contain computer viruses or other defects. Under no circumstances do we accept liability for any loss or damage which may result from your receipt of this message or any attachments. **** END OF MESSAGE ****

1 0

NDMPCOPY solved
by Haber, David J. 18 Dec '02

18 Dec '02

Let me thank everyone one more time. I was successful in copying file from the 840 filer (6.1.1R2) to the NearStore (6.2.1) without upgrading the 840. By running this command from the NearStore it worked: NEARSTORE> ndmpcopy -sa root:rootpassword -da root:rootpassword -st text -dt text 840:/vol/vol0/Test NearStore:/vol/vol0/test Thanks to everyone's input I realized I needed to add the -st option and, after awhile the obvious was revealed---the path needs is case sensitive. Once I had these two elements in place everything worked. So thanks again for everyone's help. I also now have fresh, new root passwords for all of my filers!!!

1 0

← Newer
1
2
3
4
5
6
7
8
9
Older →

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

1999

1998

1997

toasters December 2002