toasters June 2004

toasters@lists.teaparty.net

76 participants
86 discussions

RE: DR project - SnapMirroring Oracle database
by Derek Lai 02 Jul '04

02 Jul '04

I have just 1 database sitting on 2 volumes on netapp02 that are 600GB together. The database is only about 250GB or so. The volumes were sized that way to give sufficient performance (26 X 36GB drives). We are using QT SnapMirror to another filer's (netapp) volume so we can backup to tape from there. We would not want to do volume snapmirror since we will have to waste quite a bit of disk space in order to do so. So now that we are doing DR, it seems logical to do it from the qtrees that were already snapmirrored on netapp. We really want to avoid having to do it from the original volumes since these disks are already being hit very hard by the production database. We are actually considering adding more disks to the production database even though we are only 43% full just so we can get better performance. If we have to do volume snapmirrors this will at least double if not triple that requirement and it seems outrageous to have to do so. Best solution is to have cascading Qtree SnapMirror. Anyone know if this is in the works? I can't believe that I'm the only one who needs this. Derek -----Original Message----- From: James Brigman [mailto:jbrigman@nc.rr.com] Sent: Monday, June 28, 2004 8:41 PM To: toasters(a)mathworks.com Subject: RE: DR project - SnapMirroring Oracle database Derek, et. al; I'll reply to you publicly on the list and see if I get a rise out of any of the others on Toasters. First: I'd advise you to not yet rule out qtree snapmirrors because they won't cascade. Tell us all why you need more than one mirror to do DR and backup with? Second: I just finished setting up qtree mirrors for a couple databases I've got. What I have, actually, are five volumes: Orabackup Oraarch Oradata Orabase Oralog I've got five databases, and five qtrees in each volume, one per database. And let me tell you: as bad as you might think qtree mirroring is, volume mirroring is worse for this configuration. Anyway: you can do your backup any number of ways. The ways I considered are: 1) Shutdown of the database and mirror or snap of the "oradata" contents (database files themselves). 2) Do nightly full dumps and snap, mirror or backup those. 3) Put the database into hot backup mode and snap both the data and the logs. There might be other, more creative ways, but I'm not a dba. Anyway: the task was to mirror these existing volumes to an R200. Right away, a bad problem surfaced: my oralog volume on my production filer was only two 72GB drives. Doing volume mirroring would eat up huge amounts of space on the R200 destination filer for no good reason. So I mirrored all the qtrees while doing a hot backup operation. Seems to work OK. Watch out! It takes extra space for the mirror snaps on the source filers! For what it's worth: NetApp filers stand the OFA on it's head, and there don't seem to be many people walking around who know this yet. There are some good hints on Oracle configuration on now.netapp.com, but they don't go into extreme detail at all. In the next couple weeks, I hope to change this on a few of my test/dev databases by making one huge volume and exporting the database mounts from qtrees. I will recover SIX parity drives for reuse as data if I do this! Keep up the chatter. This is an important topic that few seem to understand well. JKB -----Original Message----- From: owner-toasters(a)mathworks.com [mailto:owner-toasters@mathworks.com] On Behalf Of Derek Lai Sent: Monday, June 28, 2004 1:09 PM To: 'toasters(a)mathworks.com' Subject: RE: DR project - SnapMirroring Oracle database I have not gotten much response. Hopefully it is due to the timing of my Friday afternoon email. If you have done a DR project with Database and SnapMirroring, I'd love to hear about your experience. Over the weekend I just came across another big obstacle to using SnapMirroring - you can't cascade qtree SnapMirror! This one seems to be a very big restriction as it means wasting quite a bit of space to do things like backup and DR. Derek -----Original Message----- From: Derek Lai [mailto:Derek.Lai@onyxco.com] Sent: Friday, June 25, 2004 3:12 PM To: 'toasters(a)mathworks.com' Subject: DR project - SnapMirroring Oracle database I'm in the process of architecting a DR project for mirroring our production Oracle database and would love to hear from those who have done so already. We have been SnapMirroring the daily changes from one of our cluster F940 filer to the other for backup. In looking through the SnapMirror log I see that we are snapmirroring about 23GB of data on average. This is on a database of about 200GB total. We were thinking about using the max of the daily snapmirror number to size the pipe we need to the DR site. That is figuring to be a fair size pipe. Now our DBA is telling me that using Oracle's archive log they could just ship the logs to the remote site and play it back and achieve pretty much the same result. And the archive logs are averaging only about 4.5GB. That is 1/5 the size. It is fairly significant when you are shipping it half-way across the country! (We are in Foothill Ranch, CA and our DR site is in St. Louis). If that is true then management will probably choose to go with log shipping as opposed to SnapMirroring the data. I'm going through the data that we are SnapMirroring to figure out if there is something wrong with our setup. I think we are actually SnapMirroring the archive logs as well. Has anyone else been down this path? Any other ideas or suggestions? We are thinking that the log shipping method is going to involve alot more manual work and would like to avoid it if possible but we need to have good justification for that as well. Derek

2 1

AW: Raid DP conversion?
by Engmann, Tobias 01 Jul '04

01 Jul '04

Correction the raidgroupsize setting will change but if you have already multiple raid groups THIS will not be change -----Ursprüngliche Nachricht----- Von: Engmann, Tobias Gesendet: Mittwoch, 30. Juni 2004 19:27 An: 'Brubaker, Paul' Cc: toasters(a)mathworks.com Betreff: AW: Raid DP conversion? Just set the raidlevel from raid4 to raid dp vol options "volname" raidtype dp the filer will grap a spare and do a reconstruct on it the Raidgroupsize will NOT change ! cheers tobias -----Ursprüngliche Nachricht----- Von: Brubaker, Paul [mailto:paul.brubaker@attws.com] Gesendet: Mittwoch, 30. Juni 2004 18:48 An: toasters(a)mathworks.com Betreff: Raid DP conversion? Question on Raid 4 vs. Raid DP. We just upgraded to Ontap 6.5.1 and the new feature of Raid DP is now present. Our existing volumes created on Ontap 6.4.2 are setup as Raid 4. Is there any "easy" conversion process to change the old Raid 4 volumes to the newer Raid DP config? Any benefits? Advice? Best practices? thanks! Paul M. Brubaker, Jr. RIDS - Intel System Support AT&T Wireless - Harrisburg PA e-mail: paul.brubaker(a)attws.com office #: 717-526-5011 cell #: 717-578-2254

3 2

Raid DP conversion?
by Brubaker, Paul 01 Jul '04

01 Jul '04

Question on Raid 4 vs. Raid DP. We just upgraded to Ontap 6.5.1 and the new feature of Raid DP is now present. Our existing volumes created on Ontap 6.4.2 are setup as Raid 4. Is there any "easy" conversion process to change the old Raid 4 volumes to the newer Raid DP config? Any benefits? Advice? Best practices? thanks! Paul M. Brubaker, Jr. RIDS - Intel System Support AT&T Wireless - Harrisburg PA e-mail: paul.brubaker(a)attws.com office #: 717-526-5011 cell #: 717-578-2254

3 2

RE: RAID Group Size Limits?
by Ehrhart, Rick 30 Jun '04

30 Jun '04

Don't forget there is double parity RAID on 6.5 and later. - Rick - -----Original Message----- From: Michael Christian [mailto:mchristi@yahoo-inc.com] Sent: Wednesday, June 30, 2004 4:38 PM To: James Brigman; toasters(a)mathworks.com Subject: RE: RAID Group Size Limits? Sure, just keep your raid group size at a reasonable level, to avoid double disk failures, and use multiple raid groups. I had a semi-large volume running RAC for about a year which had 6 groups of 12, so 72 spindles in the volume. (I certainly wouldn't use a raid group of 72 though!) -MC _____ From: owner-toasters(a)mathworks.com [mailto:owner-toasters@mathworks.com] On Behalf Of James Brigman Sent: Wednesday, June 30, 2004 4:06 PM To: toasters(a)mathworks.com Subject: RAID Group Size Limits? Does anyone on the list have experience with RAID group size limits? I seek to convert my qty 5 small volumes for Oracle into qty 2 volumes (one big, one little) for oracle and use lots of spindles. In fact, I'd like to just do a big, giant volume on my filer and put everything in qtrees shared/exported. Anyone out there done this yet? Hit any RAID size limits? Thx; JKB

1 0

RE: RAID Group Size Limits?
by Hill, Aaron 30 Jun '04

30 Jun '04

The absolute limit will depend on both your platform (FAS900 series, FAS250, R100, R150, R200 or F series) and ONTAP Version. Check the ONTAP release manual for your version. For example, 6.5 supports max of 28 for FAS900 & F series filers. I think this has been the same for quite a few releases. RAID DP cuts that in half. Seems you have two goals; 1) Consolidate exisiting volumes into fewer, larger volumes 2) Get some performance increase by increasing spindles You can make your volumes & raid groups larger and thus get more spindles. However, there are two questions you need to answer first; 1) Are you sure that disk I/O is the bottleneck in your DB performance? i.e. Run a statit analysis on your filer for some high and low usage periods over the next few days and examine the results. 2) How important is volume restore time? Remember, the larger the volume, the longer it will take to restore if there is a double-disk failure. (Of course, you can always mitigate some of the risk here by upgrading to 6.5.1 and using RAID DP, however this limits you to max raid size of 14). I believe NetApp recommends to keep volumes under 0.5? TB for this very reason. Now, if (1) is true and (2) is not so important, you still have one more thing to keep in mind; Increasing spindles follows the law of diminishing returns. I recall reading somewhere that the optimum raid size for performance was 10-14, anything larger than that and the performance return is exceedingly small. At one place we ran Sybase DB's on 36G disks with RAID sizes of 10 & 12 with good results. Although there are a lot of other factors that come into play with DB performance. Ok, that should be enough information to help you make some preliminary judgements. Let me know how you go. Aaron _____ From: James Brigman [mailto:jbrigman@nc.rr.com] Sent: Thursday, 1 July 2004 9:06 AM To: toasters(a)mathworks.com Subject: RAID Group Size Limits? Does anyone on the list have experience with RAID group size limits? I seek to convert my qty 5 small volumes for Oracle into qty 2 volumes (one big, one little) for oracle and use lots of spindles. In fact, I'd like to just do a big, giant volume on my filer and put everything in qtrees shared/exported. Anyone out there done this yet? Hit any RAID size limits? Thx; JKB ************** IMPORTANT MESSAGE ************** This e-mail message is intended only for the addressee(s) and contains information which may be confidential. If you are not the intended recipient please advise the sender by return email, do not use or disclose the contents, and delete the message and any attachments from your system. Unless specifically indicated, this email does not constitute formal advice or commitment by the sender or the Commonwealth Bank of Australia (ABN 48 123 123 124) or its subsidiaries. We can be contacted through our web site: www.commbank.com.au **************************************************

1 0

RE: RAID Group Size Limits?
by Michael Christian 30 Jun '04

30 Jun '04

Sure, just keep your raid group size at a reasonable level, to avoid double disk failures, and use multiple raid groups. I had a semi-large volume running RAC for about a year which had 6 groups of 12, so 72 spindles in the volume. (I certainly wouldn't use a raid group of 72 though!) -MC _____ From: owner-toasters(a)mathworks.com [mailto:owner-toasters@mathworks.com] On Behalf Of James Brigman Sent: Wednesday, June 30, 2004 4:06 PM To: toasters(a)mathworks.com Subject: RAID Group Size Limits? Does anyone on the list have experience with RAID group size limits? I seek to convert my qty 5 small volumes for Oracle into qty 2 volumes (one big, one little) for oracle and use lots of spindles. In fact, I'd like to just do a big, giant volume on my filer and put everything in qtrees shared/exported. Anyone out there done this yet? Hit any RAID size limits? Thx; JKB

1 0

AW: Raid DP conversion?
by Engmann, Tobias 30 Jun '04

30 Jun '04

Just set the raidlevel from raid4 to raid dp vol options "volname" raidtype dp the filer will grap a spare and do a reconstruct on it the Raidgroupsize will NOT change ! cheers tobias -----Ursprüngliche Nachricht----- Von: Brubaker, Paul [mailto:paul.brubaker@attws.com] Gesendet: Mittwoch, 30. Juni 2004 18:48 An: toasters(a)mathworks.com Betreff: Raid DP conversion? Question on Raid 4 vs. Raid DP. We just upgraded to Ontap 6.5.1 and the new feature of Raid DP is now present. Our existing volumes created on Ontap 6.4.2 are setup as Raid 4. Is there any "easy" conversion process to change the old Raid 4 volumes to the newer Raid DP config? Any benefits? Advice? Best practices? thanks! Paul M. Brubaker, Jr. RIDS - Intel System Support AT&T Wireless - Harrisburg PA e-mail: paul.brubaker(a)attws.com office #: 717-526-5011 cell #: 717-578-2254

1 0

How do you migrate from Novell to Netapp?
by Mike Langas 30 Jun '04

30 Jun '04

Greetings all, After a five year absence I find myself back in the NetApp world. I just convinced my employer to replace their aging novell servers with a netapp. Now the next logical question is how best to move the data? Does anyone know of a good way to migrate it all and keep ownership, permissions, etc... in place. Unfortunately I haven't been able to get the search facility for this list to work so I apologize if this question has been asked a million times before. Thanks for any help. Mike __________________________________ Do you Yahoo!? Yahoo! Mail - 50x more storage than other providers! http://promotions.yahoo.com/new_mail

2 1

RE: SOLUTION: RE: Enlarge a volume owned by a vfiler
by Avs, Mohan 30 Jun '04

30 Jun '04

Hi Michael, This option would be set to on when you do a snapmirror between filers. During a snapmirror the destination volume would be adjusted as the size of the source volume. This option will not get turned off even if you break or release the snapmirror. You need to change this manually. Thank you, Mohan AVS NetApp Global Services -----Original Message----- From: toaster(a)chopel.de [mailto:toaster@chopel.de] Sent: Wednesday, June 16, 2004 10:38 PM To: Thorsten.Lemke(a)computacenter.com; toasters(a)mathworks.com; Fox, Adam; tsuneyuki.uehara(a)gs.com; toasters(a)mathworks.com Subject: SOLUTION: RE: Enlarge a volume owned by a vfiler Hi, here is the solution for my problem: Normally it is like Adam Fox and his colleague wrote. Just adding the disks to both sites and make a 'snapmirror update' on the destination. Finish! But, first look on the "primary" Volume if the option fs_size_fixed is set to "on" before adding the disks. I don't know why this options was set to "on", maybe once we tested to rotate the mirror. But turning the option off resolve our problem. Thanks to all, Michael Thorsten.Lemke(a)computacenter.com wrote : > As I understand you have two volumes called "Volume1" that are both in > snapmirrored state? One is a normal snapmirror and the other one is a DR > backup of the vfiler? > In this case you have to do the enlargement also on the original (third) > source volume which is not in snapmirrored state and therefore does not > have the fs_size_fixed set to "on". The size of the filesystem of > snapmirrored volumes will adapt to the source volume after the next > scheduled update as Adam pointed out earlier (of course you have to add the > same amount of space to each of them first). > > Regards, > Thorsten > > > > > > toaster(a)chopel.de>@mathworks.com > on 16.06.2004 11:22:11 > > Please respond to toaster(a)chopel.de> > > Sent by: owner-toasters(a)mathworks.com > > > To: "Fox, Adam" Adam.Fox(a)netapp.com>, > "Uehara; Tsuneyuki (Gen)" > tsuneyuki.uehara(a)gs.com>, > toasters(a)mathworks.com> > cc: > Subject: RE: Enlarge a volume owned by a vfiler > > > -------------------------------------------------------------------------- > > > > Hallo, > > maybe my initial situation was not clear: > > On the "main site" I spent VfilerA a new disk. > Volume1 before "vol add" : 201 GB > Volume1 after "vol add" : also 201 GB > > vol status -b Volume1: > Vol Size (blocks) FS Size (blocks) > ----------------- ---------------- > 76996480 61597184 > > The filer notified that "..the disk was successfully added to volume > Volume1" > IMHO this results because Volume1 is in "snapmirrored-state" > an the option "fs_size_fixed" is set "on". > > On the "desaster site" I spent VfilerA' (vfiler dr) also > a new disk for volume Volume1. Same results. > > I think I must break the mirror but is this possible > without side effects on VfilerA because he is in "vfiler dr" > > Thanks a lot > > Michael > > Fox, Adam Adam.Fox(a)netapp.com> > schrieb : > > > You don't need to do this. It looks like you are doing Volume > > SnapMirror. All you have to do is add the disks to both volumes. > > You won't see the change on the destination until the next update, > > then the destination will become the size of the source. > > > > -- Adam Fox > > GS Tools Developer > > adamfox(a)netapp.com > > > > > > > > -----Original Message----- > > From: Uehara, Tsuneyuki (Gen) [mailto:tsuneyuki.uehara@gs.com] > > Sent: Tuesday, June 15, 2004 7:23 AM > > To: 'toaster(a)chopel.de'; > > toasters(a)mathworks.com > > Subject: RE: Enlarge a volume owned by a vfiler > > > > > > We perform "snapmirror quiesce" then "snapmirror break" then turn off > > "fs_size_fixed". > > > > # Gen Uehara > > > > -----Original Message----- > > From: owner-toasters(a)mathworks.com > > [mailto:owner-toasters@mathworks.com] > > On > > Behalf Of toaster(a)chopel.de > > Sent: Tuesday, June 15, 2004 10:12 PM > > To: toasters(a)mathworks.com > > Subject: Enlarge a volume owned by a vfiler > > > > Hi, > > > > I have a problem to enlarge a volume owned by a "dr"-vfiler. > > > > Vfiler1 Vfiler1 (dr backup) > > |--> vol1 |-->vol1 > > > > It was possible to add a disk to vol1 on both sites. But I can't disable > the > > volume option "fixed fs_size", because vol1 is in "snapmirrored-state". > > > > How can I enlarge the volume without deleting the dr-filer? > > > > Thanks for help, > > > > Michael > > > > > > > > > > > > COMPUTACENTER (UK) LTD > > The contents of this e-mail are intended for the named addressee only. > It contains information which may be confidential and which may also be > privileged. > Unless you are the named addressee (or authorised to receive > for the addressee) you may not copy or use it, or disclose it to anyone > else. If you received it in error please notify us immediately and then > destroy it. > > Computacenter information is available from http://www.computacenter.com > > This footnote also confirms that this email message has been swept for the > presence of computer viruses. > > ***************************************************************************************************************************

2 1

RE: Piecing together a netapp on the cheap - PART II
by Timothy Sesow 30 Jun '04

30 Jun '04

I have done Snapshots with EVMS (Enterprise Volume Management Software) under LINUX. It does basic snapshots and only has a command line interface (you do your own crontab). It is slower then NetApp snapshots (even more so the more snapshots you get), but it does work. You still have to figure out how to quiesce your filesystem during the snap (umount/mount works, but can be a challenge). The only thing close to SnapMirror I have found is to do RAID-1 via iSCSI or FCIP. Big investment in switches, though. Just have one local disk and more remote disk striped together under the "md" driver. The RAID striping isn't done in the shelf, the enclosure has a SES/CEMI card that gives the Filer information on temperatures, voltages, what power supplies are working, what the health of the local drive bus is, etc. This is either a SCSI target or a Fibre Channel port/WWN that the Filer talks to. The shelves are just JBODs and you can hook them up to any host you want and use them. It's just that when you hook them into the filer, the filer expects certain behaviors. You are on the right track on the IDE (I prefer the term ATA) drives, but the following issues present themselves: 1. LINUX has a 2TB file system limit. Solaris has a 1TB limit. NetApp has something like a 6TB limit. 96Tbytes is the most number of disk shelves (24) that can be hooked into that filer. I would much rather have 16 file systems with NetApp than 48 file systems with LINUX. 2. Getting LOTS of drives working all at once is a real challenge and starts to cost $$$ for more hardware (enclosures, SCSI/Fibre Channel interfaces, etc). The 7TB R200 is high on a $/MB basis ($.01 per MB), but a fully loaded R200 LISTS for half that per MB ($.006 per MB) and any reseller will give you a discount (maybe 20%) off that. The old R100/R150 have been discontinued (real pain to get those SCSI drive shelfs installed in the field). The R200 is easier to install/manage than the old R1x0. Plus MAXTOR doesn't make the old drive shelves anymore. I'm not saying that NetApp is always better, but they have their place in the overall scheme. Small NAS boxes are getting really good and NetApp is responding to the market that they choose to, which I would characterize as the "more fully featured solution" that has a higher return to NetApp. You might also want to look at http://www.openfiler.org. I have tried it and it has promise, but it is still a challenge. I still stick with RedHat 8 (yes, 8, not 9, cause 8 doesn't ever crash/hang) and just add in SAMBA 3.x for a really cheap NAS box. -----Original Message----- From: owner-toasters(a)mathworks.com on behalf of Joe Schmoe Sent: Mon 6/14/2004 12:26 PM To: toasters(a)mathworks.com Subject: Piecing together a netapp on the cheap - PART II Thanks for your help so far. Let me give some more details and answer some of the points you all have raised. First, I am looking for cheap and large - and am willing to sacrifice speed to get it. Second, I am very intrigued by SnapMirror (replication over WAN) and SnapShot copies (point in time backups). The reason I don't just build my own little unix fileservers with linux or FreeBSD is that I don't see how GNU tools and scripting can replicate the functionality of snapmirror and snapshot copies. Maybe I'm wrong ? So, I see that my initial plan is ill advised - you have pointed out that the disk shelves that NetApp ships are not just JABODs, but have intelligence in them - I did not know that the RAID striping was done in the shelf - I thought all RAID and logic was housed in the head unit and it just accessed big stacks of disks. So at this point, my questions are as follows: 1. Is there any reasonable implementation of snapmirror / snapshot copies in free UNIX OSs ? That is, can I get netapp like features out of a normal NFS fileserver ? 2. When I see netapp quote a certain capacity for its filers, like 96TB for the NearStore R200, does that simply mean that they can get to 96TB using the largest currently available IDE drives, and that once larger drives are available that number will naturally go up ? OR does it mean there is some logical limitation in place on the unit that will keep it from going above 96TB ? 3. What is the absolute cheapest IDE based NAS that netapp has put out ? Is it the NearStore R100 ? What if I bought the head unit and shelves from netapp (or netapp branded from ebay), but used my own 400gig hitachi IDE drives ? - I see a 7TB R100 on ebay right now for $78,500 - which is completely absurd. My 4TB freeBSD fileserver at home cost about $5k ... Again, many thanks. --------------------------------- Do you Yahoo!? Friends. Fun. Try the all-new Yahoo! Messenger

7 6

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

1999

1998

1997

toasters June 2004