toasters

toasters@lists.teaparty.net

13520 discussions

RE: howto document for initial volume architecting?
by Brian Pauley 10 Nov '05

10 Nov '05

Alternatively, you can use a "du"-like script which avoids traversing the ".snapshot" directory: #!/bin/perl # # Usage mydu [dir] # opendir (DIR, $ARGV[0]) || die "Can't open $ARGV[0] $!"; @dirs = readdir(DIR); closedir (DIR); foreach $d (@dirs) { next if ($d eq "." || $d eq ".." || $d eq ".snapshot"); system ("du -s $ARGV[0]/$d"); } Brian Pauley Systems Engineer Dataline, Inc 2551 Eltham Avenue, Suite O Norfolk, VA 23513 Phone (757)858-0613 ext. 264 Fax (757)858-0606 email: brian.pauley(a)dataline.com -----Original Message----- From: owner-toasters(a)mathworks.com [mailto:owner-toasters@mathworks.com] On Behalf Of Andrew Siegel Sent: Thursday, November 10, 2005 2:13 PM To: Lori Barfield Cc: toasters(a)mathworks.com Subject: Re: howto document for initial volume architecting? Lori Barfield wrote: > > stupidly, i > counted the space each data directory is taking with du -sk, so i got > the snaps as well and all the numbers are off. i guess today i'm > going to be adding up directory sizes with a massive find -prune loop. > (or is there a better way?) One trick for doing du's without encountering any .snapshot directories is to do the du within a snapshot.

1 0

RE: snapmirroring over WAN
by Henry.Chan＠Emulex.Com 10 Nov '05

10 Nov '05

We also had to back up our volumes to a remote site via snapmirror accross a T1 WAN. One of the vols was over 1TB and it would have taken days to create the initial baseline over the T1 so we snapped to SDLT tape, sent it down to CA and created the baseline in a "snap"! Now our nightly VSMs take about 2 hours for 5 gigs (our average delta size). Henry -----Original Message----- From: owner-toasters(a)mathworks.com on behalf of Leeds, Daniel Sent: Thu 11/10/2005 9:21 AM To: Brian Parent; toasters(a)mathworks.com Cc: Subject: RE: snapmirroring over WAN yeah but that initial transfer can be fun if you are mirroring a couple terabyte volumes or if the application is generating alot of changes. a DS3 wan link would be no problem but they may need to rate limit snapmirror on much smaller wan links. -----Original Message----- From: owner-toasters(a)mathworks.com on behalf of Brian Parent Sent: Thu 11/10/2005 9:08 AM To: toasters(a)mathworks.com Subject: Re: snapmirroring over WAN If I recall correctly, snapmirror uses very little bandwidth, and shouldn't be a problem on any modern network. It only copies the blocks which were changed, rather than entire files. Re: > Date: Thu, 10 Nov 2005 08:49:38 -0800 > From: Lori Barfield <itdirector(a)gmail.com> > To: toasters(a)mathworks.com > Subject: snapmirroring over WAN > > for offsite backups, we hope to snapmirror our new fas 3020 on a > regular schedule via vpn over the internet to an older filer at our colo. > obviously, intermittent latency is a concern since we don't have a > dedicated link. and so is the potential for overpowering the modest > firewall/network hardware currently in place. does anyone have a > suggestion for how we might throttle the snapmirrors to improve > reliability in the transmission? > > and how are you folks handling encryption for remote mirroring? > we have a small pix, sonicwall, or cisco to choose from...i'm > guessing we may have to try all three but i hope to pick the one > most likely to succeed the first time. > > > ...lori

2 1

snapmirroring over WAN
by Lori Barfield 10 Nov '05

10 Nov '05

for offsite backups, we hope to snapmirror our new fas 3020 on a regular schedule via vpn over the internet to an older filer at our colo. obviously, intermittent latency is a concern since we don't have a dedicated link. and so is the potential for overpowering the modest firewall/network hardware currently in place. does anyone have a suggestion for how we might throttle the snapmirrors to improve reliability in the transmission? and how are you folks handling encryption for remote mirroring? we have a small pix, sonicwall, or cisco to choose from...i'm guessing we may have to try all three but i hope to pick the one most likely to succeed the first time. ...lori

5 5

RE: howto document for initial volume architecting?
by Palmer, Jason 10 Nov '05

10 Nov '05

If you use Qtrees you can snapmirror on a Qtree basis (QSM) rather than a whole volume - does not bring across the snapshot history like a volume snapmirror (VSM) You can also set Qtree Quotas based upon the Qtree directory, and also you can have different security styles set on a Qtree by Qtree basis. Some backup vendors (Legatto / Symantec nee Veritas) also recognise Qtrees and allows administrators to make the backups more granular, breaking down a potentially multi Tb volume backup into a number of more manageable Gb Qtrees. Qtrees make manipulating and managing data a little easier - sometimes if you don't think you will need to use them, you find that you wish you had at a later date :o) Been there ! Jason -----Original Message----- From: owner-toasters(a)mathworks.com [mailto:owner-toasters@mathworks.com] On Behalf Of Lori Barfield Sent: 10 November 2005 18:49 To: Jerry Cc: John Stoffel; toasters(a)mathworks.com Subject: Re: howto document for initial volume architecting? On 11/10/05, Jerry <juanino(a)yahoo.com> wrote: > Whats the primary reason to use qtrees under flexvols? from my reading, it looks like the only way to define access rights in a non-unix way is to have qtrees. i'd think this could be critical to admins in heterogeneous environments. i found qtrees to be useful yesterday for their nfs ops stats. if you are stuck with one big volume of data, and all your other stats are aggregate, that's a blessing. ...lori -------------------------------------------------------------------------------- The information contained herein is confidential and is intended solely for the addressee. Access by any other party is unauthorised without the express written permission of the sender. If you are not the intended recipient, please contact the sender either via the company switchboard on +44 (0)20 7623 8000, or via e-mail return. If you have received this e-mail in error or wish to read our e-mail disclaimer statement and monitoring policy, please refer to http://www.drkw.com/disc/email/ or contact the sender. 3167 --------------------------------------------------------------------------------

1 0

howto document for initial volume architecting?
by Lori Barfield 10 Nov '05

10 Nov '05

we have the opportunity to build several filers from scratch for our LAN, which will serve all departments in the company, fileshare through Samba, host htdocs for the internal web site, etc. can someone please point me to a good document that offers guidelines for doing the rearchitecting? our architecture is currently flat- one volume per filer with the data only split up into separate directories. all hosts can mount all directories on all volumes if needed. we are overpowered so i'm not too concerned about performance. it does makes sense to have multiple volumes for varying snap schedules, for instance; however i'd like to learn more about what the trade-offs are because i do understand there is a storage penalty. the security paradigm is to depend on firewall protection on the perimeter and to have a relatively open internal network. backups will be snap mirrors to a sibling filer at our offsite colo, with an occasional archival to tape for some of the data. thanks in advance. ...lori

3 3

RE: snapmirroring over WAN
by Lai, Derek 10 Nov '05

10 Nov '05

Lori, Local initial snapmirror is a good idea whenever it's an option. I've setup DR to St. Louis first and now Plano, Texas and it's been very reliable. Snapmirroring is also pretty good at setting up a checkpoint re-start if it has problem getting through. I am also pretty confident that it knows how to handle overlapping snapmirror. I've seen it happen to us when there is networking problem. If it detects that a snapmirroring is still in progress it will not kick off the same snapmirror. Snapmirror has been really reliable. I've had lots more problem with the WAN. The only problem I've seen with Snapmirror is when you run out of disk space (We are using mostly qtree snapmirror). Derek -----Original Message----- From: owner-toasters(a)mathworks.com [mailto:owner-toasters@mathworks.com] On Behalf Of Lori Barfield Sent: Thursday, November 10, 2005 9:30 AM To: toasters(a)mathworks.com Subject: Re: snapmirroring over WAN On 11/10/05, Brian Parent <bparent(a)calvin.ucsd.edu> wrote: > If I recall correctly, snapmirror uses very little bandwidth, and > shouldn't be a problem on any modern network. It only copies the > blocks which were changed, rather than entire files. ah yes, sorry that i didn't mention this point...we're doing an initial ndmpcopy over a little gig switch here at the office before moving the filer to the colo. some of our data is going to be large image files, so we aren't sure yet how much bandwidth the syncs are going to want. i was more concerned about the jerks and jolts and irregular paths you get when you fling data out into the wild...not sure how well netapps handle hiccups in the middle of snapping. ...lori

1 0

RE: snapmirroring over WAN
by Johnson, John T. 10 Nov '05

10 Nov '05

SnapMirror will consume as much bandwidth as it can get. I have a DS3 between Minneapolis and Denver and SnapMirror will flood it 100% no matter if it's a full or incremental transfer. The only difference is how long it floods the pipe. There's two areas where you can change this. If you can set QOS on your VPN link between Filers IP addresses and set a reasonable bandwidth parameter, I would do that. Otherwise you can set in in Filerview when setting up the Snapmirror volume relationship under Maximum Transfer Rate. This can be tedious as you have to set it on each replicated volume. We set it on our Cisco router to run at 10 Mbit during the day and automatically jump to 40 Mbit after 10:00 PM. I replicate Exchange and SQL data daily with logs getting replicated every 2 hours. My delta changes are roughly 300 gigs per night and at 40 Mbits (the other 5 Mbits we reserve for AD replication and normal network traffic), it takes about 6 hours. SnapMirror also doesn't thread its transfers. It just blasts the pipe so if you can stagger your transfers you should get better throughput. Good luck, John -----Original Message----- From: owner-toasters(a)mathworks.com [mailto:owner-toasters@mathworks.com] On Behalf Of Lori Barfield Sent: Thursday, November 10, 2005 10:50 AM To: toasters(a)mathworks.com Subject: snapmirroring over WAN for offsite backups, we hope to snapmirror our new fas 3020 on a regular schedule via vpn over the internet to an older filer at our colo. obviously, intermittent latency is a concern since we don't have a dedicated link. and so is the potential for overpowering the modest firewall/network hardware currently in place. does anyone have a suggestion for how we might throttle the snapmirrors to improve reliability in the transmission? and how are you folks handling encryption for remote mirroring? we have a small pix, sonicwall, or cisco to choose from...i'm guessing we may have to try all three but i hope to pick the one most likely to succeed the first time. ...lori

1 0

RE: snapmirroring over WAN
by Leeds, Daniel 10 Nov '05

10 Nov '05

yeah but that initial transfer can be fun if you are mirroring a couple terabyte volumes or if the application is generating alot of changes. a DS3 wan link would be no problem but they may need to rate limit snapmirror on much smaller wan links. -----Original Message----- From: owner-toasters(a)mathworks.com on behalf of Brian Parent Sent: Thu 11/10/2005 9:08 AM To: toasters(a)mathworks.com Subject: Re: snapmirroring over WAN If I recall correctly, snapmirror uses very little bandwidth, and shouldn't be a problem on any modern network. It only copies the blocks which were changed, rather than entire files. Re: > Date: Thu, 10 Nov 2005 08:49:38 -0800 > From: Lori Barfield <itdirector(a)gmail.com> > To: toasters(a)mathworks.com > Subject: snapmirroring over WAN > > for offsite backups, we hope to snapmirror our new fas 3020 on a > regular schedule via vpn over the internet to an older filer at our colo. > obviously, intermittent latency is a concern since we don't have a > dedicated link. and so is the potential for overpowering the modest > firewall/network hardware currently in place. does anyone have a > suggestion for how we might throttle the snapmirrors to improve > reliability in the transmission? > > and how are you folks handling encryption for remote mirroring? > we have a small pix, sonicwall, or cisco to choose from...i'm > guessing we may have to try all three but i hope to pick the one > most likely to succeed the first time. > > > ...lori

1 0

RE: snapmirroring over WAN
by Frank, Jason 10 Nov '05

10 Nov '05

Well, that all depends on how many changed blocks you expect in a day, as well as what kind of link you have, how busy the link is currently, what exact model of PIX or Sonicwall you have, and what the operating hours of the business are. Snapmirror does include a throttle, but if you throttle it too much, you'll be perpetually behind. And if you don't throttle it enough, it's about the same as not having one. It might be possible to do your snapmirror overnight, if it's acceptable to your client, and not throttle it at all. There are really a lot of options here. -----Original Message----- From: owner-toasters(a)mathworks.com [mailto:owner-toasters@mathworks.com] On Behalf Of Lori Barfield Sent: Thursday, November 10, 2005 10:50 AM To: toasters(a)mathworks.com Subject: snapmirroring over WAN for offsite backups, we hope to snapmirror our new fas 3020 on a regular schedule via vpn over the internet to an older filer at our colo. obviously, intermittent latency is a concern since we don't have a dedicated link. and so is the potential for overpowering the modest firewall/network hardware currently in place. does anyone have a suggestion for how we might throttle the snapmirrors to improve reliability in the transmission? and how are you folks handling encryption for remote mirroring? we have a small pix, sonicwall, or cisco to choose from...i'm guessing we may have to try all three but i hope to pick the one most likely to succeed the first time. ...lori

3 2

High network traffic due to RPC Authoriztion (nfs mounts)
by Tavis Gustafson 10 Nov '05

10 Nov '05

We are seeing extremely high network traffic from single clients to nfs mounted filers. In most cases running nfswatch on the client shows almost all of the traffic due to "RPC Authorization". We are talking 50-80Mbits of traffic with nothing sticking out on the clients process list. Has anyone seen weird high network traffic via nfs over udp? thank -tavis -- ___ /\__\ /:/ / /:/__/ /::\ \ /:/\:\ \ \/__\:\ \ \:\__\ \/__/ avis tavis(a)hq.newdream.net New Dream Network

2 3

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

1999

1998

1997

toasters