toasters February 2005

toasters@lists.teaparty.net

68 participants
73 discussions

Is there a Toasters-like list for EMC?
by Michael Coxe 12 Feb '05

12 Feb '05

Just curious if there is a mailing list similar to Toasters - independent of the vendor - for EMC Clarion/Celerra/etc. - michael

1 0

Re: high performance file systems, non Netapp.
by Maren S. Leizaola 12 Feb '05

12 Feb '05

On Fri, 11 Feb 2005, John Stoffel wrote: > > Maren> Hi, We have an application where we need to store between 4 to > Maren> 20 million small files on the on a large drive or a 3 drive > Maren> raid system. > > You should read up on how newservers used to deal with this issue. > Basically, you want to keep down the number of entries per-directory > as much as possible. So instead of having /data/<millions of files> > you would have /data/a/<a* files>, /data/b/<b* files>, ... /data/z/<z* > files>. You would push this down as many levels as you needed. Yes, that may help on seeking and reading but the ammount of time required to delete 1Million files is still huge. I've not tested it though. I recall delete this kind of structure that squid would setup and it would take for ever. The application that is creating all these files is actually Mysql, and under UFS2 we got upto the limit of 1 million files in a single directory. Yes, it worked ok, but deleting the files took for ever. We are looking at hacking mysql so that it creates a directory tree. Still this only going to solve one part of the problem. The time required for dropping such databases or removing tables is long. We have use the more advance database format Innodb that Mysql offers as it is meant to be faster and it only uses one file per table instead of the default 3 files. > Maren> We are finding that file systems like UFS just don't cut it > Maren> never mind Linux extFS... These file systems can handle what we > Maren> are trying to do but they tend to slow down as they get full > Maren> and in some cases we will have to be able to delete millions of > Maren> files in one go. > > Have you looked at the ext3 with the indexed directory entries? > Supposedly it's alot faster. Google for it, using 'ext3 dir_index'. > I just did it on my 2.6.11-rc2-mm2 kernel system at home. Not > performance testing though. I will have a look at this.... > > tune2fs -O dir_index /dev/sdc1 > > Maren> In this type of configuration unfortunately we can't bundle at > Maren> netapp as the technical overhead and cost would make the system > Maren> unfeasable. > > What techincal overhead? You just turn it on and it runs. Just say > it's a cost issue and let it be. > This is part of a system that we need to hand over to customers and sticking a netapp into the equation would make most people panic i think. Even though I don't dissagree that a netapp would be ideal for this. It won't need huge throughputs just low latency and lots of random seeking. > Maren> I have so far seen the ReiserFS can do what we are looking for > Maren> but we need a fs that will work on Linux, FreeBSD, Solaris > Maren> etc... We must have a path to scale up or deploy a heavy load > Maren> system and migrate to a big sun box if required. > > Maren> Anyone knows how well Solaris 10's ZFS would work for this kind > Maren> of application? > > Well, since ZFS is only on Solaris, it doesn't meet your standard of > being available on all filesystems. The only one that I can think of, > and I don't know about FreeBSD, is VxFS from Veritas. I'd look into > that if I was you. > at least if we can start on solaris we can start from old U2' E250 to huge monster machines and one maybe could dare to go into X86 solaris... it seems that sun have been working very hard on this. > But first, I'd read up on how nntp news servers handled the exact > same issue, with millions of small files that needed to be > added/deleted all the time. Follow their lead and re-write your > application to fit the requirements. Yes i think it would a good place to where to start... Maren. > John > John Stoffel - Senior Staff Systems Administrator - System LSI Group > Toshiba America Electronic Components, Inc. - http://www.toshiba.com/taec > john.stoffel(a)taec.toshiba.com - 508-486-1087 > -------------------------------------------------------------- HKdotCOM Ltd Tel: 852 2865-4865 ext 888 Fax: 852 2865-4100 leizaola(a)hk.com AIM: MarenHKdotCOM ICQ: 39905706 MSN:MarenHKdotCOM -------------------------------------------------------------- Get your @hk.com email address on http://www.hk.com

1 0

high performance file systems, non Netapp.
by Maren S. Leizaola 11 Feb '05

11 Feb '05

Hi, We have an application where we need to store between 4 to 20 million small files on the on a large drive or a 3 drive raid system. We are finding that file systems like UFS just don't cut it never mind Linux extFS... These file systems can handle what we are trying to do but they tend to slow down as they get full and in some cases we will have to be able to delete millions of files in one go. In this type of configuration unfortunately we can't bundle at netapp as the technical overhead and cost would make the system unfeasable. I have so far seen the ReiserFS can do what we are looking for but we need a fs that will work on Linux, FreeBSD, Solaris etc... We must have a path to scale up or deploy a heavy load system and migrate to a big sun box if required. Anyone knows how well Solaris 10's ZFS would work for this kind of application? What do you guys suggest? Maren. -------------------------------------------------------------- HKdotCOM Ltd Tel: 852 2865-4865 ext 888 Fax: 852 2865-4100 leizaola(a)hk.com AIM: MarenHKdotCOM ICQ: 39905706 MSN:MarenHKdotCOM -------------------------------------------------------------- Get your @hk.com email address on http://www.hk.com

1 0

RE: Diskless Boot of Windows/iSCSI
by Jeff Mohler 10 Feb '05

10 Feb '05

This fits 1/2 the model, the key is that the base load never changes..if the end-user changes something that could have messed the encoding process up somehow, a simple reboot would repair it. To replace the base LUNs, I would have to copy from the snapshot, would take time to copy that file from snapshot space to real space, correct? Ive got Gig to the tent switch (clean workspace on refuel floor) and 100Mbit from there to the PCs. Performance should be in the OK range. The critical function for these PCs is encoding video, once loaded up network traffic for apps or I/O would be small. Also looking at some solutions that manage disk images that will always force clients to a base boot image on a reboot, but Id like to keep the technology layer thin, and I already have netapps in the design. -----Original Message----- From: owner-toasters(a)mathworks.com [mailto:owner-toasters@mathworks.com]On Behalf Of Alan Biren Sent: Sunday, February 06, 2005 7:42 PM To: toasters(a)mathworks.com Subject: RE: Diskless Boot of Windows/iSCSI Jeff, As long as you have the base snapshot you can replace the boot LUNs quickly. You can also replicate this image to all of the field sites so that each are configured identically. The one major problem is the hardware has to be identical on each PC that will boot from SAN if you want to use the same image. You'll also need an iSCSI HBA. I would make sure that Network connectivity is reliable between the 250's and the PC's. I believe that the NOW site has instructions on seeting up a diskless system and the requirements to double check. _____ From: owner-toasters(a)mathworks.com on behalf of Jeff Mohler Sent: Sun 2/6/2005 5:19 PM To: toasters(a)mathworks.com Subject: Diskless Boot of Windows/iSCSI Greets: Same name, new location.. Im working a storage solution under contract using Netapp F250s in the field using snapmirror to replicate data back to a central F270 from nuclear plant inspection projects all over the world. I have 8 windows based PCs at each site doing encoding of camera inspections (IVVI), and those write to a file and/or go live over the network to the central location based on the inspection parameters. My largest source of problems are the encoding PCs which are not in the friendliest of environments on the refueling floor. Im looking for advice on whether I should change these PCs to work as diskless boot stations using ISCSI from the F250 that is at each site. The ultimate goal would be a situation where on boot the PC is booted to a static state of base configuration, and if the end-user changes anything (within MS Media encoder/etc) they just reboot to get back to a reliable configuration. The end user isnt storing any data in this image, just USING it to create data stored in a volume on the same F250 at the remote site. Just how diskless can I get, while maintaining an easy to use environment where nobody can screw up the base load image? My window of downtime is 10mins or less to full recovery. ***************************************************************************************************** If you have received this email in error please notify the the system manager or the sender immediately and do not disclosed the to anyone or make copies. ** eSafe scanned this email for viruses, vandals or malicious content ** ** To find out eSafe(TM) and other Aladdin Security Products, contact AMC Corporation at (212) 736-0111 *****************************************************************************************************

3 3

netgroup problem
by Frank Bonnet 10 Feb '05

10 Feb '05

Hi I have s strange problem with some entries into the /etc/netgroup file which does not seem to be cleared when replacing entries. it happen on a F87 running 6.5.2R1 -- Cordialement/Regards Frank Bonnet

1 0

RE: Diskless Boot of Windows/iSCSI
by Derek Lai 09 Feb '05

09 Feb '05

We are looking at doing iSCSI boot of Windows for a different purpose - higher availability, maybe in place of clustering which is alot more expensive and complex. We have a production SQLServer which management does not feel is necessary to go through the complexity and cost to do a cluster. However, they are intersted in <30 min SLA. We currently have a spare server sitting around just in case. If we setup iSCSI boot of Windows then we can swap the spare server in place of the production server if the production server ever has hardware problem, right? Has anyone done this with one of the iSCSI accelerators? I've only setup for iSCSI with MS SW initiator. But I would imagine the iSCSI id would be fully contained on the iSCSI accelerator and if I remove it from one server and place into another server with the same configuration then the new server will just boot up as the original server? Derek

2 1

RE: Diskless Boot of Windows/iSCSI
by Derek Lai 09 Feb '05

09 Feb '05

Jeff, No sure if you have gotten additional answers. But you might want to look into writable LUN. It is a LUN based on snapshot and can be changed but at some point you can just throw it away and off goes all the changes. It is pretty quick to create the writable LUN. Derek -----Original Message----- From: Jeff Mohler [mailto:jeff.mohler@signasys.com] Sent: Monday, February 07, 2005 6:58 PM To: Alan Biren; toasters(a)mathworks.com Subject: RE: Diskless Boot of Windows/iSCSI This fits 1/2 the model, the key is that the base load never changes..if the end-user changes something that could have messed the encoding process up somehow, a simple reboot would repair it. To replace the base LUNs, I would have to copy from the snapshot, would take time to copy that file from snapshot space to real space, correct? Ive got Gig to the tent switch (clean workspace on refuel floor) and 100Mbit from there to the PCs. Performance should be in the OK range. The critical function for these PCs is encoding video, once loaded up network traffic for apps or I/O would be small. Also looking at some solutions that manage disk images that will always force clients to a base boot image on a reboot, but Id like to keep the technology layer thin, and I already have netapps in the design. -----Original Message----- From: owner-toasters(a)mathworks.com [mailto:owner-toasters@mathworks.com]On Behalf Of Alan Biren Sent: Sunday, February 06, 2005 7:42 PM To: toasters(a)mathworks.com Subject: RE: Diskless Boot of Windows/iSCSI Jeff, As long as you have the base snapshot you can replace the boot LUNs quickly. You can also replicate this image to all of the field sites so that each are configured identically. The one major problem is the hardware has to be identical on each PC that will boot from SAN if you want to use the same image. You'll also need an iSCSI HBA. I would make sure that Network connectivity is reliable between the 250's and the PC's. I believe that the NOW site has instructions on seeting up a diskless system and the requirements to double check. _____ From: owner-toasters(a)mathworks.com on behalf of Jeff Mohler Sent: Sun 2/6/2005 5:19 PM To: toasters(a)mathworks.com Subject: Diskless Boot of Windows/iSCSI Greets: Same name, new location.. Im working a storage solution under contract using Netapp F250s in the field using snapmirror to replicate data back to a central F270 from nuclear plant inspection projects all over the world. I have 8 windows based PCs at each site doing encoding of camera inspections (IVVI), and those write to a file and/or go live over the network to the central location based on the inspection parameters. My largest source of problems are the encoding PCs which are not in the friendliest of environments on the refueling floor. Im looking for advice on whether I should change these PCs to work as diskless boot stations using ISCSI from the F250 that is at each site. The ultimate goal would be a situation where on boot the PC is booted to a static state of base configuration, and if the end-user changes anything (within MS Media encoder/etc) they just reboot to get back to a reliable configuration. The end user isnt storing any data in this image, just USING it to create data stored in a volume on the same F250 at the remote site. Just how diskless can I get, while maintaining an easy to use environment where nobody can screw up the base load image? My window of downtime is 10mins or less to full recovery. **************************************************************************** ************************* If you have received this email in error please notify the the system manager or the sender immediately and do not disclosed the to anyone or make copies. ** eSafe scanned this email for viruses, vandals or malicious content ** ** To find out eSafe(TM) and other Aladdin Security Products, contact AMC Corporation at (212) 736-0111 **************************************************************************** *************************

2 1

Tracking fpolicy/vscan contribution to filer load
by Hill, Aaron 08 Feb '05

08 Feb '05

Does anyone have some information on where to look for information on vscan and fpolicy contribution to filer load over time? I would think that this would be available within the perfstat output or perhaps even the Filer Performance monitor (DFM Client performance agent). Any assistance on expected statistics would be helpful. Regards, Aaron ************** IMPORTANT MESSAGE ************** This e-mail message is intended only for the addressee(s) and contains information which may be confidential. If you are not the intended recipient please advise the sender by return email, do not use or disclose the contents, and delete the message and any attachments from your system. Unless specifically indicated, this email does not constitute formal advice or commitment by the sender or the Commonwealth Bank of Australia (ABN 48 123 123 124) or its subsidiaries. We can be contacted through our web site: commbank.com.au. If you no longer wish to receive commercial electronic messages from us, please reply to this e-mail by typing Unsubscribe in the subject line. ***************************************************************

1 0

snaplocker utility with Veritas
by Bui, Marcus 08 Feb '05

08 Feb '05

Hi Toasters, I have an R200 with SnapLock Enterprise on ONTAP 6.5R2. A batch job runs every night to set the retention period against the data on a couple of WORM volume. The application that manages these data does not have the ability to set retention period, yet. Therefore, I have to run the Snaplocker.exe utility to manually set the retention period. The batch job as follow: snaplocker.exe \\Filer\volume <file:///\\Filer\volume%201h%203m%20\> 1h 3m /F << The /F switch is used because some files written/copied to the volume may already be 'read only'. The problem is with Veritas Netbackup Enterprise v4.5 incremental backup. All incremental backup is treated as a full backup since the batch job scans all files every time it runs. This somehow causes Veritas to think a file already there is a new/modified. And so, every backup job is a full. Any suggestions? Cheers, Marcus Hiep Bui >marcus.bui @ aiminvestments. com < ----------------------------------------- Confidentiality Note: The information contained in this message, and any attachments, may contain confidential and/or privileged material. It is intended solely for the person or entity to which it is addressed. Any review, retransmission, dissemination, or taking of any action in reliance upon this information by persons or entities other than the intended recipient is prohibited. If you received this in error, please contact the sender and delete the material from any computer.

2 1

RE: F740 memory upgrade to lower CPU utilization?
by Bui, Marcus 08 Feb '05

08 Feb '05

John, More RAM will help, you can never have too much of it. These are the items to look for: 1. Are there any volumes > %90 full? Volume over %90 can degrade performance, especially on busy volume. 2. For cifs you can use "cifs top" command ( "cifs.per_client_stats.enable" must be on). These will show the top cifs clients ops. There's also one for NFS on NOW. 3. use "statit" command. This will provide a lot of very good information of what's going on. 4. If you have DFM v3.0 or higher, use the DFM client that comes with it. It's has a lot of good performance stats. As for the replacement upgrade, I suggest talk to your local Netapp. There are many things to take into consideration to make a good recommendation. Good luck, Marcus Bui -----Original Message----- From: owner-toasters(a)mathworks.com [mailto:owner-toasters@mathworks.com] On Behalf Of Stewart, John Sent: Tuesday, February 08, 2005 12:51 PM To: 'toasters(a)mathworks.com' Subject: F740 memory upgrade to lower CPU utilization? We've got an F740 with 3 fully populated FC9 shelves, 36Gb disks, 432GB used (30 free). In the last couple of months the CPU has been getting higher and higher utilization. We've only got 512MB of RAM in there. Would an upgrade (to 1GB) be advisable? CPU isn't upgradable, is it? If we're pegging an F740 like this, and have 0.5TB of data now.... assuming a growth to 2TB in the next 3 years, what sort of box should we be looking at for a replacement (need to get the bean counters prepared)? thanks! johnS ----------------------------------------- Confidentiality Note: The information contained in this message, and any attachments, may contain confidential and/or privileged material. It is intended solely for the person or entity to which it is addressed. Any review, retransmission, dissemination, or taking of any action in reliance upon this information by persons or entities other than the intended recipient is prohibited. If you received this in error, please contact the sender and delete the material from any computer.

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

1999

1998

1997

toasters February 2005