toasters

toasters@lists.teaparty.net

2 participants
13529 discussions

Diskless Boot of Windows/iSCSI
by Jeff Mohler 07 Feb '05

07 Feb '05

Greets: Same name, new location.. Im working a storage solution under contract using Netapp F250s in the field using snapmirror to replicate data back to a central F270 from nuclear plant inspection projects all over the world. I have 8 windows based PCs at each site doing encoding of camera inspections (IVVI), and those write to a file and/or go live over the network to the central location based on the inspection parameters. My largest source of problems are the encoding PCs which are not in the friendliest of environments on the refueling floor. Im looking for advice on whether I should change these PCs to work as diskless boot stations using ISCSI from the F250 that is at each site. The ultimate goal would be a situation where on boot the PC is booted to a static state of base configuration, and if the end-user changes anything (within MS Media encoder/etc) they just reboot to get back to a reliable configuration. The end user isnt storing any data in this image, just USING it to create data stored in a volume on the same F250 at the remote site. Just how diskless can I get, while maintaining an easy to use environment where nobody can screw up the base load image? My window of downtime is 10mins or less to full recovery.

2 1

Re: question about copying an Oracle instance
by Alan.McLachlan＠didata.com.au 07 Feb '05

07 Feb '05

Hi Steve, Where no updates are required, people have been doing what you describe with NetApp for over half a decade, by simply applying queries to previous night's snapshots. Where updates prior to querying are required (as you describe) the answer is different depending on whether you are using file or block I/O. Block I/O (iSCSI and FCP) has had read/write snapshot support for some time, by virtue of block-level virtualisation and indirection in a container file on top of WAFL. File I/O (NFS, CIFS) has read/write snapshot support in Ontap 7G. This is basically the same, at least in effect, as what you describe. It's acheived using block-level virtualisation and indirection underneath WAFL. Your DBA's will argue against using snapshots on the basis of disk contention, and in extreme environments this is something to be careful of. Generally though the read cache will take care of this sufficiently and they will not notice a significant hit. Where contention becomes an issue, clones are more appropriate. NetApp do a very good job at hiding contention compared to most storage solutions - temporal locality in writes to reduce seek overhead makes quite a difference to smoothing out read cache loading (I'm going a little too deep here... enough to say it works well enough that sharing snapshots for reporting on NetApp doesn't cause anywhere near the hit to production that it does on other vendor arrays). Whether you should be using file or block I/O depends on the nature of your database. In general, smaller transactions perform better with file I/O, larger records perfom better with block I/O. (opposite of traditional thinking). This is due to the metadata and CPU overhead versus the actual payload - for larger I/O's (i.e. binary large object blocks) these overheads are proportionately small and block I/O streams better. For small I/O's the opposite is true and NFS is a good solution - plus with a hard mount it is very robust (cable disconnect for extend periods won't crash the database). It sounds like your DBA's are fairly traditional and will resist a modern analysis and modern recommendations - they will still want to feel that they are managing the I/O characteristics of the disk solution. This is where EMC is appealing to many old-fashioned DBA's, as it allows them to retain a hands on approach to the disk configuration. They may have a hard time swallowing that NetApp's virtualise-everything approach can actually lead to better performance in some circumstances, and that even where it doesn't the management benefits more than make up for any small negative difference in I/O throughput. They still have an even harder time swallowing NFS mount, even when you tell them that Oracle themselves mount many of their in-house databases that way. DBA's in general today need to start learning to leave disk management to storage people and concentrate on their applications - they tend to get their backs up when this is suggested to them.... What they need to realise is that they aren't dealing with host-managed JBOD anymore and modern storage subsystems take care of I/O contention without the traditional manual disk tuning. The value a true DBA brings is in writing SQL, not partitioning hard disks. This is where Oracle Consulting Services (OCS) can step in and help - the argument carries more weight when it comes straight "from the horses mouth". Hoe this helps, Alan McLachlan ___________________________________ Solution Architect - Data Centre Solutions Dimension Data Australia Alan.McLachlan(a)didata.com.au Tel. +61 (0)2 61225123 Fax +61 (0)2 62486346 Mobile 0428 655644 Steve Losen <scl(a)sasha.acc.virginia.edu> Sent by: owner-toasters(a)mathworks.com 05/02/2005 12:47 AM To: toasters(a)mathworks.com cc: Subject: question about copying an Oracle instance We currently have our Oracle stuff running on Solaris boxes with (essentially) locally attached raid. We also have a clustered pair of Netapp filers that we use for home directories and other NAS storage. And our email system is running on Solaris with locally attached raid. We are looking into replacing our aging filers and locally attached raid disks with some kind of SAN/NAS solution. I'm the filer admin, so naturally I would like to see us buy Netapp. But I think the Oracle folks are going to push for EMC. Our Oracle folks have this situation where they need to make a copy of a 500G production Oracle instance every night and ship it to a data warehouse server. Then they do some automatic updates on the warehouse instance and make it available for querying. I know that both Netapp and EMC have efficient ways of handling this where you only need to copy part of the data. For example, on a Netapp we could use snapmirror and EMC claims to have similar functionality. However, the Oracle folks want to know what a worst case scenario would be, where you simply must copy all the data rather than just snapmirror the changes. It looks like EMC would probably be able to handle this sort of bulk copy faster than a comparable filer. But I was thinking that Netapp might have a better way to accomplish this. I don't know if this is supported or not, but theoretically WAFL should be able to handle it. Instead of actually copying the Oracle instance data (which would be either a 500G LUN or a 500G NFS file) you create a virtual copy. To do this, you would allocate a new inode for the copy, but link in the same data blocks as the original inode. This would be MUCH faster than copying the data, practically instantaneous. Of course the two copies would be in the same volume. Because WAFL always makes updates on fresh blocks, never changing existing data blocks, it would be possible to run the production instance on the original file and the warehouse instance on the virtual copy. Updates to one file would have no effect on the other. Once a day you could remove the warehouse instance and start over with a new "virtual copy". Maybe Netapp already supports something like this. But if not, they should think about it. I think it would be very handy in the database world where people need to run tests on production data, but are loathe to test on their production instance. I can see one problem right off the bat. Linking the same data block into more than one file is usually considered a filesystem no-no. So WAFL would need to be modified to support this, which might require some tricky bookkeeping. Perhaps each data block would need a file reference count, to indicate how many files it belongs to. If Netapp already supports something like this, I need to tell our folks who like EMC. If the powers that be choose EMC, how difficult will the EMC NAS product make my life? I realize this is a Netapp-centric list, so opinions will be biased. Steve Losen scl(a)virginia.edu phone: 434-924-0640 University of Virginia ITC Unix Support ****************************************************************************** - NOTICE FROM DIMENSION DATA AUSTRALIA This message is confidential, and may contain proprietary or legally privileged information. If you have received this email in error, please notify the sender and delete it immediately. Internet communications are not secure. You should scan this message and any attachments for viruses. Under no circumstances do we accept liability for any loss or damage which may result from your receipt of this message or any attachments. ******************************************************************************

1 0

Heyz Darling
by Florence Gunn 06 Feb '05

06 Feb '05

1 0

Internet Pharma!cy
by Violet Donovan 05 Feb '05

05 Feb '05

1 0

Your Match
by Sang Gray 05 Feb '05

05 Feb '05

1 0

Cheating House Wife Services
by Cornelia Brady 05 Feb '05

05 Feb '05

1 0

Snapvault - deleted source snapshot by accident
by John Stoffel 04 Feb '05

04 Feb '05

Hi all, We've got some filers, SRC (6.5.1R1P12) and DEST (6.5.2R1) which are doing snapvaults across a WAN. Someone deleted an active (I think) snapvault snapshot on the source filer to make space. At least they did delete a snapshot on the SRC filer which was our base snapshot for snapvault to the filer DEST. Currently, I'm seeing the following on the secondary: # rsh dest snapvault status -l /vol/vol2/src_vol3_proj Snapvault secondary is ON. Source: src:/vol/vol3/proj Destination: dest:/vol/vol2/src_vol3_proj Status: Quiescing Progress: - State: Snapvaulted Lag: 11:59:33 Mirror Timestamp: Fri Feb 4 01:00:01 PST 2005 Base Snapshot: dest(0050404441)_vol2-base.0 Current Transfer Type: Update Current Transfer Error: base snapshot for transfer no longer exists on the source Contents: Replica Last Transfer Type: Update Last Transfer Size: 1137360 KB Last Transfer Duration: 00:43:47 Last Transfer From: src:/vol/vol3/proj It's been hung like this for a while now, say 8 hours. The src filer now has a couple of hourly snapvault snapshots online: # rsh src snap list vol3 Volume vol3 working... %/used %/total date name ---------- ---------- ------------ -------- 0% ( 0%) 0% ( 0%) Feb 04 16:00 sv_hourly.0 0% ( 0%) 0% ( 0%) Feb 04 15:00 sv_hourly.1 Do I just need to wait for the secondary (an R200) to finish quiescing it's side of things before I can restart the snapvault process? Or do I need to re-do the entire base snapvault all over again? Thanks, John John Stoffel - Senior Staff Systems Administrator - System LSI Group Toshiba America Electronic Components, Inc. - http://www.toshiba.com/taec john.stoffel(a)taec.toshiba.com - 508-486-1087

2 2

fc woes continued : log digested
by Tavis Gustafson 04 Feb '05

04 Feb '05

######## scsi.cmd.checkCondition:error on .87,89 Fri Feb 4 11:29:54 GMT [scsi.cmd.checkCondition:error]: Device 8a.87: -- quiescing drive to allow outstanding I/O to complete. ######## FIRST MENTION OF FAILED DISK .87 ( DISK NOW LIVES IN SPARES POOL ) Fri Feb 4 11:30:20 GMT [raid.config.filesystem.disk.failed:error]: Disk -- [3HYY4K82000073408FS8] ####### scsi error continue on disk .89 Fri Feb 4 11:30:35 GMT [scsi.cmd.checkCondition:error]: Device 8a.89: Check Condition: CDB 0x2a:09cbc800:0080: Sense Data SCSI:aborted command - Fibre Channel frame CRC error (0xb - 0x47 0x0 0x3)(40387). ####### SUCCESSFUL RETRY ON DISK .87 Fri Feb 4 11:30:35 GMT [scsi.cmd.retrySuccess:info]: Device 8a.87: request successful after retry #3: cdb 0x2a:09cbc600:0080. ##### FIRST MENTION OF FC CRC FAME ERROR Fri Feb 4 11:30:36 GMT [ispfc_main:error]: Disk 8a.89: op 0x2a:09cbc800:0080 sector 164349952 SCSI:aborted command - Fibre Channel -- - Fibre Channel frame CRC error (0xb - 0x47 0x0 0x3)(41776) ###### FILER GIVES UP ON DISK .89. Fri Feb 4 11:30:36 GMT [scsi.cmd.noMorePaths:error]: Device 8a.89: No more paths to device: cdb 0x2a:09cbc800:0080. All retries have failed. ###### FIRST MENTION OF DISK 82 Fri Feb 4 11:30:55 GMT [scsi.cmd.checkCondition:error]: Device 8a.82: Check Condition: CDB 0x2a:09ac4b80:0080: Sense Data SCSI:aborted command -- - Fibre Channel frame CRC error (0xb - 0x47 0x0 0x3)(18075). ###### FIRST MENTION OF DISK 84 Fri Feb 4 11:30:58 GMT [scsi.cmd.checkCondition:error]: Device 8a.84: Check Condition: CDB 0x2a:09ac4c80:0080: Sense Data SCSI:aborted command -- - Fibre Channel frame CRC error (0xb - 0x47 0x0 0x3)(22992). ####### FIRST MENTION OF DISK 91 AND 90 (FC TIMEOUT) Fri Feb 4 11:31:00 GMT [ispfc_timeout_0:warning]: 8a.91 (0x0000005b) (0x034f4e10,0x2a:09cbcb80:0080,0/0,474/0/0,5192/0): command timeout, -- quiescing drive to allow outstanding I/O to complete. ######## GOING DOWN THE LIST NOW WITH 88 85 84 82 Fri Feb 4 11:31:00 GMT [ispfc_timeout_0:warning]: 8a.88 (0x00000058) (0x034f6f00,0x2a:09cbcb80:0080,0/0,387/0/0,5192/0): command timeout, -- request successful after retry #3: cdb 0x2a:09ac4c80:0080. ###### FIRST MENTION OF PASTTIMETOLIVE:ERROR Fri Feb 4 11:32:36 GMT [scsi.cmd.pastTimeToLive:error]: Device 8a.80: request failed after try #1: cdb 0x1c. -- request failed after try #1: cdb 0x2a:09cbcc00:0080. ###### FIRST MENTION OF WRITEERROR, DISKS 91 90 89 88 82 ( WILL STOP USING THEM!!??) Fri Feb 4 11:32:45 GMT [raid.label.io.writeError:error]: Label write on 8a.91 (S/N 3HYY4H2800007341NB79) failed with storage error 21. -- ######### THE DREADED MULTI-DISK ERROR PANIC: raid volfsm: vol boot: fatal multi-disk error. in process config_thread on release NetApp Release 6.4.5 on Fri Feb 4 11:32:46 GMT -- DUMPCORE: START #### GLOBAL TIMEOUT ( DISKS 92 86 85 84 83 81 80 ) .......................Fri Feb 4 11:32:46 GMT [config_thread:warning]: 8a.93 (0x0000005d) (0x03468f44,0x2a:109d43e8:0420,0/0,426/0/0,6197/0): -- ###############3 ##REBOOT #1 (SWAPPED MULTI-MODE CABLE BETWEEN LRC AND FC CARD) ###############3 ............................................................................................................................................ -- instance on primary side #### DISK 87, PREVIOUSLY FAILED, IS NOW ORPHANED Fri Feb 4 11:48:55 GMT [raid.assim.cls.notInCls:error]: Orphaning disk -- Loading volume boot ####### WAFL REPLAY FAILES AND PANICS THE FILER Replaying WAFL log wafl_verify_nvlog: replay failed twice in a row in log 0 request 8543 -- ............................................................................................................................................ ###### REBOOT - SWITCHED CABLE TO PORT B ON FC CARD ispfc: Fibre Channel adapter 8a appears to be unattached/disconnected. -- ore failure: ## MENTIONS DISK 80 ON A CHANNEL, MUST HAVE BEEN SOMETHING FROM LAST BOOT Error writing buffer to block 34873601 on disk 8a.80. Fri Feb 4 11:54:19 GMT [mgr.stack.saved:notice]: Reboot with saved panic information in log file #### MENTION 2 OF MULT-DISK ERROR ( PROBABLY FROM LAST BOOT ) Fri Feb 4 11:54:19 GMT [mgr.stack.string:notice]: Panic string: raid volfsm: vol boot: fatal multi-disk error. in process config_thread on -- config_thread ##### GOING TO REBUILD ON DISK 80 Fri Feb 4 11:54:20 GMT [raid.rg.recons.info:notice]: One disk is missing from RAID group /boot/plex0/rg0 -- prototype is 8b.93 ###### WEIRD STACK FRAME CRAP MENTIONS MULTIDISK ERROR Fri Feb 4 11:54:20 GMT [mgr.stack.framename:notice]: Stack frame 0: vol_multidisk_error(0xa2247c) + 0x5d -- is performed. ##### PASTTIMETOLIVE ERRORS RETURN ON DISK 80, 82,84, Fri Feb 4 12:13:59 GMT [scsi.cmd.pastTimeToLive:error]: Device 8b.80: request failed after try #1: cdb 0x1c. -- Check Condition: CDB 0x2a:09b24e98:0080: Sense Data SCSI:aborted command ##### MORE FC FRAME CRC ERRORS ( DISK 82 ) - Fibre Channel frame CRC error (0xb - 0x47 0x0 0x3)(57649). Fri Feb 4 12:14:23 GMT [scsi.cmd.checkCondition:error]: Device 8b.82: -- 10.3.38.28 was slow, op was v3 read, 63 > 60 (in seconds) ####### SCSI ERROR DISK 84 Fri Feb 4 12:14:48 GMT [scsi.cmd.checkCondition:error]: Device 8b.84: Check Condition: CDB 0x2a:09b25418:0080: Sense Data SCSI:aborted command -- - Fibre Channel frame CRC error (0xb - 0x47 0x0 0x3)(20148). ####### SCSI ERROR DISK 81 Fri Feb 4 12:14:53 GMT [ispfc_timeout_1:warning]: 8b.81 (0x01000051) (0x034f47b0,0x2a:09b25298:0080,0/0,20150/0/0,8028/0): command timeout, quiescing drive to allow outstanding I/O to complete. ###### SCSI DISK 80 Fri Feb 4 12:15:33 GMT [scsi.cmd.pastTimeToLive:error]: Device 8b.80: request failed after try #1: cdb 0x1c. -- Volume State Status Options #### PAST TOME TO LIVE ON DISKS 80 84 82 Fri Feb 4 12:16:19 GMT [scsi.cmd.pastTimeToLive:error]: Device 8b.80: request failed after try #1: cdb 0x1c. -- request failed after try #1: cdb 0x2a:09b25298:0080. #### FC TIMOUT ON DISKS 92 85 84 83 82 81 Fri Feb 4 12:16:49 GMT [ispfc_timeout_1:warning]: 8b.92 (0x0100005c) (0x034f4d00,0x2a:09b25698:0080,0/0,19912/0/0,8729/0): command timeout, -- #### FIRST MENTION OF SCSI ABORTED BY HOST DISK 92 Command aborted by host adapter: HA status 0x4: cdb 0x2a:09b25698:0080. Fri Feb 4 12:16:52 GMT [scsi.cmd.abortedByHost:error]: Device 8b.92: -- request failed after try #1: cdb 0x2a:09b25518:0080. ##### DEVICE RECOVERY DISK 84 Fri Feb 4 12:16:52 GMT [ispfc_timeout_1:error]: 8b.84 (0x01000054): global device timer timeout, initiating device recovery. ##### SCSI ABBORTED BY HOST DISK 85 Fri Feb 4 12:16:52 GMT [scsi.cmd.abortedByHost:error]: Device 8b.85: Command aborted by host adapter: HA status 0x4: cdb 0x2f:0025a800:0400. -- F #### MULTI-DISK ERROR #3 PANIC: raid volfsm: vol boot: fatal multi-disk error. in process config_thread on release NetApp Release 6.4.5 on Fri Feb 4 12:17:25 GMT -- 2032 to 2048MB ##### BOOT IN READONLY Intel Open Firmware by FirmWorks -- EMS NVRAM cache ### DISK FAILED ERROR DISK 89 (SECOND FAILED DISK) Fri Feb 4 12:17:25 GMT [raid.config.filesystem.disk.failed:error]: Disk /boot/plex0/rg0/8b.89 Shelf 5 Bay 9 [NETAPP X236_SCHT6146F10 NA05] S/N -- Autosupport, please see: http://now.netapp.com/autosupport/Fri #### DISK RECONSTRUCTION FAILED (DUH, READONLY FILESYSTEM) Feb 4 12:37:23 GMT [asup.throttle.msgHistoryErr:error]: Problems tracking message for throttling: DISK RECONSTRUCTION FAILED

1 0

question about copying an Oracle instance
by Steve Losen 04 Feb '05

04 Feb '05

We currently have our Oracle stuff running on Solaris boxes with (essentially) locally attached raid. We also have a clustered pair of Netapp filers that we use for home directories and other NAS storage. And our email system is running on Solaris with locally attached raid. We are looking into replacing our aging filers and locally attached raid disks with some kind of SAN/NAS solution. I'm the filer admin, so naturally I would like to see us buy Netapp. But I think the Oracle folks are going to push for EMC. Our Oracle folks have this situation where they need to make a copy of a 500G production Oracle instance every night and ship it to a data warehouse server. Then they do some automatic updates on the warehouse instance and make it available for querying. I know that both Netapp and EMC have efficient ways of handling this where you only need to copy part of the data. For example, on a Netapp we could use snapmirror and EMC claims to have similar functionality. However, the Oracle folks want to know what a worst case scenario would be, where you simply must copy all the data rather than just snapmirror the changes. It looks like EMC would probably be able to handle this sort of bulk copy faster than a comparable filer. But I was thinking that Netapp might have a better way to accomplish this. I don't know if this is supported or not, but theoretically WAFL should be able to handle it. Instead of actually copying the Oracle instance data (which would be either a 500G LUN or a 500G NFS file) you create a virtual copy. To do this, you would allocate a new inode for the copy, but link in the same data blocks as the original inode. This would be MUCH faster than copying the data, practically instantaneous. Of course the two copies would be in the same volume. Because WAFL always makes updates on fresh blocks, never changing existing data blocks, it would be possible to run the production instance on the original file and the warehouse instance on the virtual copy. Updates to one file would have no effect on the other. Once a day you could remove the warehouse instance and start over with a new "virtual copy". Maybe Netapp already supports something like this. But if not, they should think about it. I think it would be very handy in the database world where people need to run tests on production data, but are loathe to test on their production instance. I can see one problem right off the bat. Linking the same data block into more than one file is usually considered a filesystem no-no. So WAFL would need to be modified to support this, which might require some tricky bookkeeping. Perhaps each data block would need a file reference count, to indicate how many files it belongs to. If Netapp already supports something like this, I need to tell our folks who like EMC. If the powers that be choose EMC, how difficult will the EMC NAS product make my life? I realize this is a Netapp-centric list, so opinions will be biased. Steve Losen scl(a)virginia.edu phone: 434-924-0640 University of Virginia ITC Unix Support

2 1

RE: OnTap 7 Aggregates
by Hill, Aaron 04 Feb '05

04 Feb '05

I have not read anything yet on a transition method from standalone volumes so they can be placed into an aggregate as flexvols? i.e. Without adding more disk and doing some snapmirror (volume or qtree). I guees I am talking about a conversion(transmutation?) into the aggregate rather than a migration. Please interrupt me right now if there is a way to do this. Say I have 10-12 volumes, all the same size, all 80% full, same raid sizes (not DP), same disk geometry and speed. I want to put them into one big aggregate. Can this be done without actually creating a new aggregate from new disks beforehand? I can see a stepwise approach, but I am not sure if this will be optimal. 1) Partially populate your new aggregate with enough disks to allow one traditional volume worth of data to be migrated to a new flexvol. 2) Snapmirror and free up the old volumes disks, add these to the aggregate. 3) Repeat until all traditional volumes have become flexvols in the new aggregate. The main problem I see here is that the data will not be evenly spread over the whole aggregate as intended. It will be localised in raid groups, which is obviously marginally better than the traditional volumes. Normal aging and WAFL re-allocation will probably very slowly spread the flexvol blocks throughout the raid groups, but my guess is that this will take a very long time. Perhaps this is unavoidable? Another question is how to deal with the complexity introduced when all your traditional volumes were snapmirrored to one or more other sites. Are we effectively talking a new snapmirror baseline initialization here for the flexvols? This is a nightmare because in a Disaster Recovery scenario we will essentially be exposed during the initialization period and the rate limiting step of a WAN link or snapmirror retrieve from tape. Has anyone thought this type of thing through or can point me to reference documents? Thanks, Aaron -----Original Message----- From: Skottie Miller [mailto:skottie@anim.dreamworks.com] Sent: Friday, 4 February 2005 8:45 AM To: Holland, William L Cc: John Stoffel; 'toasters(a)mathworks.com' Subject: Re: OnTap 7 Aggregates William, you might want to read this whitepaper: http://www.netapp.com/tech_library/ftp/3356.pdf which describes OnTap 7G (the marketing name for 7.x) and has several best practices listed for aggregates and flexVolumes. as John suggests, you want to make aggregates as big as (reasonably) possible, within the configuration rules and best practices. things like: - there are max (raw) limits on aggregate size, just like there are on traditional volumes. the on-line system configuration guide doesn't list them yet, so I don't know where the limits are. - don't mix different drive sizes in the same aggregate - don't mix different drive speeds (10K, 15K) in the same aggregate - don't mix different shelf speeds (1G, 2G) in the same aggregate - make full raid groups, each one max size. - use Raid_DP - let OnTap pick the drives to assign to raid groups, rather than doing it "by hand". use to be, on nearstore especially, you had to hand-craft raid groups to get performance and protection; Ontap is smarter about hardware topology now, and seems to go a good job at this. we're testing 7.0 RC on an old f87, which is fine for learning the software, tough for experimenting with large aggregates... for your example, 6 shelves of the same type, with drives of the same type and speed, you can make one big aggregate using raid_dp, raid-group size 16, and get 5 raid groups and 4 spares. -skottie John Stoffel wrote: > William> Starting to look ahead to the upcoming OnTap 7. I'm very > William> excited about the new aggregates and flexvols. What criteria > William> should one use in determining the number of aggregates to > William> create? For example, FAS960 with 6ea DS14Mk2 shelves filled > William> with 144GB FC drives. Would it be better to create one huge > William> aggregate or several smaller aggregates? From my > William> perspective, I see no practical use for Tradional volumes > William> after upgrading to OnTap 7 and would like to convert to > William> flexvols, but was wondering if anyone with any experience > William> with any of the OnTap 7 RC's could provide feedback on this. ************** IMPORTANT MESSAGE ************** This e-mail message is intended only for the addressee(s) and contains information which may be confidential. If you are not the intended recipient please advise the sender by return email, do not use or disclose the contents, and delete the message and any attachments from your system. Unless specifically indicated, this email does not constitute formal advice or commitment by the sender or the Commonwealth Bank of Australia (ABN 48 123 123 124) or its subsidiaries. We can be contacted through our web site: commbank.com.au. If you no longer wish to receive commercial electronic messages from us, please reply to this e-mail by typing Unsubscribe in the subject line. ***************************************************************

6 6

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

1999

1998

1997

toasters