toasters February 2003

toasters@lists.teaparty.net

104 participants
136 discussions

RE: Data ONTAP Upgrade
by Sam Schorr 04 Nov '03

04 Nov '03

For all the inquiring minds, I found this under the download section: ********************************************************************** DEBUG PATCH RELEASE 6.3.1D8 Based on release 6.3.1 ********************************************************************** This is a DEBUG patch, which means it has fixes that have not been released in a regular software version yet. These fixes, if validated, will be checked in and released in a future release after adequate QA and testing. Fixes contained in this patch release are: 85294 Ineffective nlm GRANTED requested (callback) 78177 Renaming a small number of files repeatedly causes WAFL dir size to grow 83697 CPU at 100% after 6.2.1 upgrade from 6.1.3R1, with low NFS Ops and Disk Ops 80286 Problem accessing ~snapshot directory when using the "nosymlink_strict_security" share option 82176 File access can fail when using a widelink that maps to a share that supports widelinks 78570 Slow FTP performance with clustering enabled. 80332 SNMP may stop responding after accessing miscGlobalStatus on FAS9xx appliances. 81905 kma.c crash in wafl_zombie_growfile_handler 83681 CIFS file owner may be set incorrectly 85651 CIFS SETUP fails with "DSA is unwilling to perform" 85653 can't set file attributes when restoring with Veritas NetBackup 81631 isp2100 not handling port_logged_out status correctly for CT requests 86862 Master filer panic during takeover - memory leak from dup_user_field() 86464 Volume SnapMirror from an F840,F880,F9XX to a platform other than F840,F880 or F9XX on restarting may lead to corruption on destination. -----Original Message----- From: Hill, Aaron [mailto:aaron.hill@navitaire.com] Sent: Sunday, February 16, 2003 2:14 PM To: 'Chris Blackmor' Cc: 'Paul McGuinness (ext. 771)'; 'toasters(a)mathworks.com' Subject: RE: Data ONTAP Upgrade Hey there, I welcome questions. Unfortunately, my answers may not be sufficient to satisfy your mental hunger. To be honest, I don't know exactly what the D8 does. I was informed that it fixes the snapmirror code. The error was found to occur only in our "specific" circumstance. So what was/is our circumstance? 1) F880 source snapmirroring 3 batch area volumes to an F820 target across a 20Mbit Ethernet WAN. 2) The amount of data change per day is high. Approximately 10-40Gb per volume. 3) We were experiencing poor throughput across the WAN link (carrier issue) and had to frequently abort the snapmirror and delete snapshots from the F880 source in order to prevent snapshot space over-running production data space. This throughput has since been addressed (before D8). 4) The F820 target panicked a couple of times and dumped to the ok prompt. One of the target volumes was found to be corrupt. It was difficult to say if the source filer caused it or the target filer developed the corruption independently. We had some cases open and ran wafl checks and scans on both filers as a result. No errors were turned up. 5) The crash dump file was ftp'd to NetApp. This lead to the 6.3.1D8 debug release. When I get some more time, I might read into it a bit further. For now, I am more than happy that they have addressed the issue and fixed it very quickly. We have not had any problems with 6.3.1 or the D8 releases. Thanks, Aaron -----Original Message----- From: Chris Blackmor [mailto:chris.blackmor@amd.com] Sent: Sunday, February 16, 2003 6:48 AM To: Hill, Aaron Cc: 'Paul McGuinness (ext. 771)'; 'toasters(a)mathworks.com' Subject: Re: Data ONTAP Upgrade What does the D8 fix for snapmirror? Inquiring minds want to know. ;) C- On Fri, Feb 14, 2003 at 07:18:19PM -0600, Hill, Aaron wrote: > We are running 6.3.1 on one filer (F880) and 6.3.1D8 (debug release for a > snapmirror problem we were having) on another (F820). 2 others still running > 6.2.2 I believe. > > As far as I know, all new F960s being shipped have 6.3.1 on them. > > So overall, I have to say I recommend it. If you are snapmirroring, make > sure you put 6.3.1D8 on your destination filer, that is the advice from our > NetApp representative. > > Thanks, > Aaron Hill > Systems Engineer > www.navitaire.com > > > -----Original Message----- > From: Paul McGuinness (ext. 771) [mailto:Paul.McGuinness@fineos.com] > Sent: Saturday, February 15, 2003 2:58 AM > To: 'toasters(a)mathworks.com' > Subject: Data ONTAP Upgrade > > Hi, > I have a F720 currently running on Data ONTAP 6.1.2R2 and everything is > running fine. > My question is to those of you 6.6.2 (the current GA release) and those of > you on 6.3.1. > > I was going to upgrade to 6.2.2, just to keep my system up to date, but > according to the NOW site there are a good few more active systems running > 6.3.1. > > Would 6.3.1 be a better upgrade to go to? I also see that there is now a 6.4 > first customer shipment release (12th Feb) but I think I'll leave that for > those of you who have filers to play with.... > :o) > > So 6.6.2 or 6.3.1? > > > > > > ************************************************************************** > The information contained in this e-mail is confidential, > may be privileged and is intended only for the use of the > recipient named above. If you are not the intended > recipient or a representative of the intended recipient, > you have received this e-mail in error and must not copy, > use or disclose the contents of this email to anybody > else. If you have received this e-mail in error, please > notify the sender immediately by return e-mail and > permanently delete the copy you received. This email has > been swept for computer viruses. However, you should > carry out your own virus checks. > > > Registered in Ireland, No. 205721. http://www.FINEOS.com > ************************************************************************** -- -------------------------------------------------------------------- * Chris Blackmor _______ | * * Advanced Micro Devices \____ | | Toto... I've * * Phone: (512) 602-1608 /| | | | a feeling we're * * Fax: (512) 602-5155 | |___| | | not in Kansas * * Email: chris.blackmor(a)amd.com |____/ \| | anymore. * -------------------------------------------------------------------- * My comments are mine, and mine alone. * --------------------------------------------------------------------

2 1

wafl scan
by Ambrose_Earle＠shamrockfoods.com 06 Mar '03

06 Mar '03

Does anyone have any (good) experience with the wafl scan commands? I am testing Snapmirror and used it to mirror 7x34's to 10x34's. Of course, the data was only written to the first seven drives, so I used 'wafl scan reallocate <volume>' and it seemed to improve the layout quite dramatically. Is this a good use of the command? Other than affecting performance, I'm assuming everything is accessible during the run of this command? Thanks in advance,

2 1

Snapvault and 6.4
by Frank Gilmurray 28 Feb '03

28 Feb '03

Anyone using snapvault w/ 6.4? We use snapvault to update a R100 and it was working fine until we upgraded to 6.4 yesterday. We upgraded to get the 255 snapshot limit. I've opened a case on this but was wondering if anyone else has any ideas on what could be going wrong. The client snapshot is fine but the server snapshot has yesterday's contents even though it says it was xfered this morning at 2:05. A directory I created yesterday shows up in the client snapshot but not in the server snapshot. None of the qtrees were updated last night. Thanks, Frank -- Frank Gilmurray Release Engineer Nokia, Inc. phone: (650) 625-2179 313 Fairchild Drive, Blgd. A cell: (650) 892-4029 Mountain View, CA 94043 email: Frank.Gilmurray(a)nokia.com

1 0

R100 vol config for performance
by Mark Bentley 28 Feb '03

28 Feb '03

I'm already spreading disks in a volume across the 7 SCSI controllers, but I'm wondering if there is also a performance advantage/disadvantage to spreading out the disks in a volume across the three IDE-to-SCSI bridges in each R100 disk shelf? For example, if I wanted to create a volume with 3 disks, would it be better to pick three disks on the same bridge (like 3.0, 3.1, and 3.2) or would it be better to pick three across three bridges (like 3.0, 3.8, and 3.12) ? Or doesn't it make a difference? Thanks, Mark -- Mark Bentley TDG Unix Lab Ops AT&T Wireless Services Phone: 425-580-1667

2 3

RE: Filer's security information
by Kumar, Rahul 28 Feb '03

28 Feb '03

Please sync the time on netapp to the match it to the Samba Machines time. This should solve the error. Rahul Kumar, Infrastructure Analyst, EDS PLM Solutions, Pune, India. Tele: +91-20-2930812 Cell: +00-91-9422087523 Email: rahul.kumar(a)eds.com -----Original Message----- From: Frank Bonnet [mailto:bonnetf@bart.esiee.fr] Sent: Friday, February 28, 2003 4:18 PM To: toasters(a)mathworks.com Subject: Filer's security information Hi I'm STILL trying to use a Samba server 3.0 as a PDC at this point I have the following error message Fri Feb 28 11:42:00 CET [rc:CRITICAL]: CIFS - Filer's security information differs from domain controller. thanks for any infos -- Frank Bonnet

2 1

Filer's security information
by Frank Bonnet 28 Feb '03

28 Feb '03

Hi I'm STILL trying to use a Samba server 3.0 as a PDC at this point I have the following error message Fri Feb 28 11:42:00 CET [rc:CRITICAL]: CIFS - Filer's security information differs from domain controller. thanks for any infos -- Frank Bonnet

1 0

inodes/volumes/mildir/Courier IMAP
by George Kahler 28 Feb '03

28 Feb '03

Hi, I'm about to start a conversion project of our University email system based on mbox folders to Courier IMAP maildir. There are about 50K users using about 250 GB of space on F840 cluster; Netapp version 6.2.2. We have been also told that as of this year each student will keep their account for life, therefore this number will keep on growing. I'm very concern about the number of inodes/files/volume and the filer performance. From all the information I could find some of you said that you get about o one inode for every 32 K o can be increased to as much as 1 inode for every 4 K (maxfiles) There was also some discussion that when you do this, the internal WAFL tables become so huge that it will impact performance. 32 million / 50 K users gives me only about 650 files; this is clearly not enough. Will someone having gone thru this type of a conversion give me some pointers of what to do and what not to do ??? Thanks, George ------------------------------------------------------------------------------- George Kahler e-mail: george(a)yorku.ca Sr. Systems Administrator humans: (416) 736-2100 x.22699 Computing and Network Services machines: (416) 736-5830 Ontario, Canada, M3J-1P3

9 17

RE: iSCSI Buzz
by Alan McLachlan 28 Feb '03

28 Feb '03

hmmmm.... I guess so... The value add of SME is high for environments with very large Exchange databases and/or a very large number of mailboxes. If it's only a small Exchange DB and your issue is that you don't want it on a simple local disk, then just getting the iSCSI NIC and configuring a virtual disk on your existing filer is acceptable, and is supportable by Microsoft since it's "local" storage. But you will have to treat backup and restore options in the same manner as a simple local disk. I agree that the price of SME is far to high for small-to-mid size Exchange environments. -----Original Message----- From: Steve Evans [mailto:sevans@foundation.sdsu.edu] Sent: Friday, 28 February 2003 5:05 PM To: toasters(a)mathworks.com Subject: RE: iSCSI Buzz So what your saying is SME gives you Exchange snapshots. If you don't care about taking snapshots of your Exchange database then it doesn't matter. Steve Evans SDSU Foundation (619) 594-0653 -----Original Message----- From: Alan McLachlan [mailto:amclachlan@asi.com.au] Sent: Thursday, February 27, 2003 7:42 PM To: Jeff Burton; Steve Evans Cc: toasters(a)mathworks.com Steve, iSCSI is NOT a "replacement" for SnapManager for Exchange. iSCSI will let you treat a file on a filer as a block device. There ends its usefulness. In other words, without host-side tools like SnapDrive or SnapManager you aren't any better off than with a dumb local block device (except that writes will be nice and fast due to block indirection). There are three levels of data consistency: 1. Block level - this means that a physical block reported to have been written to disk has or will make it there intact. This is your typical DAS or SAN-level consistency. 2. Filesystem level - this means that all the blocks required for a consistent point-in-time image have been written successfully when the filesystem manger considers this to have been done. SnapDrive is NetApp's volume management plug-in for VLD, iSCSI and FCP "block" devices that manages this on a Windows host where that host is managing a NTFS volume laid down on the virtual block device. What happens is: you tell SnapDrive on the server to take a snapshot - it then tells the windows NTFS filesystem manager not to accept any more file I/O requests and to complete any outstanding I/O (flush writes) - then and only then does it tell the NetApp to take a WAFL-level snapshot of the volume hosting the virtual disk. It then tells the NTFS filesystem manager on the server that it can start processing requests again. If you simply took a NetApp level snapshot of the volume holding the virtual disk, then the image of the virtual disk could internally be in an inconsistent state, similar to a dirty shutdown of the Windows server. 3. Application level - this means that all of the file I/O OR block I/O operations required to keep an applications data files consistent from an application perspective (bearing in mind that the application will be managing outstanding operations in RAM) are completed at the times when the application expects, such as when you shut the application down or when you send a request through it's backup API to backup or restore it's data. Raw iSCSI gives you level 1 consistency only. There is no facility for taking consistent point in time snapshots that you can use for online backups. You can use filer level snapshots if you like, but there's arisk that you won't be able to mount the snapshotted volume without some corruption. SnapDrive gives you level 2 consistency. This is better and works will with databases that have a hot backup mode, like Oracle, or where you don't mind halting the database for a few seconds so that you don't have to deal with log files. To use this with Exchange you still have to manually manger restores with integrity checks etc. SnapManager provides Exchange-specific functions through the Exchange backup API to allow you to use snapshots to manage online backups etc. This gives you level 3 consistency for your Exchange backups as well as super fast restores from snapshots etc. iSCSI is good, but using it by itself is like taking the VLD component of SnapDrive or SnapManager without the rest. -----Original Message----- From: Jeff Burton [mailto:burtonj@pprd.abbott.com] Sent: Friday, 28 February 2003 8:57 AM To: Steve Evans Cc: toasters(a)mathworks.com Subject: Re: iSCSI Buzz On Mon, Feb 24, 2003 at 10:12:44AM -0800, Steve Evans wrote: > I've been listening to all the iSCSI buzz with NetApp in the news. > From what I've gathered all you need is OnTAP 6.4 and a $700 adapter > for the host server. > > What are people planning on doing with this. I would like to use it > with Exchange 2000 and not purchase the SnapManager for Exchange > option (at $20,000). > > Steve Evans > SDSU Foundation > (619) 594-0653 Per: http://now.netapp.com/NOW/products/iSCSI/ you also need a license for iSCSI, but it sounds like it's free as long as your filer's software support agreement is active. I was surprised to find the license free, so I may be missing something. We do not have any plans to use iSCSI anywhere in our environment anytime soon. I seem to remember Gartner Group stating that iSCSI will not become popular until late 2004 or later. -- ______________________________________________________________ Jeff Burton Storage Architect Abbott Laboratories EMAIL: Jeffrey.Burton(a)Abbott.com Dept. GB16/Bldg. AP9A-LL PHONE: 847-935-5778 100 Abbott Park Rd. FAX: 847-935-0142 Abbott Park, IL 60064-6115 **** ASI Solutions Disclaimer **** The material transmitted may contain confidential and/or privileged material and is intended only for the addressee. If you receive this in error, please notify the sender and destroy any copies of the material immediately. ASI will protect your Privacy according to the 10 Privacy Principles outlined under the new Privacy Act, Dec 2001. This email is also subject to copyright. Any use of or reliance upon this material by persons or entities other than the addressee is prohibited. E-mails may be interfered with, may contain computer viruses or other defects. Under no circumstances do we accept liability for any loss or damage which may result from your receipt of this message or any attachments. **** END OF MESSAGE **** **** ASI Solutions Disclaimer **** The material transmitted may contain confidential and/or privileged material and is intended only for the addressee. If you receive this in error, please notify the sender and destroy any copies of the material immediately. ASI will protect your Privacy according to the 10 Privacy Principles outlined under the new Privacy Act, Dec 2001. This email is also subject to copyright. Any use of or reliance upon this material by persons or entities other than the addressee is prohibited. E-mails may be interfered with, may contain computer viruses or other defects. Under no circumstances do we accept liability for any loss or damage which may result from your receipt of this message or any attachments. **** END OF MESSAGE ****

1 0

RE: iSCSI Buzz
by Steve Evans 28 Feb '03

28 Feb '03

So what your saying is SME gives you Exchange snapshots. If you don't care about taking snapshots of your Exchange database then it doesn't matter. Steve Evans SDSU Foundation (619) 594-0653 -----Original Message----- From: Alan McLachlan [mailto:amclachlan@asi.com.au] Sent: Thursday, February 27, 2003 7:42 PM To: Jeff Burton; Steve Evans Cc: toasters(a)mathworks.com Steve, iSCSI is NOT a "replacement" for SnapManager for Exchange. iSCSI will let you treat a file on a filer as a block device. There ends its usefulness. In other words, without host-side tools like SnapDrive or SnapManager you aren't any better off than with a dumb local block device (except that writes will be nice and fast due to block indirection). There are three levels of data consistency: 1. Block level - this means that a physical block reported to have been written to disk has or will make it there intact. This is your typical DAS or SAN-level consistency. 2. Filesystem level - this means that all the blocks required for a consistent point-in-time image have been written successfully when the filesystem manger considers this to have been done. SnapDrive is NetApp's volume management plug-in for VLD, iSCSI and FCP "block" devices that manages this on a Windows host where that host is managing a NTFS volume laid down on the virtual block device. What happens is: you tell SnapDrive on the server to take a snapshot - it then tells the windows NTFS filesystem manager not to accept any more file I/O requests and to complete any outstanding I/O (flush writes) - then and only then does it tell the NetApp to take a WAFL-level snapshot of the volume hosting the virtual disk. It then tells the NTFS filesystem manager on the server that it can start processing requests again. If you simply took a NetApp level snapshot of the volume holding the virtual disk, then the image of the virtual disk could internally be in an inconsistent state, similar to a dirty shutdown of the Windows server. 3. Application level - this means that all of the file I/O OR block I/O operations required to keep an applications data files consistent from an application perspective (bearing in mind that the application will be managing outstanding operations in RAM) are completed at the times when the application expects, such as when you shut the application down or when you send a request through it's backup API to backup or restore it's data. Raw iSCSI gives you level 1 consistency only. There is no facility for taking consistent point in time snapshots that you can use for online backups. You can use filer level snapshots if you like, but there's arisk that you won't be able to mount the snapshotted volume without some corruption. SnapDrive gives you level 2 consistency. This is better and works will with databases that have a hot backup mode, like Oracle, or where you don't mind halting the database for a few seconds so that you don't have to deal with log files. To use this with Exchange you still have to manually manger restores with integrity checks etc. SnapManager provides Exchange-specific functions through the Exchange backup API to allow you to use snapshots to manage online backups etc. This gives you level 3 consistency for your Exchange backups as well as super fast restores from snapshots etc. iSCSI is good, but using it by itself is like taking the VLD component of SnapDrive or SnapManager without the rest. -----Original Message----- From: Jeff Burton [mailto:burtonj@pprd.abbott.com] Sent: Friday, 28 February 2003 8:57 AM To: Steve Evans Cc: toasters(a)mathworks.com Subject: Re: iSCSI Buzz On Mon, Feb 24, 2003 at 10:12:44AM -0800, Steve Evans wrote: > I've been listening to all the iSCSI buzz with NetApp in the news. > From what I've gathered all you need is OnTAP 6.4 and a $700 adapter > for the host server. > > What are people planning on doing with this. I would like to use it > with Exchange 2000 and not purchase the SnapManager for Exchange > option (at $20,000). > > Steve Evans > SDSU Foundation > (619) 594-0653 Per: http://now.netapp.com/NOW/products/iSCSI/ you also need a license for iSCSI, but it sounds like it's free as long as your filer's software support agreement is active. I was surprised to find the license free, so I may be missing something. We do not have any plans to use iSCSI anywhere in our environment anytime soon. I seem to remember Gartner Group stating that iSCSI will not become popular until late 2004 or later. -- ______________________________________________________________ Jeff Burton Storage Architect Abbott Laboratories EMAIL: Jeffrey.Burton(a)Abbott.com Dept. GB16/Bldg. AP9A-LL PHONE: 847-935-5778 100 Abbott Park Rd. FAX: 847-935-0142 Abbott Park, IL 60064-6115 **** ASI Solutions Disclaimer **** The material transmitted may contain confidential and/or privileged material and is intended only for the addressee. If you receive this in error, please notify the sender and destroy any copies of the material immediately. ASI will protect your Privacy according to the 10 Privacy Principles outlined under the new Privacy Act, Dec 2001. This email is also subject to copyright. Any use of or reliance upon this material by persons or entities other than the addressee is prohibited. E-mails may be interfered with, may contain computer viruses or other defects. Under no circumstances do we accept liability for any loss or damage which may result from your receipt of this message or any attachments. **** END OF MESSAGE ****

1 0

RE: iSCSI Buzz
by Alan McLachlan 28 Feb '03

28 Feb '03

Steve, iSCSI is NOT a "replacement" for SnapManager for Exchange. iSCSI will let you treat a file on a filer as a block device. There ends its usefulness. In other words, without host-side tools like SnapDrive or SnapManager you aren't any better off than with a dumb local block device (except that writes will be nice and fast due to block indirection). There are three levels of data consistency: 1. Block level - this means that a physical block reported to have been written to disk has or will make it there intact. This is your typical DAS or SAN-level consistency. 2. Filesystem level - this means that all the blocks required for a consistent point-in-time image have been written successfully when the filesystem manger considers this to have been done. SnapDrive is NetApp's volume management plug-in for VLD, iSCSI and FCP "block" devices that manages this on a Windows host where that host is managing a NTFS volume laid down on the virtual block device. What happens is: you tell SnapDrive on the server to take a snapshot - it then tells the windows NTFS filesystem manager not to accept any more file I/O requests and to complete any outstanding I/O (flush writes) - then and only then does it tell the NetApp to take a WAFL-level snapshot of the volume hosting the virtual disk. It then tells the NTFS filesystem manager on the server that it can start processing requests again. If you simply took a NetApp level snapshot of the volume holding the virtual disk, then the image of the virtual disk could internally be in an inconsistent state, similar to a dirty shutdown of the Windows server. 3. Application level - this means that all of the file I/O OR block I/O operations required to keep an applications data files consistent from an application perspective (bearing in mind that the application will be managing outstanding operations in RAM) are completed at the times when the application expects, such as when you shut the application down or when you send a request through it's backup API to backup or restore it's data. Raw iSCSI gives you level 1 consistency only. There is no facility for taking consistent point in time snapshots that you can use for online backups. You can use filer level snapshots if you like, but there's arisk that you won't be able to mount the snapshotted volume without some corruption. SnapDrive gives you level 2 consistency. This is better and works will with databases that have a hot backup mode, like Oracle, or where you don't mind halting the database for a few seconds so that you don't have to deal with log files. To use this with Exchange you still have to manually manger restores with integrity checks etc. SnapManager provides Exchange-specific functions through the Exchange backup API to allow you to use snapshots to manage online backups etc. This gives you level 3 consistency for your Exchange backups as well as super fast restores from snapshots etc. iSCSI is good, but using it by itself is like taking the VLD component of SnapDrive or SnapManager without the rest. -----Original Message----- From: Jeff Burton [mailto:burtonj@pprd.abbott.com] Sent: Friday, 28 February 2003 8:57 AM To: Steve Evans Cc: toasters(a)mathworks.com Subject: Re: iSCSI Buzz On Mon, Feb 24, 2003 at 10:12:44AM -0800, Steve Evans wrote: > I've been listening to all the iSCSI buzz with NetApp in the news. From > what I've gathered all you need is OnTAP 6.4 and a $700 adapter for the > host server. > > What are people planning on doing with this. I would like to use it > with Exchange 2000 and not purchase the SnapManager for Exchange option > (at $20,000). > > Steve Evans > SDSU Foundation > (619) 594-0653 Per: http://now.netapp.com/NOW/products/iSCSI/ you also need a license for iSCSI, but it sounds like it's free as long as your filer's software support agreement is active. I was surprised to find the license free, so I may be missing something. We do not have any plans to use iSCSI anywhere in our environment anytime soon. I seem to remember Gartner Group stating that iSCSI will not become popular until late 2004 or later. -- ______________________________________________________________ Jeff Burton Storage Architect Abbott Laboratories EMAIL: Jeffrey.Burton(a)Abbott.com Dept. GB16/Bldg. AP9A-LL PHONE: 847-935-5778 100 Abbott Park Rd. FAX: 847-935-0142 Abbott Park, IL 60064-6115 **** ASI Solutions Disclaimer **** The material transmitted may contain confidential and/or privileged material and is intended only for the addressee. If you receive this in error, please notify the sender and destroy any copies of the material immediately. ASI will protect your Privacy according to the 10 Privacy Principles outlined under the new Privacy Act, Dec 2001. This email is also subject to copyright. Any use of or reliance upon this material by persons or entities other than the addressee is prohibited. E-mails may be interfered with, may contain computer viruses or other defects. Under no circumstances do we accept liability for any loss or damage which may result from your receipt of this message or any attachments. **** END OF MESSAGE ****

1 0

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

1999

1998

1997

toasters February 2003