We acquired a cluster of NetApp filers a few months ago with only an iSCSI license (no NFS or CIFS access), intending to use it for data volumes on our Linux and Windows hosts and later for virtualization. We have one production Gentoo system mounting a data volume from the filer using open-iscsi, and we have several other Gentoo and Ubuntu systems that we would like to configure that way as well.
We also have an Ubuntu system connected to a tape library. We would like to take snapshots of the data-volumes that need to be backed up, mount those snapshots on the backup host, and do an incremental backup from the mounted filesystem.
Probably we would partition each LUN with one Linux partition, and put an ext3 or reiserfs filesystem with a cluster-unique label on that partition. Then we could take a set of snapshots for the backup, map all the snapshot LUNs to the backup-host, do iSCSI discovery on the backup-host, mount each partition (listed in a configuration-file) with a "mount LABEL=..." command, perform the backup, unmount the partition, do an iSCSI logout, and delete all the snapshots.
Has anyone gotten a setup like this to work?
--
David Lee Lambert
Software Developer, Precision Motor Transport Group, LLC
517-349-3011 x223 (work) ... 586-873-8813 (cell)
I'm not 100% sure of this, but I don't think you can map and mount a snapshot itself. (I may be wrong, someone please let me know if I am) but I believe that you'll need a flexclone license to create a clone of that particular snapshot and then map the clone to your backup host.
On Wed, Nov 12, 2008 at 8:41 AM, David L. Lambert dlambert@bmtcarhaul.comwrote:
We acquired a cluster of NetApp filers a few months ago with only an iSCSI license (no NFS or CIFS access), intending to use it for data volumes on our Linux and Windows hosts and later for virtualization. We have one production Gentoo system mounting a data volume from the filer using open-iscsi, and we have several other Gentoo and Ubuntu systems that we would like to configure that way as well.
We also have an Ubuntu system connected to a tape library. We would like to take snapshots of the data-volumes that need to be backed up, mount those snapshots on the backup host, and do an incremental backup from the mounted filesystem.
Probably we would partition each LUN with one Linux partition, and put an ext3 or reiserfs filesystem with a cluster-unique label on that partition. Then we could take a set of snapshots for the backup, map all the snapshot LUNs to the backup-host, do iSCSI discovery on the backup-host, mount each partition (listed in a configuration-file) with a "mount LABEL=…" command, perform the backup, unmount the partition, do an iSCSI logout, and delete all the snapshots.
Has anyone gotten a setup like this to work?
*--***
*David Lee Lambert*
*Software Developer, Precision Motor Transport Group, LLC*
*517-349-3011 x223 (work) … 586-873-8813 (cell)*
That is mostly correct. Even though you don't want to write to the LUN, the underlying OS needs to think it can write to it in order to mount it (even if you mount it ro). Flex Clone solves this issue. If you don't have a flex clone license, you should be able to do a lun create -b with the snapshot name and create a snapshot-backed LUN which you could then map to the backup server and do your backup.
Either will work.
-- Adam Fox adamfox@netapp.com
From: Romeo Theriault [mailto:romeotheriault@gmail.com] Sent: Wednesday, November 12, 2008 10:11 AM To: David L. Lambert; Toasters Subject: Re: Mounting read-only snapshots using open-iscsi?
I'm not 100% sure of this, but I don't think you can map and mount a snapshot itself. (I may be wrong, someone please let me know if I am) but I believe that you'll need a flexclone license to create a clone of that particular snapshot and then map the clone to your backup host.
On Wed, Nov 12, 2008 at 8:41 AM, David L. Lambert dlambert@bmtcarhaul.com wrote:
We acquired a cluster of NetApp filers a few months ago with only an iSCSI license (no NFS or CIFS access), intending to use it for data volumes on our Linux and Windows hosts and later for virtualization. We have one production Gentoo system mounting a data volume from the filer using open-iscsi, and we have several other Gentoo and Ubuntu systems that we would like to configure that way as well.
We also have an Ubuntu system connected to a tape library. We would like to take snapshots of the data-volumes that need to be backed up, mount those snapshots on the backup host, and do an incremental backup from the mounted filesystem.
Probably we would partition each LUN with one Linux partition, and put an ext3 or reiserfs filesystem with a cluster-unique label on that partition. Then we could take a set of snapshots for the backup, map all the snapshot LUNs to the backup-host, do iSCSI discovery on the backup-host, mount each partition (listed in a configuration-file) with a "mount LABEL=..." command, perform the backup, unmount the partition, do an iSCSI logout, and delete all the snapshots.
Has anyone gotten a setup like this to work?
--
David Lee Lambert
Software Developer, Precision Motor Transport Group, LLC
517-349-3011 x223 (work) ... 586-873-8813 (cell)
On an instance of the NetApp Simulator, I was able to take a snapshot, export it via NFS, create a loopback device based on the snapshot, and mount the filesystem within it; something like:
Filer> lun create -b /vol/vol1/luns/lun11 -o noreserve /vol/vol1/luns/lun11_snapforback
Filer> lun share /vol/vol1/luns/lun11_snapforback read
Linux> mount 10.10.10.23:/vol/vol1 /mnt/netapp-vol1
Linux> losetup -f /mnt/nasim9-vol1/luns/lun11_snapforback
Linux> kpartx -a /dev/loop1
Linux> mount /dev/loop1p5 /mnt/lun11-snap
However, as I said, we don't have an NFS license; we've also heard that NFS performance is poorer than iSCSI performance. My guess is that some commands like the following ought to work:
Filer> lun map /vol/vol1/luns/lun11_snapforback linux-host 3
Linux> iscsiadm -m discovery --interface eth0 -t st -p 10.10.10.23 --login
Linux> iscsiadm -m node --interface eth0 -p 10.10.10.23 --login
Linux> mount /dev/sde /mnt/lun11-snap
When I do that, though, the set of available SCSI disk devices on the Linux host doesn't change. Perhaps this is an open-iscsi limitation instead?
To answer Jack's questions:
We had Bacula working on the backup-host, and it doesn't support NDMP. I don't think that would be any use for a LUN (which is just an opaque file to the NetApp), anyway.
Yes, we do want to avoid the overhead of the backup job on the host, and traffic across two network links when only one should be necessary. We also want to make as few software changes on the other hosts as possible.
We don't have a FlexClone license.
--
David Lee Lambert
Software Developer, Precision Motor Transport Group, LLC
517-349-3011 x223 (work) ... 586-873-8813 (cell)
________________________________
From: Romeo Theriault [mailto:romeotheriault@gmail.com] Sent: Wednesday, November 12, 2008 10:11 AM To: David L. Lambert; Toasters Subject: Re: Mounting read-only snapshots using open-iscsi?
I'm not 100% sure of this, but I don't think you can map and mount a snapshot itself. (I may be wrong, someone please let me know if I am) but I believe that you'll need a flexclone license to create a clone of that particular snapshot and then map the clone to your backup host.
On Wed, Nov 12, 2008 at 8:41 AM, David L. Lambert dlambert@bmtcarhaul.com wrote:
[...]
We also have an Ubuntu system connected to a tape library. We would like to take snapshots of the data-volumes that need to be backed up, mount those snapshots on the backup host, and do an incremental backup from the mounted filesystem.
Probably we would partition each LUN with one Linux partition, and put an ext3 or reiserfs filesystem with a cluster-unique label on that partition. Then we could take a set of snapshots for the backup, map all the snapshot LUNs to the backup-host, do iSCSI discovery on the backup-host, mount each partition (listed in a configuration-file) with a "mount LABEL=..." command, perform the backup, unmount the partition, do an iSCSI logout, and delete all the snapshots.
Hello all,
I have quick question about how Snapvault interacts with Snapmirror other than sharing a session count. Our environment is configured with multiple data filers that all snapmirror to a single filer, and snapvault to another single filer:
DataFiler1, DataFiler2, DataFiler3, etc. (snapmirror volume every 15 minutes to SnapmirrorFiler, snapvault qtree once a night to SnapvaultFiler)
If a snapmirror session is in progress for vol1 on Datafiler1, and a snapvault session tries to initiate for a qtree on vol1, which one will run? or both will?
Thanks! Scott
This message and any attachments (the "message") is intended solely for the addressees and is confidential. If you receive this message in error, please delete it and immediately notify the sender. Any use not in accord with its purpose, any dissemination or disclosure, either whole or partial, is prohibited except formal approval. The internet can not guarantee the integrity of this message. BNP PARIBAS (and its subsidiaries) shall (will) not therefore be liable for the message if modified. Please note that certain functions and services for BNP Paribas may be performed by BNP Paribas RCC, Inc.
I got mounting a snapshot to work. I needed to do "iscsiadm ... --rescan", following this hint:
http://www.mail-archive.com/open-iscsi@googlegroups.com/msg01072.html
We have not implemented this with an actual NetApp yet, so I have no idea whether performance will be acceptable; but thanks to all for the advice.
--
David Lee Lambert
Software Developer, Precision Motor Transport Group, LLC
517-349-3011 x223 (work) ... 586-873-8813 (cell)
________________________________
From: owner-toasters@mathworks.com [mailto:owner-toasters@mathworks.com] On Behalf Of David L. Lambert Sent: Wednesday, November 12, 2008 11:22 AM To: toasters@mathworks.com Subject: RE: Mounting read-only snapshots using open-iscsi?
[...]
However, as I said, we don't have an NFS license; we've also heard that NFS performance is poorer than iSCSI performance. My guess is that some commands like the following ought to work:
Filer> lun map /vol/vol1/luns/lun11_snapforback linux-host 3
Linux> iscsiadm -m discovery --interface eth0 -t st -p 10.10.10.23 --login
Linux> iscsiadm -m node --interface eth0 -p 10.10.10.23 --login
Linux> mount /dev/sde /mnt/lun11-snap
When I do that, though, the set of available SCSI disk devices on the Linux host doesn't change. Perhaps this is an open-iscsi limitation instead?
I hate to say it, because iSCSI is cool as hell and 'just works', but you're better off just using NFS if you can get a license... It ain't cheap tho. And FWIW, iSCSI could be faster than NFS, NFS is likely just as fast as iSCSI or faster in some cases.
Utilizing the lun create -b (as in your example below), re-mounting that newly exposed iSCSI LUN as a new device on LINUX then performing the backup is definitely the right idea. This is the 'norm' for most NetApp iSCSI environments (Exchange, SQL, etc).
Glenn
________________________________
From: owner-toasters@mathworks.com [mailto:owner-toasters@mathworks.com] On Behalf Of David L. Lambert Sent: Wednesday, November 12, 2008 4:29 PM To: toasters@mathworks.com Subject: RE: Mounting read-only snapshots using open-iscsi?
I got mounting a snapshot to work. I needed to do "iscsiadm ... --rescan", following this hint:
http://www.mail-archive.com/open-iscsi@googlegroups.com/msg01072.html
We have not implemented this with an actual NetApp yet, so I have no idea whether performance will be acceptable; but thanks to all for the advice.
--
David Lee Lambert
Software Developer, Precision Motor Transport Group, LLC
517-349-3011 x223 (work) ... 586-873-8813 (cell)
________________________________
From: owner-toasters@mathworks.com [mailto:owner-toasters@mathworks.com] On Behalf Of David L. Lambert Sent: Wednesday, November 12, 2008 11:22 AM To: toasters@mathworks.com Subject: RE: Mounting read-only snapshots using open-iscsi?
[...]
However, as I said, we don't have an NFS license; we've also heard that NFS performance is poorer than iSCSI performance. My guess is that some commands like the following ought to work:
Filer> lun map /vol/vol1/luns/lun11_snapforback linux-host 3
Linux> iscsiadm -m discovery --interface eth0 -t st -p 10.10.10.23 --login
Linux> iscsiadm -m node --interface eth0 -p 10.10.10.23 --login
Linux> mount /dev/sde /mnt/lun11-snap
When I do that, though, the set of available SCSI disk devices on the Linux host doesn't change. Perhaps this is an open-iscsi limitation instead?
I think you can mount an iscsi lun snapshot, but I would assume you would have to do a lun restore (license included) and manage it as a new lun (zoning) etc.
I assume your backup solution can't support / doesn't have an ndmp license?
I assume you have been backing up data via the host now and you are trying to eliminate the overhead of the backup job on that host?
On 11/12/08, David L. Lambert dlambert@bmtcarhaul.com wrote:
We acquired a cluster of NetApp filers a few months ago with only an iSCSI license (no NFS or CIFS access), intending to use it for data volumes on our Linux and Windows hosts and later for virtualization. We have one production Gentoo system mounting a data volume from the filer using open-iscsi, and we have several other Gentoo and Ubuntu systems that we would like to configure that way as well.
We also have an Ubuntu system connected to a tape library. We would like to take snapshots of the data-volumes that need to be backed up, mount those snapshots on the backup host, and do an incremental backup from the mounted filesystem.
Probably we would partition each LUN with one Linux partition, and put an ext3 or reiserfs filesystem with a cluster-unique label on that partition. Then we could take a set of snapshots for the backup, map all the snapshot LUNs to the backup-host, do iSCSI discovery on the backup-host, mount each partition (listed in a configuration-file) with a "mount LABEL=..." command, perform the backup, unmount the partition, do an iSCSI logout, and delete all the snapshots.
Has anyone gotten a setup like this to work?
--
David Lee Lambert
Software Developer, Precision Motor Transport Group, LLC
517-349-3011 x223 (work) ... 586-873-8813 (cell)
Hi all, We are pretty new to using iSCSI here. I have another question about backing up the LUNs. My backup software uses NDMP (with direct attached drives on the filer) and that's how I do the other volumes on this filer.
But the LUNs are just one big file. If I want to do weekly full and daily incremental, how does that work ? NDMP/dump is file level, so each night an incremental will really be a full.
Is a client side backup program my only solution to get file based backups from a LUN ? The Lun will hold an NTFS file system.
Thanks.
Paul
specifics:
Windows machine, using iSCSI initiator mounts LUN on filer as D drive. I need to do file level backup/restore of this D drive.
Don't know if this setup will work for you but there was a good discussion about how someone else backs up lun the other day on the mailing list. The thread can be found here:
http://www.nabble.com/Mounting-read-only-snapshots-using-open-iscsi--td20460...
Romeo
On Wed, Nov 19, 2008 at 9:30 AM, Paul Letta letta@jlab.org wrote:
Hi all, We are pretty new to using iSCSI here. I have another question about backing up the LUNs. My backup software uses NDMP (with direct attached drives on the filer) and that's how I do the other volumes on this filer. But the LUNs are just one big file. If I want to do weekly full and daily incremental, how does that work ? NDMP/dump is file level, so each night an incremental will really be a full. Is a client side backup program my only solution to get file based backups from a LUN ? The Lun will hold an NTFS file system.
Thanks.
Paul
specifics:
Windows machine, using iSCSI initiator mounts LUN on filer as D drive. I need to do file level backup/restore of this D drive.
Paul> We are pretty new to using iSCSI here. I have another question Paul> about backing up the LUNs. My backup software uses NDMP (with Paul> direct attached drives on the filer) and that's how I do the Paul> other volumes on this filer.
Paul> But the LUNs are just one big file. If I want to do weekly full Paul> and daily incremental, how does that work ? NDMP/dump is file Paul> level, so each night an incremental will really be a full.
Paul> Is a client side backup program my only solution to get file Paul> based backups from a LUN ? The Lun will hold an NTFS file Paul> system.
Paul> Windows machine, using iSCSI initiator mounts LUN on filer as D drive. Paul> I need to do file level backup/restore of this D drive.
Yes, in this case to get file based backups you need to have the client run some backup software.
John
On 11/19/08 8:50 AM, "John Stoffel" john.stoffel@taec.toshiba.com wrote:
Paul> We are pretty new to using iSCSI here. I have another question Paul> about backing up the LUNs. My backup software uses NDMP (with Paul> direct attached drives on the filer) and that's how I do the Paul> other volumes on this filer.
Paul> But the LUNs are just one big file. If I want to do weekly full Paul> and daily incremental, how does that work ? NDMP/dump is file Paul> level, so each night an incremental will really be a full.
Paul> Is a client side backup program my only solution to get file Paul> based backups from a LUN ? The Lun will hold an NTFS file Paul> system.
Paul> Windows machine, using iSCSI initiator mounts LUN on filer as D drive. Paul> I need to do file level backup/restore of this D drive.
Yes, in this case to get file based backups you need to have the client run some backup software.
John
I haven't played with this myself, but don't see any reason why it wouldn't work (feel free to pipe up if anyone sees any problems with it) - you could probably use OSSV (Open Systems Snap Vault) on the client (ie: inside the lun) and still use the netapp as a destination.
I think you understand it well. If you back up at the storage side, you will pretty much only do full backups of your LUNs. If you backup from the host, you can incrementals.
-- Adam Fox adamfox@netapp.com
-----Original Message----- From: Paul Letta [mailto:letta@jlab.org] Sent: Wednesday, November 19, 2008 9:30 AM To: toasters@mathworks.com Subject: Backups of iscsi luns
Hi all, We are pretty new to using iSCSI here. I have another question about
backing up the LUNs. My backup software uses NDMP (with direct attached
drives on the filer) and that's how I do the other volumes on this filer.
But the LUNs are just one big file. If I want to do weekly full and daily incremental, how does that work ? NDMP/dump is file level, so each night an incremental will really be a full.
Is a client side backup program my only solution to get file based backups from a LUN ? The Lun will hold an NTFS file system.
Thanks.
Paul
specifics:
Windows machine, using iSCSI initiator mounts LUN on filer as D drive. I need to do file level backup/restore of this D drive.
-
David L. Lambert -
Fox, Adam -
Glenn Walker -
Jack Lyons -
John Stoffel -
Nicholas Bernstein -
Paul Letta -
Romeo Theriault -
scott.belisle＠americas.bnpparibas.com