I am checking on this
Sim-01> df -gm
Filesystem total used avail capacity Mounted on
/vol/vol0/ 766MB 672MB 94MB 88% /vol/vol0/
/vol/vol0/.snapshot 40MB 0MB 40MB 0% /vol/vol0/.snapshot
/vol/vol101/ 19MB 0MB 18MB 1% /vol/vol101/
/vol/vol101/.snapshot 1MB 0MB 0MB 61% /vol/vol101/.snapshot
/vol/svm101_root/ 19MB 0MB 18MB 1% /vol/svm101_root/
/vol/svm101_root/.snapshot 1MB 0MB 0MB 58% /vol/svm101_root/.snapshot
Last time when I had this issue, it was different and I turned off all snapshot. Looks like 88%, there is something , somewhere this is getting filled up
Hmmmm…..nothing inside volume of vserver ??? Interesting…
Sim::volume*> ?
Sim::volume*> help
Press "?" or Tab for help.
Sim::volume*> cd ..
Sim::*> vserver
Sim::vserver*> ?
audit> Manage auditing of protocol requests that the Vserver services
check> The check directory
cifs> Manage the CIFS configuration of a Vserver
nfs> Manage the NFS configuration of a Vserver
services> The services directory
vscan> Manage Vscan
From: tmac [mailto:tmacmd@gmail.com] Sent: Sunday, July 26, 2015 10:37 AM To: Uddhav Regmi uregmi111@gmail.com Cc: Toasters Toasters@teaparty.net Subject: Re: unable to find master : Cdot : 8.3RC1
Looks like you inadvertently filled up your root volume.
Doing something there you should be....
May need to log in as diag user and clean up some space on /mroot
--tmac
Tim McCarthy
Principal Consultant
On Sun, Jul 26, 2015 at 10:23 AM, Uddhav Regmi <uregmi111@gmail.com mailto:uregmi111@gmail.com > wrote:
Hello all,
It just died all of a sudden. Automatic Recovery was off. Pls advise.
Cannot access the cluster level , node level is fine.
Sim::> event log show -severity emerGENCY
Time Node Severity Event
------------------- ---------------- ------------- ---------------------------
7/26/2015 09:57:50 Sim-01 EMERGENCY callhome.root.vol.recovery.reqd: Call home for ROOT VOLUME NOT WORKING PROPERLY: RECOVERY REQUIRED.
7/26/2015 09:56:36 Sim-01 EMERGENCY rdb.recovery.failed: Error: Unable to find a master.. Unable to recover the local database of Data Replication Module: Management.
2 entries were displayed.
Sim::> event log show -severity erroR
Time Node Severity Event
------------------- ---------------- ------------- ---------------------------
7/26/2015 10:00:51 Sim-01 ERROR mgmtgwd.ntp.job.failure: The ntp_resolve_job could not be started.
7/26/2015 09:56:39 Sim-01 ERROR spm.mgwd.process.exit: Management Gateway (mgwd) subsystem with ID 12774 exited as a result of signal signal 6. The subsystem will attempt to restart.
7/26/2015 09:56:36 Sim-01 ERROR ucore.panicString: 'mgwd: assertion (false) at corrupt_rdb.cc:587 failed, raising SIGABRT(6) at RIP 0x82e1ed7bc (pid 12774, uid 0, timestamp 1437918996)'
7/26/2015 09:53:50 Sim-01 ERROR spm.mgwd.process.exit: Management Gateway (mgwd) subsystem with ID 12403 exited as a result of signal normal exit (1). The subsystem will attempt to restart.
7/26/2015 09:52:15 Sim-01 ERROR spm.vldb.process.exit: Volume Location Database(VLDB) subsystem with ID 4568 exited as a result of signal normal exit (1). The subsystem will attempt to restart.
7/26/2015 09:52:15 Sim-01 ERROR rdb.env.IOFailure: Error encountered in root volume I/O: Could not force changes to /mroot/etc/cluster_config/rdb/VLDB/_unit.config.bak.
.
7/25/2015 19:29:45 Sim-01 ERROR mgmtgwd.mroot.nearlyfull: The root volume of node 'Sim-01' is nearly full. When it is full the node cannot operate properly.
7/25/2015 19:29:45 Sim-01 ERROR mgmtgwd.configbr.noSNCBackup: This cluster is effectively a single node and there is no configuration backup configured.
7/25/2015 19:29:45 Sim-01 ERROR mgmtgwd.configbr.noSNCBackup: This cluster is effectively a single node and there is no configuration backup configured.
7/25/2015 19:27:45 Sim-01 ERROR mgmtgwd.mroot.nearlyfull: The root volume of node 'Sim-01' is nearly full. When it is full the node cannot operate properly.
7/22/2015 18:34:29 Sim-01 ERROR mgmtgwd.configbr.noSNCBackup: This cluster is effectively a single node and there is no configuration backup configured.
7/22/2015 18:34:29 Sim-01 ERROR mgmtgwd.configbr.noSNCBackup: This cluster is effectively a single node and there is no configuration backup configured.
7/22/2015 18:32:11 Sim-01 ERROR mgmtgwd.mroot.nearlyfull: The root volume of node 'Sim-01' is nearly full. When it is full the node cannot operate properly.
7/21/2015 10:20:13 Sim-01 ERROR mgmtgwd.configbr.noSNCBackup: This cluster is effectively a single node and there is no configuration backup configured.
7/21/2015 09:16:17 Sim-01 ERROR mgmtgwd.configbr.noSNCBackup: This cluster is effectively a single node and there is no configuration backup configured.
7/21/2015 09:16:17 Sim-01 ERROR mgmtgwd.configbr.noSNCBackup: This cluster is effectively a single node and there is no configuration backup configured.
7/21/2015 09:14:53 Sim-01 ERROR mgmtgwd.configbr.backupFailed: The node configuration backup Sim-01.daily.2015-07-21.09_03_46.7z cannot be created. Error: 'Failed to create tarball for backup Sim-01.daily.2015-07-21.09_03_46.7z. Error: Failed to create package with name /mroot/etc/backups/config/Sim-01.daily.2015-07-21.09_03_46.7z'.
7/21/2015 09:14:51 Sim-01 ERROR mgmtgwd.configbr.backupFailed: The node configuration backup Sim-01.8hour.2015-07-21.09_03_46.7z cannot be created. Error: 'Failed to create tarball for backup Sim-01.8hour.2015-07-21.09_03_46.7z. Error: Failed to create package with name /mroot/etc/backups/config/Sim-01.8hour.2015-07-21.09_03_46.7z'.
7/21/2015 09:14:50 Sim-01 ERROR mgmtgwd.mroot.nearlyfull: The root volume of node 'Sim-01' is nearly full. When it is full the node cannot operate properly.
7/21/2015 09:14:37 Sim-01 ERROR mgmtgwd.mroot.full: The root volume of node 'Sim-01' is full. This prevents the node from operating properly.
7/21/2015 09:07:23 Sim-01 ERROR mgmtgwd.mroot.nearlyfull: The root volume of node 'Sim-01' is nearly full. When it is full the node cannot operate properly.
21 entries were displayed.
Sim::>
_______________________________________________ Toasters mailing list Toasters@teaparty.net mailto:Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters