You're reaching out to toasters after wedging yourself with a cdot simulator?
.
On Jul 26, 2015, at 7:49 AM, Uddhav Regmi <uregmi111@gmail.commailto:uregmi111@gmail.com> wrote:
I am checking on this
Sim-01> df -gm Filesystem total used avail capacity Mounted on /vol/vol0/ 766MB 672MB 94MB 88% /vol/vol0/ /vol/vol0/.snapshot 40MB 0MB 40MB 0% /vol/vol0/.snapshot /vol/vol101/ 19MB 0MB 18MB 1% /vol/vol101/ /vol/vol101/.snapshot 1MB 0MB 0MB 61% /vol/vol101/.snapshot /vol/svm101_root/ 19MB 0MB 18MB 1% /vol/svm101_root/ /vol/svm101_root/.snapshot 1MB 0MB 0MB 58% /vol/svm101_root/.snapshot
Last time when I had this issue, it was different and I turned off all snapshot. Looks like 88%, there is something , somewhere this is getting filled up
Hmmmm…..nothing inside volume of vserver ??? Interesting…
Sim::volume*> ?
Sim::volume*> help Press "?" or Tab for help.
Sim::volume*> cd ..
Sim::*> vserver
Sim::vserver*> ? audit> Manage auditing of protocol requests that the Vserver services check> The check directory cifs> Manage the CIFS configuration of a Vserver nfs> Manage the NFS configuration of a Vserver services> The services directory vscan> Manage Vscan
From: tmac [mailto:tmacmd@gmail.com] Sent: Sunday, July 26, 2015 10:37 AM To: Uddhav Regmi <uregmi111@gmail.commailto:uregmi111@gmail.com> Cc: Toasters <Toasters@teaparty.netmailto:Toasters@teaparty.net> Subject: Re: unable to find master : Cdot : 8.3RC1
Looks like you inadvertently filled up your root volume.
Doing something there you should be....
May need to log in as diag user and clean up some space on /mroot
--tmac
Tim McCarthy Principal Consultant
On Sun, Jul 26, 2015 at 10:23 AM, Uddhav Regmi <uregmi111@gmail.commailto:uregmi111@gmail.com> wrote: Hello all,
It just died all of a sudden. Automatic Recovery was off. Pls advise.
Cannot access the cluster level , node level is fine.
Sim::> event log show -severity emerGENCY Time Node Severity Event ------------------- ---------------- ------------- --------------------------- 7/26/2015 09:57:50 Sim-01 EMERGENCY callhome.root.vol.recovery.reqd: Call home for ROOT VOLUME NOT WORKING PROPERLY: RECOVERY REQUIRED. 7/26/2015 09:56:36 Sim-01 EMERGENCY rdb.recovery.failed: Error: Unable to find a master.. Unable to recover the local database of Data Replication Module: Management. 2 entries were displayed.
Sim::> event log show -severity erroR Time Node Severity Event ------------------- ---------------- ------------- --------------------------- 7/26/2015 10:00:51 Sim-01 ERROR mgmtgwd.ntp.job.failure: The ntp_resolve_job could not be started. 7/26/2015 09:56:39 Sim-01 ERROR spm.mgwd.process.exit: Management Gateway (mgwd) subsystem with ID 12774 exited as a result of signal signal 6. The subsystem will attempt to restart. 7/26/2015 09:56:36 Sim-01 ERROR ucore.panicString: 'mgwd: assertion (false) at corrupt_rdb.cc:587 failed, raising SIGABRT(6) at RIP 0x82e1ed7bc (pid 12774, uid 0, timestamp 1437918996)' 7/26/2015 09:53:50 Sim-01 ERROR spm.mgwd.process.exit: Management Gateway (mgwd) subsystem with ID 12403 exited as a result of signal normal exit (1). The subsystem will attempt to restart. 7/26/2015 09:52:15 Sim-01 ERROR spm.vldb.process.exit: Volume Location Database(VLDB) subsystem with ID 4568 exited as a result of signal normal exit (1). The subsystem will attempt to restart. 7/26/2015 09:52:15 Sim-01 ERROR rdb.env.IOFailure: Error encountered in root volume I/O: Could not force changes to /mroot/etc/cluster_config/rdb/VLDB/_unit.config.bak. . 7/25/2015 19:29:45 Sim-01 ERROR mgmtgwd.mroot.nearlyfull: The root volume of node 'Sim-01' is nearly full. When it is full the node cannot operate properly. 7/25/2015 19:29:45 Sim-01 ERROR mgmtgwd.configbr.noSNCBackup: This cluster is effectively a single node and there is no configuration backup configured. 7/25/2015 19:29:45 Sim-01 ERROR mgmtgwd.configbr.noSNCBackup: This cluster is effectively a single node and there is no configuration backup configured. 7/25/2015 19:27:45 Sim-01 ERROR mgmtgwd.mroot.nearlyfull: The root volume of node 'Sim-01' is nearly full. When it is full the node cannot operate properly. 7/22/2015 18:34:29 Sim-01 ERROR mgmtgwd.configbr.noSNCBackup: This cluster is effectively a single node and there is no configuration backup configured. 7/22/2015 18:34:29 Sim-01 ERROR mgmtgwd.configbr.noSNCBackup: This cluster is effectively a single node and there is no configuration backup configured. 7/22/2015 18:32:11 Sim-01 ERROR mgmtgwd.mroot.nearlyfull: The root volume of node 'Sim-01' is nearly full. When it is full the node cannot operate properly. 7/21/2015 10:20:13 Sim-01 ERROR mgmtgwd.configbr.noSNCBackup: This cluster is effectively a single node and there is no configuration backup configured. 7/21/2015 09:16:17 Sim-01 ERROR mgmtgwd.configbr.noSNCBackup: This cluster is effectively a single node and there is no configuration backup configured. 7/21/2015 09:16:17 Sim-01 ERROR mgmtgwd.configbr.noSNCBackup: This cluster is effectively a single node and there is no configuration backup configured. 7/21/2015 09:14:53 Sim-01 ERROR mgmtgwd.configbr.backupFailed: The node configuration backup Sim-01.daily.2015-07-21.09_03_46.7z cannot be created. Error: 'Failed to create tarball for backup Sim-01.daily.2015-07-21.09_03_46.7z. Error: Failed to create package with name /mroot/etc/backups/config/Sim-01.daily.2015-07-21.09_03_46.7z'. 7/21/2015 09:14:51 Sim-01 ERROR mgmtgwd.configbr.backupFailed: The node configuration backup Sim-01.8hour.2015-07-21.09_03_46.7z cannot be created. Error: 'Failed to create tarball for backup Sim-01.8hour.2015-07-21.09_03_46.7z. Error: Failed to create package with name /mroot/etc/backups/config/Sim-01.8hour.2015-07-21.09_03_46.7z'. 7/21/2015 09:14:50 Sim-01 ERROR mgmtgwd.mroot.nearlyfull: The root volume of node 'Sim-01' is nearly full. When it is full the node cannot operate properly. 7/21/2015 09:14:37 Sim-01 ERROR mgmtgwd.mroot.full: The root volume of node 'Sim-01' is full. This prevents the node from operating properly. 7/21/2015 09:07:23 Sim-01 ERROR mgmtgwd.mroot.nearlyfull: The root volume of node 'Sim-01' is nearly full. When it is full the node cannot operate properly. 21 entries were displayed.
Sim::>
_______________________________________________ Toasters mailing list Toasters@teaparty.netmailto:Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
_______________________________________________ Toasters mailing list Toasters@teaparty.netmailto:Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters