Looks like you inadvertently filled up your root volume.
Doing something there you should be....
May need to log in as diag user and clean up some space on /mroot
--tmac
*Tim McCarthy* *Principal Consultant*
On Sun, Jul 26, 2015 at 10:23 AM, Uddhav Regmi uregmi111@gmail.com wrote:
Hello all,
It just died all of a sudden. Automatic Recovery was off. Pls advise.
Cannot access the cluster level , node level is fine.
Sim::> event log show -severity emerGENCY
Time Node Severity Event
7/26/2015 09:57:50 Sim-01 EMERGENCY callhome.root.vol.recovery.reqd: Call home for ROOT VOLUME NOT WORKING PROPERLY: RECOVERY REQUIRED.
7/26/2015 09:56:36 Sim-01 EMERGENCY rdb.recovery.failed: Error: Unable to find a master.. Unable to recover the local database of Data Replication Module: Management.
2 entries were displayed.
Sim::> event log show -severity erroR
Time Node Severity Event
7/26/2015 10:00:51 Sim-01 ERROR mgmtgwd.ntp.job.failure: The ntp_resolve_job could not be started.
7/26/2015 09:56:39 Sim-01 ERROR spm.mgwd.process.exit: Management Gateway (mgwd) subsystem with ID 12774 exited as a result of signal signal 6. The subsystem will attempt to restart.
7/26/2015 09:56:36 Sim-01 ERROR ucore.panicString: 'mgwd: assertion (false) at corrupt_rdb.cc:587 failed, raising SIGABRT(6) at RIP 0x82e1ed7bc (pid 12774, uid 0, timestamp 1437918996)'
7/26/2015 09:53:50 Sim-01 ERROR spm.mgwd.process.exit: Management Gateway (mgwd) subsystem with ID 12403 exited as a result of signal normal exit (1). The subsystem will attempt to restart.
7/26/2015 09:52:15 Sim-01 ERROR spm.vldb.process.exit: Volume Location Database(VLDB) subsystem with ID 4568 exited as a result of signal normal exit (1). The subsystem will attempt to restart.
7/26/2015 09:52:15 Sim-01 ERROR rdb.env.IOFailure: Error encountered in root volume I/O: Could not force changes to /mroot/etc/cluster_config/rdb/VLDB/_unit.config.bak.
.
7/25/2015 19:29:45 Sim-01 ERROR mgmtgwd.mroot.nearlyfull: The root volume of node 'Sim-01' is nearly full. When it is full the node cannot operate properly.
7/25/2015 19:29:45 Sim-01 ERROR mgmtgwd.configbr.noSNCBackup: This cluster is effectively a single node and there is no configuration backup configured.
7/25/2015 19:29:45 Sim-01 ERROR mgmtgwd.configbr.noSNCBackup: This cluster is effectively a single node and there is no configuration backup configured.
7/25/2015 19:27:45 Sim-01 ERROR mgmtgwd.mroot.nearlyfull: The root volume of node 'Sim-01' is nearly full. When it is full the node cannot operate properly.
7/22/2015 18:34:29 Sim-01 ERROR mgmtgwd.configbr.noSNCBackup: This cluster is effectively a single node and there is no configuration backup configured.
7/22/2015 18:34:29 Sim-01 ERROR mgmtgwd.configbr.noSNCBackup: This cluster is effectively a single node and there is no configuration backup configured.
7/22/2015 18:32:11 Sim-01 ERROR mgmtgwd.mroot.nearlyfull: The root volume of node 'Sim-01' is nearly full. When it is full the node cannot operate properly.
7/21/2015 10:20:13 Sim-01 ERROR mgmtgwd.configbr.noSNCBackup: This cluster is effectively a single node and there is no configuration backup configured.
7/21/2015 09:16:17 Sim-01 ERROR mgmtgwd.configbr.noSNCBackup: This cluster is effectively a single node and there is no configuration backup configured.
7/21/2015 09:16:17 Sim-01 ERROR mgmtgwd.configbr.noSNCBackup: This cluster is effectively a single node and there is no configuration backup configured.
7/21/2015 09:14:53 Sim-01 ERROR mgmtgwd.configbr.backupFailed: The node configuration backup Sim-01.daily.2015-07-21.09_03_46.7z cannot be created. Error: 'Failed to create tarball for backup Sim-01.daily.2015-07-21.09_03_46.7z. Error: Failed to create package with name /mroot/etc/backups/config/Sim-01.daily.2015-07-21.09_03_46.7z'.
7/21/2015 09:14:51 Sim-01 ERROR mgmtgwd.configbr.backupFailed: The node configuration backup Sim-01.8hour.2015-07-21.09_03_46.7z cannot be created. Error: 'Failed to create tarball for backup Sim-01.8hour.2015-07-21.09_03_46.7z. Error: Failed to create package with name /mroot/etc/backups/config/Sim-01.8hour.2015-07-21.09_03_46.7z'.
7/21/2015 09:14:50 Sim-01 ERROR mgmtgwd.mroot.nearlyfull: The root volume of node 'Sim-01' is nearly full. When it is full the node cannot operate properly.
7/21/2015 09:14:37 Sim-01 ERROR mgmtgwd.mroot.full: The root volume of node 'Sim-01' is full. This prevents the node from operating properly.
7/21/2015 09:07:23 Sim-01 ERROR mgmtgwd.mroot.nearlyfull: The root volume of node 'Sim-01' is nearly full. When it is full the node cannot operate properly.
21 entries were displayed.
Sim::>
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters