Nutanix CE のクラスタ全体を、無難そうな手順でシャットダウンしてみました。
今回は、4 ノードのクラスタです。
停止するときの順序
- CVM 以外の VMすべて
- クラスタ
- CVM
- Acropolis Hypervisor ～ サーバの電源オフ
起動するときの順序
- サーバの電源オン ※Acropolis Hypervisor ～ CVM は自動起動。
- クラスタ
- CVM 以外の VM
実際のシャットダウンの様子
CVM 以外の VM 停止
CVM 以外の VM は、何らかの方法でシャットダウンしておきます。
クラスタの停止
CVM のどれか 1台に SSH 接続して、クラスタを停止します。
- CVM には、リアル IP アドレスで SSH 接続します。
※クラスタの VIP だと、cluster stop したときに切断されてしまうため。
※今回の CVM の IP アドレスは 10.1.1.111 ～ 114 です。 - ログインユーザは、nutanix ユーザです。
nutanix@NTNX-18b197fa-A-CVM:10.1.1.111:~$ cluster stop
2015-11-22 02:15:16 INFO cluster:1835 Executing action stop on SVMs 10.1.1.111,10.1.1.112,10.1.1.113,10.1.1.114
2015-11-22 02:15:16 INFO cluster:1844
***** CLUSTER NAME *****
ntnx-cloud
This operation will stop the Nutanix storage services and any VMs using Nutanix storage will become unavailable. Do you want to proceed? (Y/[N]): Y
Waiting on 10.1.1.111 (Up) to stop: Scavenger SSLTerminator SecureFileSync Acropolis Medusa DynamicRingChanger Pithos Stargate Cerebro Chronos Curator Prism CIM AlertManager Arithmos Snmp SysStatCollector Tunnel ClusterHealth Janus
Waiting on 10.1.1.112 (Up, ZeusLeader) to stop: Zeus Scavenger SSLTerminator SecureFileSync Acropolis Medusa DynamicRingChanger Pithos Stargate Cerebro Chronos Curator Prism CIM AlertManager Arithmos Snmp SysStatCollector Tunnel ClusterHealth
Waiting on 10.1.1.113 (Up) to stop: Zeus Scavenger SSLTerminator SecureFileSync Acropolis Medusa DynamicRingChanger Pithos Stargate Cerebro Chronos Curator Prism CIM AlertManager Arithmos Snmp SysStatCollector
Waiting on 10.1.1.114 (Up) to stop: Zeus Scavenger SSLTerminator SecureFileSync Acropolis Medusa DynamicRingChanger Pithos Stargate Cerebro Chronos Curator Prism CIM AlertManager Arithmos Snmp
Waiting on 10.1.1.111 (Up) to stop: Scavenger SSLTerminator SecureFileSync Acropolis Medusa DynamicRingChanger Pithos Stargate Cerebro Chronos Curator Prism CIM AlertManager Arithmos Snmp
Waiting on 10.1.1.112 (Up, ZeusLeader) to stop: Zeus Scavenger SSLTerminator SecureFileSync Acropolis Medusa DynamicRingChanger Pithos Stargate Cerebro Chronos Curator Prism CIM
Waiting on 10.1.1.113 (Up) to stop: Zeus Scavenger SSLTerminator SecureFileSync Acropolis Medusa DynamicRingChanger Pithos Stargate Cerebro
Waiting on 10.1.1.114 (Up) to stop: Zeus Scavenger SSLTerminator SecureFileSync Acropolis
Waiting on 10.1.1.111 (Up) to stop: Scavenger SSLTerminator
Waiting on 10.1.1.112 (Up, ZeusLeader) to stop: Zeus Scavenger
Waiting on 10.1.1.113 (Up) to stop: Zeus Scavenger
Waiting on 10.1.1.114 (Up) to stop: Zeus Scavenger
Waiting on 10.1.1.111 (Up) to stop: Scavenger
Waiting on 10.1.1.112 (Up, ZeusLeader) to stop: Zeus Scavenger
Waiting on 10.1.1.113 (Up) to stop: Zeus Scavenger
Waiting on 10.1.1.114 (Up) to stop: Zeus Scavenger
The state of the cluster: stop
Lockdown mode: Disabled
CVM: 10.1.1.111 Up
Scavenger UP [10514, 10542, 10543, 10650]
SSLTerminator DOWN []
SecureFileSync DOWN []
Acropolis DOWN []
Medusa DOWN []
DynamicRingChanger DOWN []
Pithos DOWN []
Stargate DOWN []
Cerebro DOWN []
Chronos DOWN []
Curator DOWN []
Prism DOWN []
CIM DOWN []
AlertManager DOWN []
Arithmos DOWN []
Snmp DOWN []
SysStatCollector DOWN []
Tunnel DOWN []
ClusterHealth DOWN []
Janus DOWN []
NutanixGuestTools DOWN []
CVM: 10.1.1.112 Up, ZeusLeader
Zeus UP [9857, 9888, 9889, 9894, 9934, 9947]
Scavenger UP [10639, 10668, 10669, 10792]
SSLTerminator DOWN []
SecureFileSync DOWN []
Acropolis DOWN []
Medusa DOWN []
DynamicRingChanger DOWN []
Pithos DOWN []
Stargate DOWN []
Cerebro DOWN []
Chronos DOWN []
Curator DOWN []
Prism DOWN []
CIM DOWN []
AlertManager DOWN []
Arithmos DOWN []
Snmp DOWN []
SysStatCollector DOWN []
Tunnel DOWN []
ClusterHealth DOWN []
Janus DOWN []
NutanixGuestTools DOWN []
CVM: 10.1.1.113 Up
Zeus UP [9795, 9831, 9832, 9839, 9874, 9888]
Scavenger UP [10640, 10669, 10670, 10747]
SSLTerminator DOWN []
SecureFileSync DOWN []
Acropolis DOWN []
Medusa DOWN []
DynamicRingChanger DOWN []
Pithos DOWN []
Stargate DOWN []
Cerebro DOWN []
Chronos DOWN []
Curator DOWN []
Prism DOWN []
CIM DOWN []
AlertManager DOWN []
Arithmos DOWN []
Snmp DOWN []
SysStatCollector DOWN []
Tunnel DOWN []
ClusterHealth DOWN []
Janus DOWN []
NutanixGuestTools DOWN []
CVM: 10.1.1.114 Up
Zeus UP [9733, 9770, 9772, 9777, 9812, 9826]
Scavenger UP [10833, 10861, 10862, 11002]
SSLTerminator DOWN []
SecureFileSync DOWN []
Acropolis DOWN []
Medusa DOWN []
DynamicRingChanger DOWN []
Pithos DOWN []
Stargate DOWN []
Cerebro DOWN []
Chronos DOWN []
Curator DOWN []
Prism DOWN []
CIM DOWN []
AlertManager DOWN []
Arithmos DOWN []
Snmp DOWN []
SysStatCollector DOWN []
Tunnel DOWN []
ClusterHealth DOWN []
Janus DOWN []
NutanixGuestTools DOWN []
2015-11-22 02:16:19 INFO cluster:1942 Success!
CVM の停止
CVM に SSH 接続して、shutdon -h now や、init 0 などで停止します。
- これも nutanix ユーザでログインしています。
- CVM それぞれを停止します。
- root ではなく nutanix ユーザでログインしているので、sudo で実行します。
- 1台の CVM から、ほかの CVM を ssh のリモート実行で停止して、最後に自分を停止しています。
ちなみに、CVM の nutanix ユーザ同士は ssh でパスワードなしログインができるようになっています。
nutanix@NTNX-18b197fa-A-CVM:10.1.1.111:~$ ssh 10.1.1.114 sudo shutdown -h now
Warning: Permanently added '10.1.1.114' (RSA) to the list of known hosts.
Nutanix Controller VM
nutanix@NTNX-18b197fa-A-CVM:10.1.1.111:~$ ssh 10.1.1.113 sudo shutdown -h now
Warning: Permanently added '10.1.1.113' (RSA) to the list of known hosts.
Nutanix Controller VM
nutanix@NTNX-18b197fa-A-CVM:10.1.1.111:~$ ssh 10.1.1.112 sudo shutdown -h now
Warning: Permanently added '10.1.1.112' (RSA) to the list of known hosts.
Nutanix Controller VM
nutanix@NTNX-18b197fa-A-CVM:10.1.1.111:~$ sudo shutdown -h now
Broadcast message from nutanix@NTNX-18b197fa-A-CVM
(/dev/pts/0) at 2:23 ...
The system is going down for halt NOW!
Acropolis Hypervisor （AHV）の停止
AHV に SSH 接続して、クラスタを停止します。
- SSH 接続して、shutdon -h now や、init 0 などで停止します。
※AHV それぞれを停止します。
※ちなみに、root ユーザはパスワードなしログイン設定はされていないようです。
virsh コマンドで、VM が何も起動していないことを確認します。
[root@NTNX-18b197fa-A ~]# virsh list Id Name State ----------------------------------------------------
AHV それぞれをシャットダウンします。
[root@NTNX-18b197fa-A ~]# shutdown -h now
実際の起動の様子
Nutanix CE のサーバの電源オン
Acropolis Hypervisor ～ CVM は自動起動されます。
クラスタの起動
クラスタは、cluster stop で停止した時の status と同じ状態までしか自動起動しないようです。CVM のどれか 1台から、cluster start コマンドでクラスタを起動します。
- クラスタの VIP がまだ付与されていないので、リアル IP で接続します。
- nutanix ユーザで接続します。
clustar start が成功してしばらく待つと、Prism にログインできるようになります。
nutanix@NTNX-18b197fa-A-CVM:10.1.1.111:~$ cluster start
2015-11-23 03:16:16 INFO cluster:1835 Executing action start on SVMs 10.1.1.111,10.1.1.112,10.1.1.113,10.1.1.114
Waiting on 10.1.1.111 (Up) to start: SSLTerminator SecureFileSync Acropolis Medusa DynamicRingChanger Pithos Stargate Cerebro Chronos Curator Prism CIM AlertManager Arithmos Snmp SysStatCollector Tunnel ClusterHealth Janus NutanixGuestTools
Waiting on 10.1.1.112 (Up, ZeusLeader) to start: Medusa DynamicRingChanger Pithos Stargate Cerebro Chronos Curator Prism CIM AlertManager Arithmos Snmp SysStatCollector Tunnel ClusterHealth Janus NutanixGuestTools
Waiting on 10.1.1.113 (Up) to start: DynamicRingChanger Pithos Stargate Cerebro Chronos Curator Prism CIM AlertManager Arithmos Snmp SysStatCollector Tunnel ClusterHealth Janus NutanixGuestTools
Waiting on 10.1.1.114 (Up) to start: DynamicRingChanger Pithos Stargate Cerebro Chronos Curator Prism CIM AlertManager Arithmos Snmp SysStatCollector Tunnel ClusterHealth Janus NutanixGuestTools
Waiting on 10.1.1.111 (Up) to start: DynamicRingChanger Pithos Stargate Cerebro Chronos Curator Prism CIM AlertManager Arithmos Snmp SysStatCollector Tunnel ClusterHealth Janus NutanixGuestTools
Waiting on 10.1.1.112 (Up, ZeusLeader) to start: DynamicRingChanger Pithos Stargate Cerebro Chronos Curator Prism CIM AlertManager Arithmos Snmp SysStatCollector Tunnel ClusterHealth Janus NutanixGuestTools
Waiting on 10.1.1.113 (Up) to start: DynamicRingChanger Pithos Stargate Cerebro Chronos Curator Prism CIM AlertManager Arithmos Snmp SysStatCollector Tunnel ClusterHealth Janus NutanixGuestTools
Waiting on 10.1.1.114 (Up) to start: DynamicRingChanger Pithos Stargate Cerebro Chronos Curator Prism CIM AlertManager Arithmos Snmp SysStatCollector Tunnel ClusterHealth Janus NutanixGuestTools
Waiting on 10.1.1.111 (Up) to start:
Waiting on 10.1.1.112 (Up, ZeusLeader) to start:
Waiting on 10.1.1.113 (Up) to start:
Waiting on 10.1.1.114 (Up) to start:
The state of the cluster: start
Lockdown mode: Disabled
CVM: 10.1.1.111 Up
Scavenger UP [2739, 2772, 2773, 2825]
SSLTerminator UP [10912, 10951, 10952, 11028]
SecureFileSync UP [10921, 10976, 10977, 11019]
Acropolis UP [10943, 10997, 10998, 11192]
Medusa UP [11138, 11166, 11167, 11199, 11264]
DynamicRingChanger UP [11304, 11341, 11342, 11642]
Pithos UP [11309, 11361, 11362, 11557]
Stargate UP [11363, 11398, 11399, 11687, 11688]
Cerebro UP [11369, 11424, 11425, 11768]
Chronos UP [11421, 11470, 11471, 11697]
Curator UP [11431, 11495, 11496, 11811]
Prism UP [11452, 11513, 11514, 11707, 12170]
CIM UP [11490, 11547, 11548, 11624]
AlertManager UP [11511, 11574, 11575, 11749]
Arithmos UP [11551, 11604, 11605, 11748]
Snmp UP [11585, 11626, 11627, 12115]
SysStatCollector UP [11638, 11769, 11772, 12223, 12384, 12385]
Tunnel UP [11700, 11833, 11834]
ClusterHealth UP [11744, 11934, 11936, 12121, 12187, 12188, 12291, 12292]
Janus UP [11841, 11922, 11923]
NutanixGuestTools UP [11928, 12100, 12101, 12122]
CVM: 10.1.1.112 Up, ZeusLeader
Zeus UP [2220, 2246, 2247, 2248, 2293, 2308]
Scavenger UP [3307, 3339, 3340, 3381]
SSLTerminator UP [11639, 11708, 11709, 11838]
SecureFileSync UP [11657, 11739, 11740, 11797]
Acropolis UP [11699, 11756, 11757, 12005]
Medusa UP [11976, 12003, 12004, 12006, 12071]
DynamicRingChanger UP [12275, 12312, 12313, 12633]
Pithos UP [12279, 12329, 12330, 12601]
Stargate UP [12337, 12381, 12382, 12646, 12648]
Cerebro UP [12361, 12450, 12451, 12786]
Chronos UP [12405, 12484, 12485, 12709]
Curator UP [12428, 12508, 12509, 12731]
Prism UP [12465, 12532, 12533, 12698, 13054]
CIM UP [12502, 12613, 12614, 12676]
AlertManager UP [12528, 12631, 12632, 12770]
Arithmos UP [12595, 12696, 12697, 12791]
Snmp UP [12628, 12722, 12723, 13156]
SysStatCollector UP [12969, 13009, 13010, 13188]
Tunnel UP [12998, 13045, 13046]
ClusterHealth UP [13041, 13139, 13140, 13184, 13231, 13232, 13346, 13347]
Janus UP [13064, 13116, 13117]
NutanixGuestTools UP [13095, 13161, 13162, 13193]
CVM: 10.1.1.113 Up
Zeus UP [2220, 2246, 2247, 2248, 2282, 2296]
Scavenger UP [3297, 3330, 3331, 3377]
SSLTerminator UP [11581, 11625, 11626, 11700]
SecureFileSync UP [11586, 11649, 11650, 11687]
Acropolis UP [11613, 11666, 11667, 11997]
Medusa UP [11790, 11835, 11836, 11964, 12031]
DynamicRingChanger UP [12183, 12221, 12222, 12459]
Pithos UP [12188, 12236, 12238, 12377]
Stargate UP [12239, 12270, 12271, 12549, 12550]
Cerebro UP [12246, 12298, 12299, 12604]
Chronos UP [12282, 12343, 12344, 12531]
Curator UP [12307, 12361, 12362, 12636]
Prism UP [12331, 12398, 12399, 12585, 13060]
CIM UP [12364, 12441, 12442, 12566]
AlertManager UP [12381, 12465, 12466, 12611]
Arithmos UP [12426, 12540, 12541, 12667]
Snmp UP [12468, 12582, 12583, 13132]
SysStatCollector UP [12562, 12657, 12658, 13130]
Tunnel UP [12593, 12705, 12706]
ClusterHealth UP [12673, 12937, 12938, 13042, 13107, 13109, 13222, 13223]
Janus UP [12707, 12810, 12811]
NutanixGuestTools UP [12779, 12945, 12946, 12989]
CVM: 10.1.1.114 Up
Zeus UP [2216, 2242, 2243, 2244, 2305, 2320]
Scavenger UP [3294, 3326, 3327, 3380]
SSLTerminator UP [13545, 13585, 13586, 13665]
SecureFileSync UP [13552, 13609, 13610, 13653]
Acropolis UP [13579, 13631, 13632, 13828]
Medusa UP [13750, 13792, 13793, 13823, 13988]
DynamicRingChanger UP [14147, 14185, 14186, 14423]
Pithos UP [14152, 14200, 14201, 14355]
Stargate UP [14203, 14235, 14236, 14426, 14428]
Cerebro UP [14209, 14262, 14263, 14538]
Chronos UP [14253, 14291, 14292, 14417]
Curator UP [14276, 14352, 14353, 14617]
Prism UP [14354, 14388, 14389, 14486, 15034, 15077]
CIM UP [14364, 14412, 14413, 14466]
AlertManager UP [14419, 14460, 14461, 14609]
Arithmos UP [14472, 14531, 14532, 14650]
Snmp UP [14500, 14569, 14570, 15101]
SysStatCollector UP [14536, 14607, 14608, 15134]
Tunnel UP [14589, 14709, 14710]
ClusterHealth UP [14662, 14809, 14810, 14999, 15073, 15074, 15212, 15215]
Janus UP [14744, 14791, 14792]
NutanixGuestTools UP [14789, 14932, 14933, 14973]
2015-11-23 03:18:22 INFO cluster:1942 Success!
CVM 以外の VM の起動
Prism などから、CVM 以外の VM を起動します。
以上。
