NTNX>日記

個人的な趣味による Nutanix Community Edition 日記。Japanese のみですみません。

Nutanix CE のクラスタ全体シャットダウン~起動。

Nutanix CE のクラスタ全体を、無難そうな手順でシャットダウンしてみました。
今回は、4 ノードのクラスタです。

停止するときの順序

  1. CVM 以外の VMすべて
  2. クラスタ
  3. CVM
  4. Acropolis Hypervisor ~ サーバの電源オフ

起動するときの順序

  1. サーバの電源オン ※Acropolis Hypervisor  ~ CVM は自動起動。
  2. クラスタ
  3. CVM 以外の VM

実際のシャットダウンの様子

CVM 以外の VM 停止

CVM 以外の VM は、何らかの方法でシャットダウンしておきます。

クラスタの停止

CVM のどれか 1台に SSH 接続して、クラスタを停止します。

  • CVM には、リアル IP アドレスで SSH 接続します。
    ※クラスタの VIP だと、cluster stop したときに切断されてしまうため。
    ※今回の CVM の IP アドレスは 10.1.1.111 ~ 114 です。
  • ログインユーザは、nutanix ユーザです。
nutanix@NTNX-18b197fa-A-CVM:10.1.1.111:~$ cluster stop
2015-11-22 02:15:16 INFO cluster:1835 Executing action stop on SVMs 10.1.1.111,10.1.1.112,10.1.1.113,10.1.1.114
2015-11-22 02:15:16 INFO cluster:1844

***** CLUSTER NAME *****
ntnx-cloud

This operation will stop the Nutanix storage services and any VMs using Nutanix storage will become unavailable. Do you want to proceed? (Y/[N]): Y
Waiting on 10.1.1.111 (Up) to stop:  Scavenger SSLTerminator SecureFileSync Acropolis Medusa DynamicRingChanger Pithos Stargate Cerebro Chronos Curator Prism CIM AlertManager Arithmos Snmp SysStatCollector Tunnel ClusterHealth Janus
Waiting on 10.1.1.112 (Up, ZeusLeader) to stop:  Zeus Scavenger SSLTerminator SecureFileSync Acropolis Medusa DynamicRingChanger Pithos Stargate Cerebro Chronos Curator Prism CIM AlertManager Arithmos Snmp SysStatCollector Tunnel ClusterHealth
Waiting on 10.1.1.113 (Up) to stop:  Zeus Scavenger SSLTerminator SecureFileSync Acropolis Medusa DynamicRingChanger Pithos Stargate Cerebro Chronos Curator Prism CIM AlertManager Arithmos Snmp SysStatCollector
Waiting on 10.1.1.114 (Up) to stop:  Zeus Scavenger SSLTerminator SecureFileSync Acropolis Medusa DynamicRingChanger Pithos Stargate Cerebro Chronos Curator Prism CIM AlertManager Arithmos Snmp
Waiting on 10.1.1.111 (Up) to stop:  Scavenger SSLTerminator SecureFileSync Acropolis Medusa DynamicRingChanger Pithos Stargate Cerebro Chronos Curator Prism CIM AlertManager Arithmos Snmp
Waiting on 10.1.1.112 (Up, ZeusLeader) to stop:  Zeus Scavenger SSLTerminator SecureFileSync Acropolis Medusa DynamicRingChanger Pithos Stargate Cerebro Chronos Curator Prism CIM
Waiting on 10.1.1.113 (Up) to stop:  Zeus Scavenger SSLTerminator SecureFileSync Acropolis Medusa DynamicRingChanger Pithos Stargate Cerebro
Waiting on 10.1.1.114 (Up) to stop:  Zeus Scavenger SSLTerminator SecureFileSync Acropolis
Waiting on 10.1.1.111 (Up) to stop:  Scavenger SSLTerminator
Waiting on 10.1.1.112 (Up, ZeusLeader) to stop:  Zeus Scavenger
Waiting on 10.1.1.113 (Up) to stop:  Zeus Scavenger
Waiting on 10.1.1.114 (Up) to stop:  Zeus Scavenger
Waiting on 10.1.1.111 (Up) to stop:  Scavenger
Waiting on 10.1.1.112 (Up, ZeusLeader) to stop:  Zeus Scavenger
Waiting on 10.1.1.113 (Up) to stop:  Zeus Scavenger
Waiting on 10.1.1.114 (Up) to stop:  Zeus Scavenger
The state of the cluster: stop
Lockdown mode: Disabled

        CVM: 10.1.1.111 Up
                           Scavenger   UP       [10514, 10542, 10543, 10650]
                       SSLTerminator DOWN       []
                      SecureFileSync DOWN       []
                           Acropolis DOWN       []
                              Medusa DOWN       []
                  DynamicRingChanger DOWN       []
                              Pithos DOWN       []
                            Stargate DOWN       []
                             Cerebro DOWN       []
                             Chronos DOWN       []
                             Curator DOWN       []
                               Prism DOWN       []
                                 CIM DOWN       []
                        AlertManager DOWN       []
                            Arithmos DOWN       []
                                Snmp DOWN       []
                    SysStatCollector DOWN       []
                              Tunnel DOWN       []
                       ClusterHealth DOWN       []
                               Janus DOWN       []
                   NutanixGuestTools DOWN       []

        CVM: 10.1.1.112 Up, ZeusLeader
                                Zeus   UP       [9857, 9888, 9889, 9894, 9934, 9947]
                           Scavenger   UP       [10639, 10668, 10669, 10792]
                       SSLTerminator DOWN       []
                      SecureFileSync DOWN       []
                           Acropolis DOWN       []
                              Medusa DOWN       []
                  DynamicRingChanger DOWN       []
                              Pithos DOWN       []
                            Stargate DOWN       []
                             Cerebro DOWN       []
                             Chronos DOWN       []
                             Curator DOWN       []
                               Prism DOWN       []
                                 CIM DOWN       []
                        AlertManager DOWN       []
                            Arithmos DOWN       []
                                Snmp DOWN       []
                    SysStatCollector DOWN       []
                              Tunnel DOWN       []
                       ClusterHealth DOWN       []
                               Janus DOWN       []
                   NutanixGuestTools DOWN       []

        CVM: 10.1.1.113 Up
                                Zeus   UP       [9795, 9831, 9832, 9839, 9874, 9888]
                           Scavenger   UP       [10640, 10669, 10670, 10747]
                       SSLTerminator DOWN       []
                      SecureFileSync DOWN       []
                           Acropolis DOWN       []
                              Medusa DOWN       []
                  DynamicRingChanger DOWN       []
                              Pithos DOWN       []
                            Stargate DOWN       []
                             Cerebro DOWN       []
                             Chronos DOWN       []
                             Curator DOWN       []
                               Prism DOWN       []
                                 CIM DOWN       []
                        AlertManager DOWN       []
                            Arithmos DOWN       []
                                Snmp DOWN       []
                    SysStatCollector DOWN       []
                              Tunnel DOWN       []
                       ClusterHealth DOWN       []
                               Janus DOWN       []
                   NutanixGuestTools DOWN       []

        CVM: 10.1.1.114 Up
                                Zeus   UP       [9733, 9770, 9772, 9777, 9812, 9826]
                           Scavenger   UP       [10833, 10861, 10862, 11002]
                       SSLTerminator DOWN       []
                      SecureFileSync DOWN       []
                           Acropolis DOWN       []
                              Medusa DOWN       []
                  DynamicRingChanger DOWN       []
                              Pithos DOWN       []
                            Stargate DOWN       []
                             Cerebro DOWN       []
                             Chronos DOWN       []
                             Curator DOWN       []
                               Prism DOWN       []
                                 CIM DOWN       []
                        AlertManager DOWN       []
                            Arithmos DOWN       []
                                Snmp DOWN       []
                    SysStatCollector DOWN       []
                              Tunnel DOWN       []
                       ClusterHealth DOWN       []
                               Janus DOWN       []
                   NutanixGuestTools DOWN       []
2015-11-22 02:16:19 INFO cluster:1942 Success!

CVM の停止

CVM に SSH 接続して、shutdon -h now や、init 0 などで停止します。

  • これも nutanix ユーザでログインしています。
  • CVM それぞれを停止します。
  • root ではなく nutanix ユーザでログインしているので、sudo で実行します。
  • 1台の CVM から、ほかの CVM を ssh のリモート実行で停止して、最後に自分を停止しています。
    ちなみに、CVM の nutanix ユーザ同士は ssh でパスワードなしログインができるようになっています。
nutanix@NTNX-18b197fa-A-CVM:10.1.1.111:~$ ssh 10.1.1.114 sudo shutdown -h now
Warning: Permanently added '10.1.1.114' (RSA) to the list of known hosts.
Nutanix Controller VM
nutanix@NTNX-18b197fa-A-CVM:10.1.1.111:~$ ssh 10.1.1.113 sudo shutdown -h now
Warning: Permanently added '10.1.1.113' (RSA) to the list of known hosts.
Nutanix Controller VM
nutanix@NTNX-18b197fa-A-CVM:10.1.1.111:~$ ssh 10.1.1.112 sudo shutdown -h now
Warning: Permanently added '10.1.1.112' (RSA) to the list of known hosts.
Nutanix Controller VM
nutanix@NTNX-18b197fa-A-CVM:10.1.1.111:~$ sudo shutdown -h now

Broadcast message from nutanix@NTNX-18b197fa-A-CVM
        (/dev/pts/0) at 2:23 ...

The system is going down for halt NOW!

Acropolis Hypervisor (AHV)の停止

AHV に SSH 接続して、クラスタを停止します。

  • SSH 接続して、shutdon -h now や、init 0 などで停止します。
    ※AHV それぞれを停止します。
    ※ちなみに、root ユーザはパスワードなしログイン設定はされていないようです。

virsh コマンドで、VM が何も起動していないことを確認します。

[root@NTNX-18b197fa-A ~]# virsh list
Id    Name                           State
----------------------------------------------------

AHV それぞれをシャットダウンします。

[root@NTNX-18b197fa-A ~]# shutdown -h now

実際の起動の様子

Nutanix CE のサーバの電源オン

Acropolis Hypervisor ~ CVM は自動起動されます。

クラスタの起動

クラスタは、cluster stop で停止した時の status と同じ状態までしか自動起動しないようです。CVM のどれか 1台から、cluster start コマンドでクラスタを起動します。

  • クラスタの VIP がまだ付与されていないので、リアル IP で接続します。
  • nutanix ユーザで接続します。

clustar start が成功してしばらく待つと、Prism にログインできるようになります。

nutanix@NTNX-18b197fa-A-CVM:10.1.1.111:~$ cluster start
2015-11-23 03:16:16 INFO cluster:1835 Executing action start on SVMs 10.1.1.111,10.1.1.112,10.1.1.113,10.1.1.114
Waiting on 10.1.1.111 (Up) to start:  SSLTerminator SecureFileSync Acropolis Medusa DynamicRingChanger Pithos Stargate Cerebro Chronos Curator Prism CIM AlertManager Arithmos Snmp SysStatCollector Tunnel ClusterHealth Janus NutanixGuestTools
Waiting on 10.1.1.112 (Up, ZeusLeader) to start:  Medusa DynamicRingChanger Pithos Stargate Cerebro Chronos Curator Prism CIM AlertManager Arithmos Snmp SysStatCollector Tunnel ClusterHealth Janus NutanixGuestTools
Waiting on 10.1.1.113 (Up) to start:  DynamicRingChanger Pithos Stargate Cerebro Chronos Curator Prism CIM AlertManager Arithmos Snmp SysStatCollector Tunnel ClusterHealth Janus NutanixGuestTools
Waiting on 10.1.1.114 (Up) to start:  DynamicRingChanger Pithos Stargate Cerebro Chronos Curator Prism CIM AlertManager Arithmos Snmp SysStatCollector Tunnel ClusterHealth Janus NutanixGuestTools
Waiting on 10.1.1.111 (Up) to start:  DynamicRingChanger Pithos Stargate Cerebro Chronos Curator Prism CIM AlertManager Arithmos Snmp SysStatCollector Tunnel ClusterHealth Janus NutanixGuestTools
Waiting on 10.1.1.112 (Up, ZeusLeader) to start:  DynamicRingChanger Pithos Stargate Cerebro Chronos Curator Prism CIM AlertManager Arithmos Snmp SysStatCollector Tunnel ClusterHealth Janus NutanixGuestTools
Waiting on 10.1.1.113 (Up) to start:  DynamicRingChanger Pithos Stargate Cerebro Chronos Curator Prism CIM AlertManager Arithmos Snmp SysStatCollector Tunnel ClusterHealth Janus NutanixGuestTools
Waiting on 10.1.1.114 (Up) to start:  DynamicRingChanger Pithos Stargate Cerebro Chronos Curator Prism CIM AlertManager Arithmos Snmp SysStatCollector Tunnel ClusterHealth Janus NutanixGuestTools
Waiting on 10.1.1.111 (Up) to start:
Waiting on 10.1.1.112 (Up, ZeusLeader) to start:
Waiting on 10.1.1.113 (Up) to start:
Waiting on 10.1.1.114 (Up) to start:
The state of the cluster: start
Lockdown mode: Disabled

        CVM: 10.1.1.111 Up
                           Scavenger   UP       [2739, 2772, 2773, 2825]
                       SSLTerminator   UP       [10912, 10951, 10952, 11028]
                      SecureFileSync   UP       [10921, 10976, 10977, 11019]
                           Acropolis   UP       [10943, 10997, 10998, 11192]
                              Medusa   UP       [11138, 11166, 11167, 11199, 11264]
                  DynamicRingChanger   UP       [11304, 11341, 11342, 11642]
                              Pithos   UP       [11309, 11361, 11362, 11557]
                            Stargate   UP       [11363, 11398, 11399, 11687, 11688]
                             Cerebro   UP       [11369, 11424, 11425, 11768]
                             Chronos   UP       [11421, 11470, 11471, 11697]
                             Curator   UP       [11431, 11495, 11496, 11811]
                               Prism   UP       [11452, 11513, 11514, 11707, 12170]
                                 CIM   UP       [11490, 11547, 11548, 11624]
                        AlertManager   UP       [11511, 11574, 11575, 11749]
                            Arithmos   UP       [11551, 11604, 11605, 11748]
                                Snmp   UP       [11585, 11626, 11627, 12115]
                    SysStatCollector   UP       [11638, 11769, 11772, 12223, 12384, 12385]
                              Tunnel   UP       [11700, 11833, 11834]
                       ClusterHealth   UP       [11744, 11934, 11936, 12121, 12187, 12188, 12291, 12292]
                               Janus   UP       [11841, 11922, 11923]
                   NutanixGuestTools   UP       [11928, 12100, 12101, 12122]

        CVM: 10.1.1.112 Up, ZeusLeader
                                Zeus   UP       [2220, 2246, 2247, 2248, 2293, 2308]
                           Scavenger   UP       [3307, 3339, 3340, 3381]
                       SSLTerminator   UP       [11639, 11708, 11709, 11838]
                      SecureFileSync   UP       [11657, 11739, 11740, 11797]
                           Acropolis   UP       [11699, 11756, 11757, 12005]
                              Medusa   UP       [11976, 12003, 12004, 12006, 12071]
                  DynamicRingChanger   UP       [12275, 12312, 12313, 12633]
                              Pithos   UP       [12279, 12329, 12330, 12601]
                            Stargate   UP       [12337, 12381, 12382, 12646, 12648]
                             Cerebro   UP       [12361, 12450, 12451, 12786]
                             Chronos   UP       [12405, 12484, 12485, 12709]
                             Curator   UP       [12428, 12508, 12509, 12731]
                               Prism   UP       [12465, 12532, 12533, 12698, 13054]
                                 CIM   UP       [12502, 12613, 12614, 12676]
                        AlertManager   UP       [12528, 12631, 12632, 12770]
                            Arithmos   UP       [12595, 12696, 12697, 12791]
                                Snmp   UP       [12628, 12722, 12723, 13156]
                    SysStatCollector   UP       [12969, 13009, 13010, 13188]
                              Tunnel   UP       [12998, 13045, 13046]
                       ClusterHealth   UP       [13041, 13139, 13140, 13184, 13231, 13232, 13346, 13347]
                               Janus   UP       [13064, 13116, 13117]
                   NutanixGuestTools   UP       [13095, 13161, 13162, 13193]

        CVM: 10.1.1.113 Up
                                Zeus   UP       [2220, 2246, 2247, 2248, 2282, 2296]
                           Scavenger   UP       [3297, 3330, 3331, 3377]
                       SSLTerminator   UP       [11581, 11625, 11626, 11700]
                      SecureFileSync   UP       [11586, 11649, 11650, 11687]
                           Acropolis   UP       [11613, 11666, 11667, 11997]
                              Medusa   UP       [11790, 11835, 11836, 11964, 12031]
                  DynamicRingChanger   UP       [12183, 12221, 12222, 12459]
                              Pithos   UP       [12188, 12236, 12238, 12377]
                            Stargate   UP       [12239, 12270, 12271, 12549, 12550]
                             Cerebro   UP       [12246, 12298, 12299, 12604]
                             Chronos   UP       [12282, 12343, 12344, 12531]
                             Curator   UP       [12307, 12361, 12362, 12636]
                               Prism   UP       [12331, 12398, 12399, 12585, 13060]
                                 CIM   UP       [12364, 12441, 12442, 12566]
                        AlertManager   UP       [12381, 12465, 12466, 12611]
                            Arithmos   UP       [12426, 12540, 12541, 12667]
                                Snmp   UP       [12468, 12582, 12583, 13132]
                    SysStatCollector   UP       [12562, 12657, 12658, 13130]
                              Tunnel   UP       [12593, 12705, 12706]
                       ClusterHealth   UP       [12673, 12937, 12938, 13042, 13107, 13109, 13222, 13223]
                               Janus   UP       [12707, 12810, 12811]
                   NutanixGuestTools   UP       [12779, 12945, 12946, 12989]

        CVM: 10.1.1.114 Up
                                Zeus   UP       [2216, 2242, 2243, 2244, 2305, 2320]
                           Scavenger   UP       [3294, 3326, 3327, 3380]
                       SSLTerminator   UP       [13545, 13585, 13586, 13665]
                      SecureFileSync   UP       [13552, 13609, 13610, 13653]
                           Acropolis   UP       [13579, 13631, 13632, 13828]
                              Medusa   UP       [13750, 13792, 13793, 13823, 13988]
                  DynamicRingChanger   UP       [14147, 14185, 14186, 14423]
                              Pithos   UP       [14152, 14200, 14201, 14355]
                            Stargate   UP       [14203, 14235, 14236, 14426, 14428]
                             Cerebro   UP       [14209, 14262, 14263, 14538]
                             Chronos   UP       [14253, 14291, 14292, 14417]
                             Curator   UP       [14276, 14352, 14353, 14617]
                               Prism   UP       [14354, 14388, 14389, 14486, 15034, 15077]
                                 CIM   UP       [14364, 14412, 14413, 14466]
                        AlertManager   UP       [14419, 14460, 14461, 14609]
                            Arithmos   UP       [14472, 14531, 14532, 14650]
                                Snmp   UP       [14500, 14569, 14570, 15101]
                    SysStatCollector   UP       [14536, 14607, 14608, 15134]
                              Tunnel   UP       [14589, 14709, 14710]
                       ClusterHealth   UP       [14662, 14809, 14810, 14999, 15073, 15074, 15212, 15215]
                               Janus   UP       [14744, 14791, 14792]
                   NutanixGuestTools   UP       [14789, 14932, 14933, 14973]
2015-11-23 03:18:22 INFO cluster:1942 Success! 

CVM 以外の VM の起動

Prism などから、CVM 以外の VM を起動します。

以上。