NKP の Kubernetes クラスタに、Nutanix Enterprise AI（NAI）2.6 を展開してみます。NAI を NKP で使用するための機能では NKP Pro 以上のライセンスが必要になるので、ここで NKP 管理クラスタにライセンス キーを追加しておきます。
前回はこちら。
今回の内容です。
1. NKP ライセンス キーの入手
NKP のライセンス キーは、Nutanix Support & Insights Portal の Licenses 画面から入手しておく必要があります。
ライセンス キーを入手するにあたり、この先の画面で下記の情報が必要になるので、事前に決定しておきます。
- NKP 管理クラスタの名前（今回の例であれば、nkpnai）
- NKP クラスタの Worker ノードの vCPU 数（Control Plane ノードの vCPU は不要）
手順については、下記のあたりを参考にするとよいと思います。
- https://portal.nutanix.com/page/documents/details?targetId=Nutanix-Kubernetes-Platform-v2_17-ja-JP:top-add-nkp-license-t.html
- https://portal.nutanix.com/page/documents/details?targetId=License-Manager-ja-JP:lmg-licmgr-generating-nkp-license-keys-t.html
2. NKP Dashboard でのライセンス キー追加
Web ブラウザで NKP ダッシュボードにログインして「Seggings」→「Licensing」画面を開き、「Activete License」をクリックします。
Nutanix Support & Insights Portal で入手したライセンス キーを入力して、「Activete」をクリックします。
これで、ライセンス キーが追加されました。
画面を更新すると、NKP Starter では存在しなかったメニューなどが表示されます。
おまけ
NKP Pro / Ultimate のライセンスを追加すると、NKP の管理クラスタでは Pod が自動的に追加起動されます。NKP では Kubernetes ノードの CPU / メモリ リソース要件が高めですが、起動される Pod が多いため、実際に要件どおりのリソースが必要とされます。
Kubernetes では Pod ごとに CPU / メモリの要求量（requests）を定義しています。そのため、Linux から見た Kubernetes ノードのリソース使用状況だけでなく、Kubernetes の観点（「kubectl describe node」コマンドなど）でのリソース割り当てについても、確認が必要になります。
NKP Starter の Pod の様子
Starter でも大量の Pod が起動されるので、NKP のバージョンが上がるごとに、必要リソースも増加しているようです。
gowatana@nkp-work-02:~$ kubectl get pods -A NAMESPACE NAME READY STATUS RESTARTS AGE caaph-system caaph-controller-manager-76d9f5c957-6rpjc 1/1 Running 0 31h capa-system capa-controller-manager-948899f8b-x4clk 1/1 Running 0 31h capg-system capg-controller-manager-85976cf9c6-pd5rl 1/1 Running 0 31h capi-kubeadm-bootstrap-system capi-kubeadm-bootstrap-controller-manager-85fc9c88bd-p4tpj 1/1 Running 0 31h capi-kubeadm-control-plane-system capi-kubeadm-control-plane-controller-manager-75bb9ffd66-fvz8t 1/1 Running 0 31h capi-system capi-controller-manager-54dfcd4b6-xhwpt 1/1 Running 0 31h cappp-system cappp-controller-manager-dfc49968b-hvl5l 1/1 Running 0 31h capv-system capv-controller-manager-5bb5dff6b4-wtsth 1/1 Running 0 31h capx-system capx-controller-manager-5bd68554db-pgsvd 1/1 Running 0 31h capz-system azureserviceoperator-controller-manager-55cc766f5-5tv7r 1/1 Running 0 31h capz-system capz-controller-manager-748cd47646-9hnld 1/1 Running 0 31h caren-system cluster-api-runtime-extensions-nutanix-85f5cb84bf-bdg57 1/1 Running 0 31h caren-system helm-repository-59d9fdf9b4-kq2nt 1/1 Running 0 31h cert-manager cert-manager-5997d6b7d9-kd9r6 1/1 Running 0 31h cert-manager cert-manager-cainjector-56b9b6f7-4fxps 1/1 Running 0 31h cert-manager cert-manager-webhook-7c75989cb-swxw2 1/1 Running 0 31h container-object-storage-system container-object-storage-controller-667886cd7f-9hh42 1/1 Running 0 31h default cluster-autoscaler-019d2865-0092-7165-95a0-95f18543452b-6dwz8vc 1/1 Running 0 31h external-secrets external-secrets-56977fdb44-vc6gg 1/1 Running 0 31h external-secrets external-secrets-webhook-6d5f75d6f5-66jcf 1/1 Running 0 31h git-operator-system git-operator-admin-credentials-rotate-29576160-qbprx 0/1 Completed 0 12h git-operator-system git-operator-controller-manager-65f4b7c957-vptzq 2/2 Running 1 (12h ago) 31h git-operator-system git-operator-git-0 4/4 Running 0 31h kommander-default-workspace karma-traefik-certs-kommander-default-workspace-cert-federj6w7v 1/1 Running 0 31h kommander-default-workspace prometheus-traefik-certs-kommander-default-workspace-cert-lwj9v 1/1 Running 0 31h kommander-flux flux-oci-mirror-5c948b8cc5-zzbpg 1/1 Running 0 31h kommander-flux helm-controller-fb7455768-kx5tg 1/1 Running 0 31h kommander-flux image-automation-controller-57fdc5cc95-ctrgl 1/1 Running 0 31h kommander-flux image-reflector-controller-6f4c4669f8-qqd9f 1/1 Running 0 31h kommander-flux kustomize-controller-68bc587994-d6qrj 1/1 Running 0 31h kommander-flux notification-controller-6b8fcc4d9f-m54wn 1/1 Running 0 31h kommander-flux source-controller-6b47b69475-s7grc 1/1 Running 0 31h kommander cluster-observer-2360587938-865656b749-jkrrt 1/1 Running 0 31h kommander dex-7586959cc9-lvgcv 1/1 Running 0 31h kommander dex-dex-controller-649447fc7c-lg4wm 2/2 Running 0 31h kommander dex-k8s-authenticator-6f9484d94c-tv77h 1/1 Running 0 31h kommander gatekeeper-audit-76f484c6cb-jhlb7 1/1 Running 0 31h kommander gatekeeper-controller-manager-65c6f4bc7b-cr7z9 1/1 Running 0 31h kommander gatekeeper-controller-manager-65c6f4bc7b-zfzpw 1/1 Running 0 31h kommander karma-traefik-certs-kommander-cert-federation-6d45cc9555-b5czl 1/1 Running 0 31h kommander kommander-appmanagement-c7f68fd56-wzl76 2/2 Running 0 31h kommander kommander-appmanagement-webhook-77fdfdc87b-69p4g 1/1 Running 0 31h kommander kommander-authorizedlister-7d4dc759b-fzt45 1/1 Running 0 31h kommander kommander-bootstrap-s285w 0/1 Completed 0 31h kommander kommander-capimate-dc4f94fb8-h9b8s 1/1 Running 0 31h kommander kommander-capimate-dc4f94fb8-tpbfk 1/1 Running 0 31h kommander kommander-cm-78748b84c5-g7xmz 2/2 Running 0 31h kommander kommander-flux-operator-67775ff756-h2n9v 2/2 Running 0 31h kommander kommander-kommander-ui-5f9b9bf458-2zzzs 1/1 Running 0 31h kommander kommander-licensing-cm-5b67c659df-v77sx 2/2 Running 0 31h kommander kommander-licensing-webhook-678d85db95-mbwhv 1/1 Running 0 31h kommander kommander-operator-6696f9d78d-6bx2d 1/1 Running 0 31h kommander kommander-reloader-reloader-6468cdff65-b8jqr 1/1 Running 0 31h kommander kommander-traefik-6d769b69b6-mc674 1/1 Running 0 31h kommander kommander-traefik-6d769b69b6-p85xh 1/1 Running 0 31h kommander kommander-webhook-6989b5f474-5487z 1/1 Running 0 31h kommander kube-oidc-proxy-56b6d5f7b7-n7r66 1/1 Running 0 31h kommander prometheus-traefik-certs-kommander-cert-federation-7f96c6b7p2gg 1/1 Running 0 31h kommander runtime-extension-kommander-5545b9dd54-dkkf2 1/1 Running 0 31h kommander traefik-forward-auth-mgmt-675d8cdb54-s6vjp 1/1 Running 0 31h kube-federation-system kubefed-admission-webhook-75dd7fdb9-kwnqg 1/1 Running 0 31h kube-federation-system kubefed-controller-manager-5545598dd9-glfv2 1/1 Running 0 31h kube-federation-system kubefed-controller-manager-5545598dd9-vdqrc 1/1 Running 0 31h kube-system cilium-b2d5t 1/1 Running 0 31h kube-system cilium-envoy-49hlj 1/1 Running 0 31h kube-system cilium-envoy-98464 1/1 Running 0 31h kube-system cilium-envoy-gdgjf 1/1 Running 0 31h kube-system cilium-envoy-jr64q 1/1 Running 0 31h kube-system cilium-envoy-mqfp5 1/1 Running 0 31h kube-system cilium-envoy-mwlww 1/1 Running 0 31h kube-system cilium-envoy-r78sh 1/1 Running 0 31h kube-system cilium-jxkg6 1/1 Running 0 31h kube-system cilium-krktq 1/1 Running 0 31h kube-system cilium-nvxw5 1/1 Running 0 31h kube-system cilium-operator-555cf9bf96-2ncss 1/1 Running 0 31h kube-system cilium-operator-555cf9bf96-2p9td 1/1 Running 0 31h kube-system cilium-pcdpk 1/1 Running 0 31h kube-system cilium-pzcqx 1/1 Running 0 31h kube-system cilium-zzs2d 1/1 Running 0 31h kube-system coredns-66bc5c9577-m6fzt 1/1 Running 0 31h kube-system coredns-66bc5c9577-tgwvm 1/1 Running 0 31h kube-system etcd-nkpnai-d9vzg-h5cbv 1/1 Running 0 31h kube-system etcd-nkpnai-d9vzg-km4cq 1/1 Running 0 31h kube-system etcd-nkpnai-d9vzg-pk5z7 1/1 Running 0 31h kube-system hubble-relay-95b99b996-z5bvk 1/1 Running 1 (31h ago) 31h kube-system kube-apiserver-nkpnai-d9vzg-h5cbv 1/1 Running 0 31h kube-system kube-apiserver-nkpnai-d9vzg-km4cq 1/1 Running 0 31h kube-system kube-apiserver-nkpnai-d9vzg-pk5z7 1/1 Running 0 31h kube-system kube-controller-manager-nkpnai-d9vzg-h5cbv 1/1 Running 0 31h kube-system kube-controller-manager-nkpnai-d9vzg-km4cq 1/1 Running 0 31h kube-system kube-controller-manager-nkpnai-d9vzg-pk5z7 1/1 Running 0 31h kube-system kube-scheduler-nkpnai-d9vzg-h5cbv 1/1 Running 0 31h kube-system kube-scheduler-nkpnai-d9vzg-km4cq 1/1 Running 0 31h kube-system kube-scheduler-nkpnai-d9vzg-pk5z7 1/1 Running 0 31h kube-system kube-vip-nkpnai-d9vzg-h5cbv 1/1 Running 0 31h kube-system kube-vip-nkpnai-d9vzg-km4cq 1/1 Running 0 31h kube-system kube-vip-nkpnai-d9vzg-pk5z7 1/1 Running 0 31h kube-system nutanix-cloud-controller-manager-577bdb9865-xx75w 1/1 Running 0 31h kube-system snapshot-controller-58cb99fc45-k4nf7 1/1 Running 0 31h metallb-system metallb-controller-84cfb9944d-fkw2t 1/1 Running 0 31h metallb-system metallb-speaker-h7zsc 4/4 Running 1 (31h ago) 31h metallb-system metallb-speaker-mgfr8 4/4 Running 1 (31h ago) 31h metallb-system metallb-speaker-ppxk8 4/4 Running 0 31h metallb-system metallb-speaker-tkpgk 4/4 Running 1 (31h ago) 31h metallb-system metallb-speaker-tn9zv 4/4 Running 1 (31h ago) 31h metallb-system metallb-speaker-wr4zr 4/4 Running 1 (31h ago) 31h metallb-system metallb-speaker-wssrj 4/4 Running 1 (31h ago) 31h node-feature-discovery node-feature-discovery-gc-6c44b4465b-6mkv7 1/1 Running 0 31h node-feature-discovery node-feature-discovery-master-6d49f6485d-2jq47 1/1 Running 0 31h node-feature-discovery node-feature-discovery-worker-5bfw6 1/1 Running 0 31h node-feature-discovery node-feature-discovery-worker-6fwzt 1/1 Running 0 31h node-feature-discovery node-feature-discovery-worker-6xxt2 1/1 Running 0 31h node-feature-discovery node-feature-discovery-worker-7vvgx 1/1 Running 0 31h node-feature-discovery node-feature-discovery-worker-plksh 1/1 Running 0 31h node-feature-discovery node-feature-discovery-worker-r842c 1/1 Running 0 31h node-feature-discovery node-feature-discovery-worker-s4gmq 1/1 Running 0 31h ntnx-system konnector-agent-b48c9d66d-tj299 1/1 Running 0 31h ntnx-system nutanix-csi-controller-5bd9c7f8d-jpd2m 7/7 Running 0 31h ntnx-system nutanix-csi-controller-5bd9c7f8d-kkqlz 7/7 Running 0 31h ntnx-system nutanix-csi-node-2gb4q 3/3 Running 0 31h ntnx-system nutanix-csi-node-2xh74 3/3 Running 0 31h ntnx-system nutanix-csi-node-54bjg 3/3 Running 0 31h ntnx-system nutanix-csi-node-5vcc2 3/3 Running 2 (31h ago) 31h ntnx-system nutanix-csi-node-7w6jh 3/3 Running 0 31h ntnx-system nutanix-csi-node-979rw 3/3 Running 2 (31h ago) 31h ntnx-system nutanix-csi-node-lg4zh 3/3 Running 0 31h
NKP Ultimate の Pod の様子
ライセンス キー追加から追加 Pod の起動完了まで、20分くらいかかります。
gowatana@nkp-work-02:~$ kubectl get pods -A NAMESPACE NAME READY STATUS RESTARTS AGE caaph-system caaph-controller-manager-76d9f5c957-6rpjc 1/1 Running 0 32h capa-system capa-controller-manager-948899f8b-x4clk 1/1 Running 0 32h capg-system capg-controller-manager-85976cf9c6-pd5rl 1/1 Running 0 32h capi-kubeadm-bootstrap-system capi-kubeadm-bootstrap-controller-manager-85fc9c88bd-p4tpj 1/1 Running 0 32h capi-kubeadm-control-plane-system capi-kubeadm-control-plane-controller-manager-75bb9ffd66-fvz8t 1/1 Running 0 32h capi-system capi-controller-manager-54dfcd4b6-xhwpt 1/1 Running 0 32h cappp-system cappp-controller-manager-dfc49968b-hvl5l 1/1 Running 0 32h capv-system capv-controller-manager-5bb5dff6b4-wtsth 1/1 Running 0 32h capx-system capx-controller-manager-5bd68554db-pgsvd 1/1 Running 0 32h capz-system azureserviceoperator-controller-manager-55cc766f5-5tv7r 1/1 Running 0 32h capz-system capz-controller-manager-748cd47646-9hnld 1/1 Running 0 32h caren-system cluster-api-runtime-extensions-nutanix-85f5cb84bf-bdg57 1/1 Running 0 32h caren-system helm-repository-59d9fdf9b4-kq2nt 1/1 Running 0 32h cert-manager cert-manager-5997d6b7d9-kd9r6 1/1 Running 0 32h cert-manager cert-manager-cainjector-56b9b6f7-4fxps 1/1 Running 0 32h cert-manager cert-manager-webhook-7c75989cb-swxw2 1/1 Running 0 32h container-object-storage-system container-object-storage-controller-667886cd7f-9hh42 1/1 Running 0 32h default cluster-autoscaler-019d2865-0092-7165-95a0-95f18543452b-6dwz8vc 1/1 Running 0 32h external-secrets external-secrets-56977fdb44-vc6gg 1/1 Running 0 32h external-secrets external-secrets-webhook-6d5f75d6f5-66jcf 1/1 Running 0 32h git-operator-system git-operator-admin-credentials-rotate-29576160-qbprx 0/1 Completed 0 12h git-operator-system git-operator-controller-manager-65f4b7c957-vptzq 2/2 Running 1 (12h ago) 32h git-operator-system git-operator-git-0 4/4 Running 0 32h kommander-default-workspace karma-traefik-certs-kommander-default-workspace-cert-federj6w7v 1/1 Running 0 32h kommander-default-workspace prometheus-traefik-certs-kommander-default-workspace-cert-lwj9v 1/1 Running 0 32h kommander-flux flux-oci-mirror-5c948b8cc5-zzbpg 1/1 Running 0 32h kommander-flux helm-controller-fb7455768-kx5tg 1/1 Running 0 32h kommander-flux image-automation-controller-57fdc5cc95-ctrgl 1/1 Running 0 32h kommander-flux image-reflector-controller-6f4c4669f8-qqd9f 1/1 Running 0 32h kommander-flux kustomize-controller-68bc587994-d6qrj 1/1 Running 0 32h kommander-flux notification-controller-6b8fcc4d9f-m54wn 1/1 Running 0 32h kommander-flux source-controller-6b47b69475-s7grc 1/1 Running 0 32h kommander ai-navigator-app-668bf7958f-cp4z5 1/1 Running 0 22m kommander alertmanager-kube-prometheus-stack-alertmanager-0 2/2 Running 0 21m kommander centralized-grafana-779fbcfc54-qb8rt 2/2 Running 0 20m kommander centralized-opencost-5fcbbbbf78-xm6fp 1/1 Running 4 (19m ago) 21m kommander ceph-cosi-driver-544cf94bf5-fjc9m 2/2 Running 0 18m kommander cluster-observer-2360587938-865656b749-jkrrt 1/1 Running 0 32h kommander create-kommander-thanos-query-stores-configmap-x7j9g 0/1 Completed 0 20m kommander dex-7586959cc9-lvgcv 1/1 Running 0 32h kommander dex-dex-controller-649447fc7c-lg4wm 2/2 Running 0 32h kommander dex-k8s-authenticator-6f9484d94c-tv77h 1/1 Running 0 31h kommander dkp-ceph-prereq-job-f4vrp 0/2 Completed 0 21m kommander etcd-metrics-proxy-4zc5g 1/1 Running 0 21m kommander etcd-metrics-proxy-qkhct 1/1 Running 0 21m kommander etcd-metrics-proxy-s5d28 1/1 Running 0 21m kommander gatekeeper-audit-76f484c6cb-jhlb7 1/1 Running 0 32h kommander gatekeeper-controller-manager-65c6f4bc7b-cr7z9 1/1 Running 0 32h kommander gatekeeper-controller-manager-65c6f4bc7b-zfzpw 1/1 Running 0 32h kommander grafana-logging-6d6cbf446f-qrzsm 2/2 Running 0 22m kommander grafana-loki-loki-distributed-compactor-0 1/1 Running 0 15m kommander grafana-loki-loki-distributed-distributor-697bd4cf8d-t8frt 1/1 Running 0 15m kommander grafana-loki-loki-distributed-gateway-5f9d9c4f88-cjzz2 1/1 Running 0 15m kommander grafana-loki-loki-distributed-ingester-0 1/1 Running 0 15m kommander grafana-loki-loki-distributed-querier-0 1/1 Running 0 15m kommander grafana-loki-loki-distributed-query-frontend-7bcfb8879d-gvh6d 1/1 Running 0 15m kommander grafana-loki-pre-install-mcfts 0/1 Completed 0 21m kommander karma-666c56d4c8-vrgg8 1/1 Running 0 20m kommander karma-traefik-certs-kommander-cert-federation-6d45cc9555-b5czl 1/1 Running 0 32h kommander kommander-appmanagement-c7f68fd56-wzl76 2/2 Running 0 32h kommander kommander-appmanagement-webhook-77fdfdc87b-69p4g 1/1 Running 0 32h kommander kommander-authorizedlister-7d4dc759b-fzt45 1/1 Running 0 32h kommander kommander-bootstrap-s285w 0/1 Completed 0 32h kommander kommander-capimate-dc4f94fb8-h9b8s 1/1 Running 0 32h kommander kommander-capimate-dc4f94fb8-tpbfk 1/1 Running 0 32h kommander kommander-cm-78748b84c5-g7xmz 2/2 Running 0 32h kommander kommander-flux-operator-67775ff756-h2n9v 2/2 Running 0 32h kommander kommander-kommander-ui-5f9b9bf458-2zzzs 1/1 Running 0 32h kommander kommander-licensing-cm-5b67c659df-v77sx 2/2 Running 0 32h kommander kommander-licensing-webhook-678d85db95-mbwhv 1/1 Running 0 32h kommander kommander-operator-6696f9d78d-6bx2d 1/1 Running 0 32h kommander kommander-reloader-reloader-6468cdff65-b8jqr 1/1 Running 0 32h kommander kommander-traefik-6d769b69b6-mc674 1/1 Running 0 32h kommander kommander-traefik-6d769b69b6-p85xh 1/1 Running 0 32h kommander kommander-webhook-6989b5f474-5487z 1/1 Running 0 32h kommander kube-oidc-proxy-56b6d5f7b7-n7r66 1/1 Running 0 32h kommander kube-prometheus-stack-grafana-6497697455-wz76k 2/2 Running 0 21m kommander kube-prometheus-stack-kube-state-metrics-56c4495588-2hhmm 1/1 Running 0 21m kommander kube-prometheus-stack-operator-78fb97cfc8-745zs 1/1 Running 0 21m kommander kube-prometheus-stack-prometheus-node-exporter-2tpg7 2/2 Running 0 21m kommander kube-prometheus-stack-prometheus-node-exporter-c5trj 2/2 Running 0 21m kommander kube-prometheus-stack-prometheus-node-exporter-fg8qb 2/2 Running 0 21m kommander kube-prometheus-stack-prometheus-node-exporter-gkcnk 2/2 Running 0 21m kommander kube-prometheus-stack-prometheus-node-exporter-lgtpn 2/2 Running 0 21m kommander kube-prometheus-stack-prometheus-node-exporter-rlgsl 2/2 Running 0 21m kommander kube-prometheus-stack-prometheus-node-exporter-ts6t9 2/2 Running 0 21m kommander kubernetes-dashboard-api-7cdfc7bff5-xhpn4 1/1 Running 0 21m kommander kubernetes-dashboard-auth-57c5c866bf-vtjjb 1/1 Running 0 21m kommander kubernetes-dashboard-kong-f6975978f-wdm9s 1/1 Running 0 21m kommander kubernetes-dashboard-metrics-scraper-594566d86-kl2sl 1/1 Running 0 21m kommander kubernetes-dashboard-web-6db4fd6676-t5pdm 1/1 Running 0 21m kommander kubetunnel-79665989b8-x4s2c 1/1 Running 0 22m kommander kubetunnel-webhook-cf5f48b84-r8wrz 1/1 Running 0 22m kommander logging-operator-68f677757-bf7r8 1/1 Running 0 22m kommander logging-operator-logging-fluentbit-5qb8p 1/1 Running 0 22m kommander logging-operator-logging-fluentbit-6lhzm 1/1 Running 0 22m kommander logging-operator-logging-fluentbit-cbmtl 1/1 Running 0 22m kommander logging-operator-logging-fluentbit-jr5qg 1/1 Running 0 22m kommander logging-operator-logging-fluentbit-kc9gs 1/1 Running 0 22m kommander logging-operator-logging-fluentbit-qg9fs 1/1 Running 0 22m kommander logging-operator-logging-fluentbit-zzrch 1/1 Running 0 22m kommander logging-operator-logging-fluentd-0 3/3 Running 0 22m kommander logging-operator-logging-fluentd-configch-f649990b 0/1 Completed 0 22m kommander nkp-insights-management-auth-server-6cb64fc654-9vbzt 1/1 Running 0 19m kommander nkp-insights-management-mgmt-cm-8968f6b77-vj5gr 1/1 Running 0 19m kommander object-bucket-claims-check-dkp-loki-vpgrh 0/1 Completed 0 18m kommander object-bucket-claims-check-dkp-velero-qcgmf 0/1 Completed 0 11m kommander opencost-66686b9fb5-cflqr 2/2 Running 3 (20m ago) 21m kommander opencost-pre-install-qmrh6 0/1 Completed 0 21m kommander prometheus-adapter-7645ff5bbd-5xhp4 1/1 Running 0 18m kommander prometheus-kube-prometheus-stack-prometheus-0 3/3 Running 0 21m kommander prometheus-traefik-certs-kommander-cert-federation-7f96c6b7p2gg 1/1 Running 0 32h kommander rook-ceph-crashcollector-nkpnai-md-0-7pmtn-lsqps-72dtl-556f5hfw 1/1 Running 0 17m kommander rook-ceph-crashcollector-nkpnai-md-0-7pmtn-lsqps-hsxfl-6762wxhs 1/1 Running 0 16m kommander rook-ceph-crashcollector-nkpnai-md-0-7pmtn-lsqps-rst8k-766k9rgq 1/1 Running 0 17m kommander rook-ceph-crashcollector-nkpnai-md-0-7pmtn-lsqps-tcrtj-f4ffl2lv 1/1 Running 0 15m kommander rook-ceph-exporter-nkpnai-md-0-7pmtn-lsqps-72dtl-6df97f7b5d6mj9 1/1 Running 0 17m kommander rook-ceph-exporter-nkpnai-md-0-7pmtn-lsqps-hsxfl-7fc4455cbk6z7g 1/1 Running 0 16m kommander rook-ceph-exporter-nkpnai-md-0-7pmtn-lsqps-rst8k-79cbf8ddftbz76 1/1 Running 0 17m kommander rook-ceph-exporter-nkpnai-md-0-7pmtn-lsqps-tcrtj-5579c9b99cv2mb 1/1 Running 0 15m kommander rook-ceph-mgr-a-9f86b55f9-xlpq5 3/3 Running 0 17m kommander rook-ceph-mgr-b-66d7f6b8d-k88d8 3/3 Running 0 17m kommander rook-ceph-mon-a-9954dc6d5-2wzbk 2/2 Running 0 19m kommander rook-ceph-mon-b-8846c875-4xg86 2/2 Running 0 18m kommander rook-ceph-mon-c-75dc8c4987-s69n6 2/2 Running 0 17m kommander rook-ceph-operator-765fd6f969-zbbl7 1/1 Running 0 21m kommander rook-ceph-osd-0-bb6876f48-6vgm6 2/2 Running 0 16m kommander rook-ceph-osd-1-7dbf66cb44-vqfxf 2/2 Running 0 16m kommander rook-ceph-osd-2-6b98d97866-dwvwl 2/2 Running 0 15m kommander rook-ceph-osd-3-b878d8d56-68kth 2/2 Running 0 15m kommander rook-ceph-osd-prepare-rook-ceph-osd-set1-data-0pnwjx-jsrr7 0/1 Completed 0 16m kommander rook-ceph-osd-prepare-rook-ceph-osd-set1-data-1dr4dq-646d5 0/1 Completed 0 16m kommander rook-ceph-osd-prepare-rook-ceph-osd-set1-data-22prfw-rvp95 0/1 Completed 0 16m kommander rook-ceph-osd-prepare-rook-ceph-osd-set1-data-32ztsd-lj6t2 0/1 Completed 0 16m kommander rook-ceph-rgw-dkp-object-store-a-56bf59495-9xnvr 2/2 Running 0 14m kommander rook-ceph-rgw-dkp-object-store-a-56bf59495-nrp6n 2/2 Running 0 14m kommander rook-ceph-tools-5678c68d57-8dk86 1/1 Running 0 21m kommander runtime-extension-kommander-5545b9dd54-dkkf2 1/1 Running 0 32h kommander thanos-query-5f94b79955-hvr5d 1/1 Running 0 19m kommander traefik-forward-auth-mgmt-675d8cdb54-s6vjp 1/1 Running 0 31h kommander velero-backup-storage-location-updater-9f7ff88f-r64ll 1/1 Running 0 12m kommander velero-c65f68747-s76sb 1/1 Running 0 15m kommander velero-pre-install-bndld 0/1 Completed 0 21m kube-federation-system kubefed-admission-webhook-75dd7fdb9-kwnqg 1/1 Running 0 32h kube-federation-system kubefed-controller-manager-5545598dd9-glfv2 1/1 Running 0 32h kube-federation-system kubefed-controller-manager-5545598dd9-vdqrc 1/1 Running 0 32h kube-system cilium-b2d5t 1/1 Running 0 32h kube-system cilium-envoy-49hlj 1/1 Running 0 32h kube-system cilium-envoy-98464 1/1 Running 0 32h kube-system cilium-envoy-gdgjf 1/1 Running 0 32h kube-system cilium-envoy-jr64q 1/1 Running 0 32h kube-system cilium-envoy-mqfp5 1/1 Running 0 32h kube-system cilium-envoy-mwlww 1/1 Running 0 32h kube-system cilium-envoy-r78sh 1/1 Running 0 32h kube-system cilium-jxkg6 1/1 Running 0 32h kube-system cilium-krktq 1/1 Running 0 32h kube-system cilium-nvxw5 1/1 Running 0 32h kube-system cilium-operator-555cf9bf96-2ncss 1/1 Running 0 32h kube-system cilium-operator-555cf9bf96-2p9td 1/1 Running 0 32h kube-system cilium-pcdpk 1/1 Running 0 32h kube-system cilium-pzcqx 1/1 Running 0 32h kube-system cilium-zzs2d 1/1 Running 0 32h kube-system coredns-66bc5c9577-m6fzt 1/1 Running 0 32h kube-system coredns-66bc5c9577-tgwvm 1/1 Running 0 32h kube-system etcd-nkpnai-d9vzg-h5cbv 1/1 Running 0 32h kube-system etcd-nkpnai-d9vzg-km4cq 1/1 Running 0 32h kube-system etcd-nkpnai-d9vzg-pk5z7 1/1 Running 0 32h kube-system hubble-relay-95b99b996-z5bvk 1/1 Running 1 (32h ago) 32h kube-system kube-apiserver-nkpnai-d9vzg-h5cbv 1/1 Running 0 32h kube-system kube-apiserver-nkpnai-d9vzg-km4cq 1/1 Running 0 32h kube-system kube-apiserver-nkpnai-d9vzg-pk5z7 1/1 Running 0 32h kube-system kube-controller-manager-nkpnai-d9vzg-h5cbv 1/1 Running 0 32h kube-system kube-controller-manager-nkpnai-d9vzg-km4cq 1/1 Running 0 32h kube-system kube-controller-manager-nkpnai-d9vzg-pk5z7 1/1 Running 0 32h kube-system kube-scheduler-nkpnai-d9vzg-h5cbv 1/1 Running 0 32h kube-system kube-scheduler-nkpnai-d9vzg-km4cq 1/1 Running 0 32h kube-system kube-scheduler-nkpnai-d9vzg-pk5z7 1/1 Running 0 32h kube-system kube-vip-nkpnai-d9vzg-h5cbv 1/1 Running 0 32h kube-system kube-vip-nkpnai-d9vzg-km4cq 1/1 Running 0 32h kube-system kube-vip-nkpnai-d9vzg-pk5z7 1/1 Running 0 32h kube-system nutanix-cloud-controller-manager-577bdb9865-xx75w 1/1 Running 0 32h kube-system snapshot-controller-58cb99fc45-k4nf7 1/1 Running 0 32h metallb-system metallb-controller-84cfb9944d-fkw2t 1/1 Running 0 32h metallb-system metallb-speaker-h7zsc 4/4 Running 1 (32h ago) 32h metallb-system metallb-speaker-mgfr8 4/4 Running 1 (32h ago) 32h metallb-system metallb-speaker-ppxk8 4/4 Running 0 32h metallb-system metallb-speaker-tkpgk 4/4 Running 1 (32h ago) 32h metallb-system metallb-speaker-tn9zv 4/4 Running 1 (32h ago) 32h metallb-system metallb-speaker-wr4zr 4/4 Running 1 (32h ago) 32h metallb-system metallb-speaker-wssrj 4/4 Running 1 (32h ago) 32h node-feature-discovery node-feature-discovery-gc-6c44b4465b-6mkv7 1/1 Running 0 32h node-feature-discovery node-feature-discovery-master-6d49f6485d-2jq47 1/1 Running 0 32h node-feature-discovery node-feature-discovery-worker-5bfw6 1/1 Running 0 32h node-feature-discovery node-feature-discovery-worker-6fwzt 1/1 Running 0 32h node-feature-discovery node-feature-discovery-worker-6xxt2 1/1 Running 0 32h node-feature-discovery node-feature-discovery-worker-7vvgx 1/1 Running 0 32h node-feature-discovery node-feature-discovery-worker-plksh 1/1 Running 0 32h node-feature-discovery node-feature-discovery-worker-r842c 1/1 Running 0 32h node-feature-discovery node-feature-discovery-worker-s4gmq 1/1 Running 0 32h ntnx-system konnector-agent-b48c9d66d-tj299 1/1 Running 0 32h ntnx-system nutanix-csi-controller-5bd9c7f8d-jpd2m 7/7 Running 0 32h ntnx-system nutanix-csi-controller-5bd9c7f8d-kkqlz 7/7 Running 0 32h ntnx-system nutanix-csi-node-2gb4q 3/3 Running 0 32h ntnx-system nutanix-csi-node-2xh74 3/3 Running 0 32h ntnx-system nutanix-csi-node-54bjg 3/3 Running 0 32h ntnx-system nutanix-csi-node-5vcc2 3/3 Running 2 (32h ago) 32h ntnx-system nutanix-csi-node-7w6jh 3/3 Running 0 32h ntnx-system nutanix-csi-node-979rw 3/3 Running 2 (32h ago) 32h ntnx-system nutanix-csi-node-lg4zh 3/3 Running 0 32h
つづく。
