Skip to content
UMKIMDocs

fleet-site

Experimental: FleetClusterJoin and its optional integrations require independent operational validation.

fleet-site manages FleetSite, FleetClusterJoin, FleetApp, and FleetChart resources and coordinates greenfield workload-cluster bootstrap from the hub. Its CRDs use fleet.umkim.com/v1alpha1.

Resource Purpose
FleetSite Site registry, edge.ingress[], optional hubGateway, jumphost, optional provisioner
FleetClusterJoin Cluster bootstrap, validation, provisioning, node/storage/platform ops
FleetApp Helm chart install onto a selected Ready cluster
FleetChart Hub library entry for a published OCI chart
FleetAcme, FleetIngressAuth, FleetLLM Optional integrations

An FCJ normally progresses through Pending → ScriptIssued → Enrolled → Validated → Provisioning → Ready. Failures are reported in resource status. Review the resource, controller Job, and cluster state before retrying.

Cluster deletion defaults to detach: hub-managed resources are removed while K3s and workloads on the spoke are preserved. Full Kubernetes cleanup is opt-in (cleanupKubernetes=true).

Site ingress

Section titled “Site ingress”

Each site carries spec.edge.ingress[] (port_forward and/or reverse_proxy) and optional hubGateway. Clusters bind with FleetClusterJoin.spec.edge.ingressId. When a site has exactly one port_forward ingress, an empty ingressId may auto-bind at reconcile time.

Match per-app FleetApp route.mode to site edge:

  • hub with site hubGateway (and hub zone DNS)
  • site with a bound port_forward or reverse_proxy ingress
  • none for chart-owned or internal ingress

Inventory examples: examples/site-profiles/. See fleet-edge.

Optional provisioner connector

Section titled “Optional provisioner connector”

FleetSite.spec.provision binds an optional metal/VM provisioner (noop default, yaidap). The hub stores a native Ansible/YAIDAP YAML inventory tree and syncs it to the site jumphost.

Operators edit inventory via console, umkim site inventory, or API. Provision actions (create_vm, install_bm, discover, …) run async on the jumphost. When guests are Ready, cluster-apply creates FleetClusterJoin from {cluster}.yml + group_vars/{cluster}.yml.

See YAIDAP integration for connector binding, API, and troubleshooting. Operator runbook: Site provisioning.

Hub independence

Section titled “Hub independence”

Spokes are independent K3s control planes. The hub registers and manages them via OCM; it does not run their workloads.

  • Hub down: console and hub-routed URLs stop; workloads on spokes keep running.
  • Detach: default delete keeps the spoke cluster and Helm releases.

Helm charts (FleetApp)

Section titled “Helm charts (FleetApp)”

After the cluster is Ready, install OCI charts with FleetApp. You must set spec.clusterName; the controller does not choose a cluster. Use capability candidate filters when helpful.

KubeVirt is an optional spoke platform add-on (Multus + CDI + KubeVirt), not a chart-install path. See Fleet KubeVirt platform.

Post-Ready cluster operations

Section titled “Post-Ready cluster operations”

When a cluster is Ready, operators use the console Operations panel, umkim, or the fleet-site API:

Operation CLI / console
Add / remove / replace node `umkim node add
Operation history / logs `umkim node-op history
Enable local RWX (nfs_local) umkim cluster storage enable-local
Enable KubeVirt platform umkim cluster platform enable-kubevirt
List / power / console VMs `umkim vm list

Node lifecycle, storage configure, and KubeVirt install share one job queue (mutual exclusion). Completed storage and KubeVirt runs are recorded in nodeOperationHistory as configureStorage / configureKubevirt.

Hub Ansible jobs reach cluster nodes through FleetClusterJoin.spec.jumphost, or FleetSite.spec.jumphost when the cluster field is empty. Use user@host:port (port optional). When set, jobs SSH to management LAN addresses via ProxyJump instead of the WireGuard overlay.

Edit jumphost in console Sites detail (Connectivity) or umkim site patch. Edit ingress entries in Sites detail or umkim site ingress list|add|set|delete. Per-cluster ingress binding: console Clusters detail or umkim cluster patch --ingress-id.

HA bundle hub provision jobs use a 90-minute deadline (single-node bundle jobs remain 30 minutes). Use umkim cluster wait --timeout 90m for HA clusters.

Each node operation follows validate → execute → validate. Completed operations are appended to FleetClusterJoin.status.nodeOperationHistory (newest first, capped at 20).

Management API

Section titled “Management API”

Public fleet-site management API access requires authenticated use through fleet-console-auth. Use the console or an approved service-account client; do not publish direct controller endpoints, credentials, or bootstrap material.

Section titled “Related”