Skip to content
UMKIMDocs

Fleet Llama profiles

Hub-stored FleetLlamaProfile custom resources (fleet console namespace) describe llama-cpp-helm value overlays. The fleet-site API merges each profile with fleet runtime values and the Harbor image overlay — same as files under deploy/llama-cpp-helm/ in the UMKIM repo.

CRDs

Section titled “CRDs”
Kind Purpose
FleetLlamaProfile spec.profileYaml, chart pin, optional spec.system for seeded built-ins
FleetLlamaProfileTest One verify session (quick or extended)

System profiles (for example qwen35-2b-ud-4g) are created on controller start when missing.

HTTP API (/llama-profiles)

Section titled “HTTP API (/llama-profiles)”
Method Path Notes
GET/POST /llama-profiles List / create
GET/PATCH/DELETE /llama-profiles/{name} System delete needs ?force=true
GET /llama-profiles/{name}/values Merged valuesYaml
POST /llama-profiles/{name}/verify clusterName, mode (quick | extended), optional wait
GET /llama-profiles/tests Optional extended=true, active=true
GET/DELETE /llama-profiles/tests/{id} Status / end extended test
POST /llama-profiles/tests/{id}/chat/completions Hub proxy to spoke (extended, Ready only)

Legacy GET /charts/llama-cpp-helm/profiles lists profile ids for the Apps install picker.

Verify modes

Section titled “Verify modes”
  • Quick — install FleetApp on a Ready cluster, smoke GET /v1/models over WireGuard, delete app, mark test Succeeded.
  • Extended — keep FleetApp, hub proxy chat; end with DELETE on the test.

Test FleetApps use annotations fleet.umkim.com/llama-profile and fleet.umkim.com/llama-profile-test.

CLI and console

Section titled “CLI and console”
  • umkim llm profile … — CRUD, values, verify, test chat, test end
  • Console Apps → Llama profiles — CRUD, verify modal, extended chat

Install with umkim app create --llama-profile NAME or the console profile picker.

Spoke DNS and hostNetwork

Section titled “Spoke DNS and hostNetwork”

Chart 0.1.4+ sets dnsPolicy: Default when hostNetwork is enabled so Hugging Face downloads use the node resolver (avoids pod DNS failures on some GPU spokes).

Section titled “Related”