The Problem

I have been slowly migrating my Proxmox infrastructure to Kubernetes on Talos Linux. Most workloads containerize just fine, but some things stubbornly remain VMs - appliances, one-off test boxes, anything that wants a full kernel of its own. Running a whole separate Proxmox host for a handful of VMs means two platforms to patch, two backup strategies, and two ways to do networking.

KubeVirt solves this by making VMs first-class Kubernetes resources. The VM definition lives in git next to everything else, ArgoCD deploys it, and the same storage and network stack applies to both pods and VMs.

Why KubeVirt

• Unified management - one platform for containers and VMs
• Declarative configuration - VMs are just Kubernetes resources
• Live migration - move running VMs between nodes without downtime
• GitOps friendly - VM definitions version controlled and deployed by ArgoCD

Components

Additional pieces on the metal environment:

• Multus CNI - secondary network support for VMs
• Passt CNI - user-space networking without root privileges
• Ubuntu 24.04 base image - golden image for VM cloning

Everything is deployed with Kustomize per environment:

# Bare-metal with LINSTOR storage
kubectl apply -k envs/metal

Storage

Storage is split between two purposes:

Block mode with RWX access is what makes live migration possible - both the source and destination node need to attach the same disk during the move. The linstor-kubevirt-tier2 class sets allow-two-primaries on the DRBD resource for exactly this reason.

Networking with Passt

Passt provides user-space networking for VMs without requiring root or elevated privileges. This matters on Talos, where the immutable filesystem and locked-down host make traditional bridge networking painful.

The binding plugin configuration in the KubeVirt CR:

spec:
  configuration:
    network:
      binding:
        passt:
          networkAttachmentDefinition: kubevirt/netbindingpasst
          sidecarImage: quay.io/kubevirt/network-passt-binding:v1.6.0
          migration: {}
          computeResourceOverhead:
            requests:
              memory: "250Mi"

And the NetworkAttachmentDefinition it points at:

apiVersion: "k8s.cni.cncf.io/v1"
kind: NetworkAttachmentDefinition
metadata:
  name: netbindingpasst
  namespace: kubevirt
spec:
  config: |
    {
      "cniVersion": "1.0.0",
      "name": "netbindingpasst",
      "plugins": [
        {
          "type": "kubevirt-passt-binding"
        }
      ]
    }

KubeVirt Configuration

The KubeVirt CustomResource with the feature gates that matter:

apiVersion: kubevirt.io/v1
kind: KubeVirt
metadata:
  name: kubevirt
  namespace: kubevirt
spec:
  configuration:
    developerConfiguration:
      featureGates:
        - LiveMigration
        - NetworkBindingPlugins
        - ExpandDisks
    smbios:
      sku: "TalosCloud"
      manufacturer: "Talos Virtualization"
      product: "talosvm"
  workloadUpdateStrategy:
    workloadUpdateMethods:
    - LiveMigrate

workloadUpdateMethods: LiveMigrate means KubeVirt upgrades don’t restart my VMs - they migrate them to an updated virt-launcher instead.

VM Creation Pattern

I use a golden image approach:

1. Base image - Ubuntu 24.04 cloud image imported once as a DataVolume in the kubevirt namespace
2. Clone source - RBAC allows other namespaces to clone from the base image
3. New VMs - created by cloning, so provisioning is seconds instead of a full image download

The RBAC for cross-namespace clone access:

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: cdi-clone-source
  namespace: kubevirt
rules:
- apiGroups: ["cdi.kubevirt.io"]
  resources: ["datavolumes", "datavolumes/source"]
  verbs: ["get", "list", "watch", "create"]
- apiGroups: [""]
  resources: ["persistentvolumeclaims"]
  verbs: ["get", "list", "watch"]

Verification

# Check KubeVirt is running
kubectl get kubevirt -n kubevirt

# Check CDI is running
kubectl get cdi

# Verify each node exposes KVM devices
kubectl get nodes -o json | jq '.items[].status.allocatable."devices.kubevirt.io/kvm"'

# Access KubeVirt Manager UI
kubectl port-forward -n kubevirt-manager svc/kubevirt-manager 8080:8080

The virtctl plugin provides the VM-specific commands:

kubectl krew install virt

virtctl console vm-name
virtctl vnc vm-name
virtctl migrate vm-name

Sharp Edges

The upstream KubeVirt Manager patch:

patches:
  - target:
      kind: Deployment
      name: kubevirt-manager
    patch: |-
      - op: replace
        path: /spec/selector/matchLabels/kubevirt-manager.io~1version
        value: v1.5.3
      - op: replace
        path: /spec/template/spec/containers/0/image
        value: kubevirtmanager/kubevirt-manager:1.5.3

Outstanding Work

1. Migrate remaining Proxmox VMs - the whole point of this exercise
2. VM backup strategy - snapshot-based backups of LINSTOR volumes vs guest-level backups
3. Windows guest testing - virtio driver injection via CDI

References

• https://kubevirt.io/
• https://kubevirt-manager.io/
• https://docs.siderolabs.com/talos/v1.9/advanced-guides/install-kubevirt