Automatic SR-IOV Networking with DRA and Multus

A workload requests devices; its allocated VFs determine the network. Administrators define reusable network templates and map eligible VFs to them. The DRA driver binds each allocated VF to a private NetworkAttachmentDefinition (NAD), then supplies the Pod’s network selections. Multus executes the resulting CNI configuration, including the template’s IPAM.

• 01 / REQUESTDescribe device needs

  The workload uses DRA claims and topology constraints, with no manual NAD or rail selection.

• 02 / BINDFreeze the attachment

  The driver creates one Pod-local shadow NAD per VF, with its PCI address and original IPAM settings.

• 03 / ATTACHRun the CNI configuration

  Multus reads the current Pod and NADs, saves the delegates, and invokes SR-IOV CNI and IPAM.

Revision 5 replaces workload rail/NAD selection with two GPU/NIC pairs matched by PCIe root; each allocated VF determines its own network/IPAM template. It retains optional Pod-relative source namespaces, Pod-local protected shadows, and delegated IPAM. This specification is for maintainers and implementers working across dra-driver-sriov, multus-cni, and sriov-network-operator.

Reader map

Start with the architecture, follow the attachment sequence, or jump to the GPU/NIC example. Each diagram links to a full-size SVG; on narrow screens, scroll the diagram horizontally to read the details.

1. Decision and implementation assignment

All three requested UX outcomes are feasible:

1. A workload requests SR-IOV resources through DRA; the integration supplies the Multus network selections.
2. An administrator supplies reusable NAD templates without a resource-name annotation or a static PCI address; the integration binds each template to the allocated VF.
3. Administrator-managed device attributes select a different template for each VF or rail, including rail-specific IPAM. In the operator-managed path, users configure these attributes on SriovNetworkNodePolicy and reference SriovNetwork; the operator generates the driver-facing mapping.

Implementation target: the DRA driver creates a device-bound shadow NAD and patches the Pod; Multus executes that ordinary CNI configuration without a DRA/resourceName lookup. Keep allocation, per-VF template selection, and PCI injection inside the driver. Do not introduce a new Multus claim resolver, a ResourceClaim-status handoff, or another resource-name matching convention. Multus still performs its ordinary Pod and NAD reads.

This is a source-backed engineering specification, not a claim that the feature exists or has passed live-cluster tests. The reviewed upstream code does not provide this automatic flow today. The example manifests describe the target behavior after implementation. Exact API names introduced here are proposals, not existing portable Multus standards.

The most important design correction is mandatory: a successful Pod patch in NodePrepare does not make the thick Multus informer’s Pod fresh. Multus must read the current Pod before CNI ADD. A cached Pod without the new annotation can otherwise succeed with only its primary network. A newly created NAD has a separate informer visibility race. These are object-read consistency fixes; they do not require DRA lookup logic. M1 M2

The simplifying boundary is explicit: the driver understands allocation; Multus understands a Pod’s network selections and NAD configuration; sriov-cni consumes deviceID. A second resourceName attribute is unnecessary on this path. Internal ownership metadata supports retries and prevents accidental reuse, but users do not configure or synchronize it.

1.1 Nonnegotiable correctness rules

• Multus is the sole owner of secondary CNI ADD and DEL. Do not enable the driver’s NRI attachment path in MULTUS mode.
• An automatic attachment identifies an exact allocated device, not a position in a resource-name queue.
• An existing-but-stale Pod cache must not cause ADD to omit the driver-published network selections.
• Never change the source NAD to contain a Pod-specific PCI address.
• Each runtime NAD is specific to one Pod, one claim, and one allocated VF. Its Create includes the Pod ownerReference and the driver protection finalizer atomically.
• Pod/NAD deletion requests enqueue cleanup; only authorized resource release permits hardware cleanup and finalizer removal.
• Preserve the logical CNI network name and all IPAM allocation-domain fields.
• Prepare succeeds only after durable local state, shadows, and Pod selections are ready.
• Recovery must work without another NodePrepare invocation.
• DEL uses the configuration captured for that sandbox, even after API objects are changed or deleted.
• Unsupported ownership or topology must produce a precise error before a VF is moved.

2. Reviewed revisions and evidence

Rebase the implementation onto the current intended branches before coding, read repository instructions, and recheck these paths. Record any behavior that changed since these snapshots. Do not silently substitute an unreleased PR for released functionality.

The driver README’s broader minimum-version statement is not proof of this new feature’s compatibility. Start with Kubernetes v1.36.3, pinned compatible images, and actual API/feature-gate verification. Test any additional Kubernetes or OpenShift version explicitly. A dependency’s minor version is not, by itself, a supported-server-version declaration.

2.1 Existing code paths

Detailed source references are at the end. D1 D2 D5 D6 M1 M2 M5 O1 O3 O4

3. Feasibility and lifecycle

3.1 Does NodePrepare run before Multus ADD?

In the reviewed kubelet, kuberuntime_manager.go calls PrepareDynamicResources at line 1587, then createPodSandbox at line 1599. The runtime performs network setup during sandbox creation. Therefore synchronous completion of the driver work is early enough for the initial sandbox. K1

This ordering is necessary but insufficient. Multus independently reads Kubernetes objects. Its cached Pod can predate the driver patch, and its NAD informer can predate shadow creation. Do not solve these races with sleeps, initial informer HasSynced, or a refresh conditional on a newly written annotation: the stale Pod may not contain that annotation either. M1 M2

The kubelet DRA API carries claim namespace/name/UID, not a Pod. reservedFor is a consumer list, not a guarantee that the first element is the current Pod. The driver’s existing single-consumer restriction makes an initial exclusive-Pod implementation practical, but it does not remove the need for identity checks. K2 D1

3.2 Prepare is not a per-sandbox event

The kubelet caches prepared claims and can skip the driver RPC for a sandbox recreation or another consumer. In v1.36.3 the in-memory prepared flag is reset when kubelet restores its own checkpoint, so a kubelet restart is different from an ordinary retry. Do not assume every sandbox recreation causes prepare, or that every restart has identical behavior. K3 K7

A second Pod sharing a claim can reach Multus without a new prepare callback. Changing the driver’s len(reservedFor) > 1 check would not make exclusive network devices safely shareable. Shared claims remain unsupported. A lightweight check of a generated NAD’s bound Pod UID can reject accidental reference by another Pod without reading a ResourceClaim.

3.3 Target sequence

4. Supported scope and feature controls

4.1 First supported scope

Support Linux, Kubernetes v1.36.3 as the first implementation and test target, MULTUS mode, ordinary Pods using explicit ResourceClaims or ResourceClaimTemplates, multiple claims per Pod, multiple exclusive VFs per claim, Ethernet kernel/netdevice VFs, one device-consuming sriov plugin per NAD, and optional approved metaplugins such as tuning in a conflist. Include regular Whereabouts and host-local with documented single-node/disjoint-range limits.

An explicit claim may be reused sequentially after successful unprepare. Its new Pod gets a new binding identity and source snapshot. Concurrent reuse is unsupported. The driver rejects multiple consumers when prepare runs; the metadata guard rejects a Pod selecting another Pod’s shadow. Neither can detect a second consumer that skips NodePrepare and has no generated metadata. Prefer one ResourceClaimTemplate-generated claim per Pod, and prohibit sharing managed explicit claims through workload policy. Do not claim that the no-lookup path independently enforces future exclusivity.

Keep these paths outside the initial managed-mode support envelope, with explicit errors rather than accidental behavior: hostNetwork Pods; missing/Node/PodGroup/unknown reservation kinds; more than one reserved consumer; adminAccess: true; share IDs or shared/consumable-capacity allocations; VFIO/DPDK; InfiniBand; source NAD with empty file-backed configuration; multiple device-consuming plugins in one list; arbitrary CNI topology/bond construction; dynamic network hot-plug; Whereabouts fast node slices. Existing manual mode remains available for its previously supported combinations.

Extended-resource DRA claims are a separate optional milestone. Resolve pod.status.extendedResourceClaimStatus explicitly if implementing it. Otherwise reject an extended-resource allocation for the managed driver with UnsupportedManagedClaimKind; do not claim extended-resource UX works merely because the old Multus path supports it. GPU/other-driver allocations are unaffected.

4.2 Proposed flags and configuration

Add driver flag --multus-auto-attach / environment variable MULTUS_AUTO_ATTACH, default false. It is valid only with CONFIGURATION_MODE=MULTUS. When true, every allocated VF handled by that driver on the configured node must satisfy this document’s managed-mode rules and have a resolvable template. Do not silently fall back to manual mode for an unmapped VF.

The generated runtime NAD always lives in the consuming Pod’s namespace. Resolve the original source NAD from the effective VfConfig or per-device reference attributes: use an explicit source namespace when provided, otherwise the verified Pod namespace. There is no workload-namespace list or source-template namespace allowlist in the driver. Normal Kubernetes API authorization applies to reads and writes. The default installation grants the node driver the Pod/NAD access needed across namespaces, including orphan inspection.

Add a generic Multus read-consistency setting, rather than a DRA resolver setting:

{
  "liveNetworkConfigOnAdd": true
}

This proposed setting must be wired through both thin and thick configuration parsing. Default disabled preserves old behavior. When enabled, use live Pod and NAD GETs for that ADD request. A minimal implementation makes a request-local copy of ClientInfo and clears its PodInformer and NetDefInformer; existing helpers then use their API paths. Never clear the informers on the shared daemon client. Thin mode already uses live reads when no informers are configured. No ResourceClaim or ResourceSlice reads are added.

The first implementation intentionally uses a node/deployment-level opt-in. Do not invent a partially compatible per-claim switch during implementation. Mixing old manual checkpoints and new automatic bindings on an enabled node requires a separate migration design; use drained nodes and recreated Pods for activation.

4.3 Deployment compatibility is a correctness prerequisite

Existing Multus can execute a fixed-PCI shadow. Thick mode still needs the read-consistency fix, and the small metadata-only ownership guard below requires its patched build. Enable the driver flag only after the required Multus behavior is deployed and verified on every target node. Deliver a preflight for the actual daemon/shim configuration, build identity, live-read setting, required binaries, persistent cache mounts, and RBAC. Do not add an allocation protocol handshake.

A first release can enforce this operationally through pinned images and a mandatory preflight. Do not describe a Pod annotation, a stale Node annotation, or an image tag alone as proof of the running daemon’s read behavior.

5. Administrator configuration and precedence

5.1 Proposed per-device attributes

The driver consumes the existing DeviceAttributes CRD and SriovResourcePolicy.spec.configs[].deviceAttributesSelector. Direct-driver users may author these objects. Operator-managed users configure the proposed SriovNetworkNodePolicy.spec.dra.attributes instead; section 12.2 defines how the operator resolves SriovNetwork references and generates this contract automatically:

When attributes select a template, netAttachDefName must be a nonempty string. netAttachDefNamespace may be absent or a string with value “”; both mean the verified consuming Pod’s namespace. A nonempty namespace must be a valid Kubernetes namespace name. Reject a namespace-only reference, an empty name, or a wrong value type. If neither reference key is present, the object carries ordinary custom metadata and the existing VfConfig fallback remains available. Attribute strings are limited to 64 characters; namespace names fit, but longer valid NAD names do not. Fail clearly and never truncate. Longer NAD names can use the typed VfConfig reference path. Enforce the total final device attribute/capacity limit before ResourceSlice publication. D10 K4

Keep name and optional namespace as separate attributes. Resolve a missing/empty namespace only after the consuming Pod is verified; never default it to the DeviceAttributes, policy, driver or operator namespace. For example, a name-only attribute object in dra-driver-sriov used by a Pod in ai-workloads selects ai-workloads/. An explicit infra/ source is also valid, while its generated shadow still lives in ai-workloads.

There is no predefined rail attribute or special rail handling. In the topology example, workloads request GPUs and interfaces with PCIe-root matching constraints; their DeviceClasses and claims do not select a NAD name, resourceName or rail. After allocation, the SR-IOV driver resolves each selected VF’s own netAttachDefName reference and IPAM configuration. Administrators may still supply ordinary custom attributes through the generic attribute APIs when their scheduling policy needs them.

The driver already discovers k8s.cni.cncf.io/deviceID from the VF PCI address. Users do not need to add that hardware attribute themselves today. What this design removes from user configuration is the synthetic resourceName linkage and repeated Pod network selection. Obtain the PCI address from trusted discovery for the allocated device, never from a user-written attribute override. D2 D3

5.2 Resolution precedence

For each actual allocation result:

1. Resolve the effective opaque VfConfig using the correct class/claim and request/subrequest matching rules. The pinned Kubernetes resourceclaim.ConfigForResult helper already handles wildcard, parent and exact subrequest matching. Filter its results by this driver’s Opaque.Driver, apply class configs then claim configs, and retain list order within each source. K5
2. For a first managed prepare, atomically snapshot the allocated device’s current policy eligibility and authoritative attributes. Reject an ineligible device before configuration fallback or hardware changes. Preserve full discovered inventory; eligibility is the advertised policy set, not physical existence. Cached Pending/Ready bindings and release use their frozen checkpoint instead. See section 12.2.5 for unresolved selectors.
3. If a valid per-device NAD name reference exists, resolve its optional namespace against the verified Pod namespace. If an effective explicit VfConfig reference exists too, resolve that namespace the same way and require the concrete namespace/name pairs to agree. A disagreement is an error, not a tenant override of the physical mapping.
4. If no per-device reference exists, use VfConfig.netAttachDefName and its namespace; an omitted/empty namespace resolves to the verified Pod namespace. Do not use fallback to hide malformed reference attributes.
5. If neither path yields a valid source name, return MissingNetworkTemplate before hardware changes. Resolve the namespace exactly once from the verified Pod and enforce normal API read authorization.

Specify reference replacement during VfConfig layering: a later config that supplies netAttachDefName replaces the name and namespace together, including resetting the namespace to empty when it omits that field. Default the final empty namespace to the Pod namespace after merging. Thus class A@infra plus claim B@omitted resolves to B@PodNamespace, never B@infra. A later nonempty namespace without a new name may override the current name’s namespace; an empty namespace-only string cannot express a distinct reset with the current non-pointer API fields. Test these rules in Override and matching-config resolution.

An interface name in VfConfig is supported only if it resolves to one device or is otherwise uniquely specified per device. A common explicit name on a count-two request is ambiguous and must fail.

For the common case where every VF in a DeviceClass uses one NAD, set the existing VfConfig reference once in DeviceClass.spec.config. No new DeviceAttributes object or NAD-reference attribute is required. Use per-device reference attributes only when the physical VF/rail determines the template. Section 14 includes both patterns.

5.3 Existing policy precedence must be visible

The reviewed driver selects the alphabetically first matching policy, then the first matching config. It does not merge across matching configs/policies. Within the winning selector, matching DeviceAttributes are merged in name order and the last duplicate key wins. D5

For the semantic NAD-reference keys, reject conflicting references among matching objects and identify the conflicting objects. Preserve ordinary custom-attribute precedence. Each object supplying reference metadata must contain its own nonempty name and, optionally, its namespace; never assemble a name from one object and a namespace-only fragment from another.

The policy controller has no consuming Pod. Normalize missing and empty namespace to one symbolic Pod-namespace value and compare duplicate references in that form. Require matching objects to agree on name and symbolic/explicit namespace. Conservatively reject a defaulted reference combined with an explicit-namespace reference at this stage, even when they might agree for a future Pod; do not let generic last-writer merging change a default into an explicit namespace. Only NodePrepare resolves the symbolic namespace and compares the chosen reference with the effective VfConfig. A valid name-only reference is eligible for publication; do not attempt to find its source in the policy namespace.

Prefer nonoverlapping PF/VF filters. A supplemental policy is not a safe metadata overlay. Avoid unimplemented drivers filters, ignored matchFields, and pfName#vf-range syntax until explicitly supported; reject them for the managed configuration instead of pretending they filter correctly.

5.4 Source changes and snapshot semantics

Freeze the resolved nonempty source namespace/name, source NAD UID, resourceVersion, fully materialized configuration, and configuration digest when a binding plan is first durably created. Retries and release use that concrete source identity; they never re-default a namespace from mutable policy/configuration. Later changes to the source NAD or device attributes affect new bindings, not retries or sandbox recreations for an existing binding. Reading a mutable ResourceSlice again is not a substitute for the frozen plan.

Class/claim configuration should be read from the allocation result’s stored config, rather than refetching a mutable DeviceClass. Per-device policy attributes still need the stable-mapping constraint described in section 15.

The source may be deleted after preparation. Existing prepared bindings continue from their snapshot; new bindings fail until a valid template exists. A configuration fix to a stuck binding that already has a durable snapshot requires recreating the workload/claim binding, unless an explicit administrative reset operation is added later. Document this so automatic repair does not unexpectedly change a VF’s L3 network.

6. Identity, persisted data, and interface assignment

6.1 Resolve and verify the consumer

Implement ResolvePodForClaim(ctx, claim) in a focused new driver package, suggested path pkg/multus:

1. Require a nonnil claim, namespace/name/UID, and allocation.
2. Require exactly one status.reservedFor entry with empty API group, resource pods, and nonempty name and UID. The consumer namespace is the claim namespace.
3. Live GET that Pod. Verify exact UID, spec.nodeName equals the configured node, and the node’s UID equals the process’s recorded node identity.
4. For a new prepare, reject a terminating Pod and hostNetwork. For retry/release, use the lifecycle-specific behavior below rather than treating a deleted Pod as a different replacement.
5. Resolve every pod.spec.resourceClaims entry using Kubernetes resourceclaim.Name semantics. Direct resourceClaimName comes from spec; a template name resolves through status.resourceClaimStatuses. When Name returns mustCheckOwner, use resourceclaim.IsForPod(pod, claim, false) on the pinned v0.36.3 API; false excludes PodGroup ownership from this initial scope. A nil resolved name without error means an intentionally absent claim and must be skipped. Deduplicate the same actual claim UID referenced through aliases.
6. Verify that this Pod actually references this exact claim and that the reservation authorizes it. Do not search for an arbitrary Pod using a label or rely solely on a claim ownerReference.
7. Validate each owned allocation’s driver, pool, and device against the node’s discovery snapshot; reject unsupported admin/shared results. Device names alone are not global identities.

This resolver belongs entirely in the driver. The reviewed Multus DRA client has a separate status-list assumption; this implementation bypasses it. Add driver tests for direct claims with an empty template-status list. K5 M3

6.2 Stable binding identity

Define an attachment identity from this canonical tuple:

protocolVersion, nodeUID, podUID, claimUID, driver, pool, device

Use a typed deterministic serialization with named fields and SHA-256. Keep the full digest as attachmentID; generate the runtime NAD name as dra-sriov- plus the first 32 lowercase hexadecimal characters. Verify the full identity on lookup to detect an unlikely truncated-name collision. Do not include source resourceVersion, timestamps, map order, or an RPC batch index in the identity.

Request names are associated metadata, not a reason to attach the same physical device twice. Deduplicate one allocated device associated with multiple requests only after verifying that all applicable template/config/interface requirements agree. Conflicting requirements are an error. Distinct claim allocations resolving to the same PCI are an error, not aliases to deduplicate. The initial managed mode rejects share IDs. A future sharing design must version the identity and network ownership semantics explicitly.

Create shadows in the Pod namespace. Store managed-by, Pod UID, claim UID, and node-UID labels, plus a versioned provenance annotation. Construct metadata from scratch. Do not copy source labels, finalizers, managedFields, ownerReferences, UID, resourceVersion, generation, or the operator’s owner marker.

Give every generated NAD a same-namespace ownerReference to its consuming Pod and the following finalizer:

sriovnetwork.k8snetworkplumbingwg.io/attachment-protection

Set both in the initial NAD Create, together with binding provenance. There must be no interval in which the generated NAD is published without its protection finalizer. Copy neither field from the source template.

Example generated metadata, with illustrative runtime identities:

metadata:
  name: dra-sriov-0123456789abcdef0123456789abcdef
  namespace: ai-workloads
  ownerReferences:
    - apiVersion: v1
      kind: Pod
      name: gpu-sriov-pairs-demo
      uid: 11111111-2222-3333-4444-555555555555
      blockOwnerDeletion: false
  finalizers:
    - sriovnetwork.k8snetworkplumbingwg.io/attachment-protection

The Pod UID must be the actual verified consumer UID, and the NAD must be in the Pod namespace. Omit controller; a plain owner reference is sufficient for garbage collection. Set blockOwnerDeletion: false explicitly. The NAD’s protection must not make foreground Pod deletion depend on the NAD disappearing first. Do not add a Pod finalizer. K8 K9 K10

The ownerReference makes the NAD eligible for cascading deletion when the Pod is removed. The NAD finalizer retains the object while the driver finishes authorized cleanup. The finalizer is removed as part of Pod resource release, rather than immediately on observing Pod deletion or NotFound. NodeUnprepare remains the normal authority to release the VF; the Pod may still be visible at that point. This combines Kubernetes ownership with the driver’s existing checkpoint and teardown lifecycle. K8 K9

Use the workload Pod as owner, not the source NAD, ResourceClaim, or operator. Deleting/editing a source template must not delete an active binding. The driver also explicitly requests shadow deletion during unprepare so cleanup works for completed-but-retained Pods and orphan deletion propagation.

A finalizer key is declarative metadata; the driver must implement the controller code that removes it. If the driver is absent or cleanup fails, the NAD can remain Terminating. That preserves the original configuration and makes unfinished cleanup visible; it does not itself execute CNI DEL or prove the VF was released. K8

6.3 Extend the existing checkpoint

Keep the existing PodManager checkpoint and its atomic write/checksum behavior. Add an optional, versioned managed-binding record to each PreparedDevice. Do not introduce a second database, a global write-ahead log, or a redesign of the existing hardware preparation transaction for this feature. Existing PreparedDevice fields already hold claim/device identity, PCI address, original driver, effective VfConfig, interface name, and NetAttachDefConfig. Reuse those fields.

The added record needs:

The surrounding claim record and PreparedDevice supply the remaining allocation fields. Do not duplicate the entire CNI JSON into several metadata annotations. Do not store it in ResourceClaim status for this path.

The sequence is validation and source rendering, existing hardware/CDI preparation, then PodManager.Set with Pending bindings, before creating a shadow or patching a Pod. A crash before that checkpoint retains the repository’s existing hardware recovery limitations; addressing those is a separate host-lifecycle improvement. This feature must not claim to have made preexisting hardware operations transactional. It does ensure that every newly introduced externally visible binding can be recovered from the existing checkpoint.

Add narrow snapshot and mutation methods, such as SnapshotClaim, SnapshotPod, UpdateManagedBindings, MarkManagedDeleting, and MarkManagedReleaseComplete. Each mutation runs under the PodManager lock, writes the checkpoint, and rolls back its in-memory change on write failure, as existing Set/DeleteClaim do. Never mutate a returned PreparedDevice pointer in place. Deep-copy maps/configuration for the policy controller, prepare handlers, and recovery worker.

Use a per-claim operation lock to serialize reuse of one claim UID, a per-Pod operation lock for managed prepare/repair/release, and an RWMutex for discovery snapshots. Acquire the claim lock before the Pod lock; never acquire them in reverse order. An Ensure helper already under the Pod lock must not recursively acquire it or acquire another claim lock. Never hold the global checkpoint/device-map lock across Kubernetes API or hardware operations.

Reject a new Pod preparing a claim UID while any old binding for that UID remains, including ReleaseComplete; return ClaimReleaseInProgress until the old record is removed. Enforce claim-UID uniqueness atomically in PodManager.Set as well. This prevents its existing first-match claim lookup/deletion from selecting an old record. ReleaseComplete means local cleanup finished and only API/finalizer completion remains; exclude those records from active CDI consumer counts. Exercise lock order and reuse with race tests.

An additive optional field with its own binding version is sufficient if the existing checksum serialization remains compatible; test a real pre-change checkpoint fixture. If a top-level version change is required, implement explicit migration. Unknown managed-binding versions and corrupt checkpoints must produce actionable failures without deleting state. A binary that does not understand managed bindings is not a supported downgrade while they exist: drain and clean first.

The deployment must maintain one active driver instance per node/driver identity against its existing persistent state directory. Keep that property in DaemonSet updates and readiness checks; do not add a new distributed locking service.

6.4 Internal ownership metadata only

The shadow itself is the prepared CNI configuration. Do not add a required ResourceClaim status format or make Multus inspect device status.

Use one versioned, driver-generated NAD annotation, sriovnetwork.k8snetworkplumbingwg.io/binding, with compact JSON containing attachmentID, Pod UID, claim UID, node name/UID, source NAD namespace/name/UID, and rendered configuration hash. This data is generated by the driver; the user does not supply it or match it to another resourceName.

The recommended lightweight Multus guard compares bound Pod UID with the Pod it already fetched, compares the recorded node with pod.spec.nodeName, requires the generated NAD namespace to match the Pod namespace, and verifies the config hash. Multus keeps its existing namespace checks on the actual selected NAD. Source namespace in the provenance is diagnostic; add no separate source-template namespace restriction to this guard. It needs no claim, ResourceSlice, device-plugin, or DRA client. It prevents accidental use of a generated fixed-PCI NAD by a different Pod and detects changed configuration.

Treat this as an ownership guard within the existing trusted-NAD-administrator model, not cryptographic proof of allocation. Restrict writes to generated NADs and the driver-managed attributes accordingly. A user allowed to create arbitrary privileged CNI configurations can already specify a static PCI address; metadata alone does not revoke that existing authority. Do not add a new authorization protocol to this feature.

The driver’s ordinary status.devices reporting can continue for diagnostics. Failure to update diagnostic status must not create a new Multus dependency. If adding useful source/shadow information there, preserve the existing UID/conflict-safe merge rules and API size limit, but keep it optional to connectivity. D8 K4

Use these exact JSON field names for the version-1 cross-repository contract. Values in angle brackets are explanatory placeholders; fixtures must substitute valid UUIDs, full 64-character hexadecimal hashes, names and PCI values.

{
  "version": 1,
  "attachmentID": "<full-identity-sha256>",
  "podUID": "<pod-uid>",
  "claimUID": "<claim-uid>",
  "nodeName": "worker-a",
  "nodeUID": "<node-uid>",
  "source": {
    "namespace": "ai-workloads",
    "name": "rail-a-net",
    "uid": "<source-nad-uid>",
    "resourceVersion": "<source-version>"
  },
  "pciAddress": "0000:03:00.1",
  "interface": "dra0",
  "configSHA256": "<exact-rendered-config-sha256>"
}

The Pod registry uses this shape; its entry order is the same deterministic managed selection order:

{
  "version": 1,
  "podUID": "<pod-uid>",
  "entries": [
    {
      "attachmentID": "<full-identity-sha256>",
      "claimUID": "<claim-uid>",
      "namespace": "ai-workloads",
      "name": "dra-sriov-<32-hex-digest>",
      "uid": "<shadow-nad-uid>",
      "interface": "dra0",
      "configSHA256": "<exact-rendered-config-sha256>"
    }
  ]
}

Compare hashes and IDs exactly; validate type/length/format and reject duplicate attachmentIDs/selection identities. The shadow’s own UID is recorded in the Pod registry/checkpoint, not fabricated before Create. Node UID is driver recovery provenance; Multus compares the node name available from the already fetched Pod and does not fetch Nodes just to validate the UID.

Use labels under the driver domain for managed-by, pod-uid, claim-uid and node-uid. Keep the full 64-character attachment hash in annotations/checkpoint, because it exceeds a Kubernetes label value’s 63-character limit. Do not put arbitrary-length node names or source NAD names into label values without proper validation.

6.5 Deterministic interfaces

The default names are dra0, dra1, and so on. Derive order from all allocated devices for this driver across the Pod’s resolved claims, sorting by canonical claim alias, request/subrequest name, pool, and device. Use the lexically first alias for a claim referenced more than once. Deduplicate physical identities before numbering. Read allocations for this calculation; do not wait for all other claims to be prepared, which could introduce an RPC-order dependency.

Persist the resolved names in the first plan. They must not change on retries, process restart, map iteration, or callback batching. Explicit names are validated as Linux interface names, at most 15 bytes, unique, and not the primary interface. A repeated explicit name for multiple VFs fails.

Preserve unrelated manual network order. Append managed selections in deterministic order. Multus performs a final collision check over the actual full delegate list, including configured additional networks and default-generated manual interface names, before executing any delegate. An explicit collision is an error; do not silently rename the intended interface.

7. Safe template transformation

Implement a pure, unit-testable RenderShadowConfig(sourceNAD, allocatedPCI) helper. Do not reuse AddDeviceIDToNetConf unchanged: it writes only at the root and panics on JSON null. D6

Algorithm:

1. Require nonempty inline spec.config. File-backed/empty NAD configuration is unsupported in this mode.
2. Decode using json.RawMessage maps/lists, rejecting null, arrays, scalar roots, duplicate critical object keys, missing/wrong-type fields, and malformed JSON. Do not convert arbitrary integers through float64. RawMessage map decoding alone does not detect duplicate keys; use a token-validation pass for critical fields. Do not use DisallowUnknownFields to reject legitimate plugin extensions.
3. Materialize the source’s effective CNI name before copying: keep a nonempty valid name; otherwise use the source NAD’s metadata.name, as the NAD-client helper would. The source namespace does not automatically become part of the existing CNI name; administrators should choose distinct explicit names where networks are independent.
4. For a single-plugin configuration, require type: sriov and no plugins key. Reject any ambiguous root containing both type and plugins; Multus chooses its single-plugin path from a nonempty root type.
5. For a conflist, require no root type key and a nonempty plugins array of objects and exactly one device-consuming sriov entry. For the first implementation require it to be the first entry, followed only by a documented allowlist of compatible metaplugins, initially tuning. Extend the allowlist with plugin-specific tests. Do not assume arbitrary plugins are harmless just because their type is not sriov.
6. Reject nested Multus, two SR-IOV entries, host-device/macvlan/bridge/other device-creating chains, or an ambiguous multi-input bond graph. Those require another binding model.
7. Validate and normalize the plain allocated PCI BDF against trusted discovery, e.g. 0000:03:00.1, without a pci: prefix. At the selected SR-IOV plugin only, reject a preexisting nonempty deviceID or pciBusID that differs from the allocation. Set deviceID to that PCI. If retaining/setting pciBusID for compatibility, set the same value only at that selected plugin. Do not broadcast the value into every metaplugin.
8. Preserve CNI version, list name, IPAM config, ipam.network_name, VLAN, trust, spoof checking, routes, capabilities, tuning fields, and other supported values without lossy decoding. Whitespace/key ordering may change during the first render; the resulting bytes are then frozen. Validate the CNI version against the supported libcni/plugin combination and fail on a configuration that cannot be safely transformed.
9. Serialize the effective result, compute its hash, and retain the exact bytes. The shadow’s Kubernetes name must never replace the CNI JSON name.
10. Construct a fresh NAD with only explicitly owned metadata. Delete the k8s.v1.cni.cncf.io/resourceName annotation key entirely, along with any allocation-injector metadata; do not copy the source annotations wholesale. An empty value is insufficient: Multus tests whether the key exists, and an existing empty key still enters the lookup branch.

Example:

{
  "cniVersion": "1.0.0",
  "name": "ai-workloads-rail-a",
  "plugins": [
    {
      "type": "sriov",
      "deviceID": "0000:03:00.1",
      "ipam": {"type": "whereabouts", "range": "192.0.2.0/24"}
    },
    {"type": "tuning", "mtu": 9000}
  ]
}

A scalar ID does not assign different VFs to two SR-IOV plugins. Current Multus’s generic conflist injector writes the same ID to every entry; do not use it for a multi-VF chain. M4

7.1 IPAM correctness: preserve configuration and delegate allocation

No host-local or Whereabouts allocator implementation is needed in the DRA driver. Multus invokes sriov-cni with the rendered configuration. sriov-cni already invokes the configured IPAM plugin for ADD and DEL, including rollback after applicable ADD failures. The existing plugins allocate and release addresses. DRA creates the binding/shadow and preserves the configuration; it must not call IPAM executables, edit host-local allocation files, or create/update Whereabouts lease/IPPools itself. C2

The required DRA work is generic CNI rendering:

1. Determine the source’s effective root CNI name: preserve its nonempty value, or use the original source NAD’s metadata.name when it is missing/empty. Materialize this before creating the shadow. This reproduces the NAD-client helper’s existing default using the source identity. N1
2. Copy the supported CNI configuration without losing unknown fields or numeric precision, then inject the allocated PCI deviceID into its one sriov plugin entry.
3. Preserve the complete ipam subtree, including range, routes, storage settings and whether optional fields are omitted or empty. Do not derive IPAM fields from a Pod, VF or shadow name.
4. Freeze these rendered bytes in the binding checkpoint. Multus retains the effective delegate configuration for DEL, so cleanup uses the same logical network even after the source changes. External IPAM configuration files and credentials must also remain compatible until allocations are released. M5 C2

For example, a source NAD ai-workloads/rail-a-net whose JSON omits name produces a shadow with a generated Kubernetes object name, but the CNI JSON contains "name": "rail-a-net". A second Pod gets another shadow object with that same CNI name. If the source already specifies "name": "ai-workloads-rail-a", preserve that exact value instead. Do not prepend the namespace automatically: that would change the existing logical identity.

host-local. Its allocation directory is dataDir/<CNI network name>. A unique CNI name per shadow would create separate local allocation stores over the same CIDR; two Pods could then receive the same address on the same node. Keeping the source’s effective name avoids that introduced bug. This is why copying an unnamed source verbatim is insufficient: Multus’s NAD helper would otherwise fill in the new shadow metadata.name. There is no need for a host-local branch in the renderer. I1 N1

host-local remains node-local. For one subnet shared across nodes, use a coordinated allocator such as regular Whereabouts, or administrator-assigned disjoint per-node ranges. Preserving the CNI name does not change host-local’s coordination model. I1

Regular Whereabouts. The Kubernetes backend identifies a pool by its storage namespace and normalized CIDR, optionally prefixed by ipam.network_name. If the effective configuration leaves network_name empty, allocation uses the unnamed domain. Omitting it does not automatically default it to the root CNI name; external Whereabouts defaults can still supply a value. Preserve both an explicit value and its omission. Creating a per-shadow network_name can split both pools and overlap-reservation domains, allowing duplicate addresses on the same physical network. Conversely, the same range and same effective network_name (including empty) share the ordinary pool in the same storage namespace even if root CNI names differ. I2 I4 I5

In the reviewed ordinary Whereabouts path, storage namespace comes from the IPAM kubeconfig’s current context, defaulting to kube-system when that context has no namespace. It is not automatically the Pod or NAD namespace. Moving the shadow into the Pod namespace therefore does not itself require rewriting storage namespace or network_name. Preserve supplied settings and keep external kubeconfig/defaults stable through ADD/DEL; Whereabouts also merges external configuration. I4 I5

Scope and tests. Preserve configuration generically for both plugins; use plugin-specific fixtures and real allocation/DEL tests to verify the result. Whereabouts fast node-slice mode remains outside the initial support matrix because its NAD-watching controller and pool ownership add a separate lifecycle interaction. Reject a nonempty/enabled inline node_slice_size in this initial mode and have deployment preflight check external defaults that could enable it. This compatibility guard does not require implementing either allocator in DRA. I3 I5

8. Prepare orchestration and recovery

8.1 Forward sequence

Keep orchestration separate from pure resolution/transformation helpers. For each verified Pod group, and each claim owned by this driver:

1. Validate and render: resolve complete allocation identities, authorized templates, supported configs, deterministic interfaces, annotation size, and manual-selection conflicts. Fetch and render all templates needed for this claim before hardware changes.
2. Prepare and checkpoint: use existing device/CDI preparation. An empty VfConfig.driver preserves the current binding; literal default requests existing default-driver binding. Verify actual supported kernel binding/netdevice presence, rejecting an already-VFIO or unbound VF even when the driver field was omitted. Add Pending managed bindings to its PreparedDevices and persist through PodManager.Set before any shadow creation or Pod patch. Keep existing rollback behavior for failures before that checkpoint. D11
3. Ensure shadows: create each deterministic runtime NAD. On a lost create response, GET that name and verify the complete expected identity and content. Persist its observed UID through a narrow checkpoint mutation. An unrelated or altered object is an error; never overwrite an arbitrary NAD.
4. Ensure Pod selections: optimistic-concurrency merge the intended managed entries and owned-entry registry into the live Pod. Verify Pod UID, node, and valid reservation on each retry. Include every successfully checkpointed claim for this Pod, and preserve other writers’ entries.
5. Ready: mark these bindings Ready durably and return devices. Success requires the checkpoint, shadows, and Pod patch. Diagnostic claim status remains best-effort and is not a connectivity barrier.

After the Pending checkpoint, failures retain prepared state and retry metadata forward. Do not route a failed shadow create, Pod patch, or final Ready checkpoint through a deferred hardware rollback that destroys a binding another consumer may have observed. A lost API response may still mean the write succeeded. Explicit unprepare is the cleanup path.

An error on a new claim must not tear down another prepared claim. Preserve the existing per-claim result contract. Group Pod-global CDI work by verified Pod UID instead of assuming the first claim’s consumer owns the whole RPC batch.

8.2 Pseudocode

This is orchestration pseudocode. The names beginning with Ensure and Snapshot are proposed new methods, not claims about existing symbols.

func prepareManagedClaim(ctx context.Context, claim *ResourceClaim) PrepareResult {
    pod, err := resolver.VerifyConsumer(ctx, claim)
    if err != nil { return failed(err) }
    unlockClaim := claimLocks.Lock(claim.UID)
    defer unlockClaim()
    unlock := podLocks.Lock(pod.UID)
    defer unlock()
    if podManager.HasBindingForOtherPod(claim.UID, pod.UID) {
        return failed(ErrClaimReleaseInProgress)
    }

    devices, found, err := podManager.SnapshotClaim(pod.UID, claim.UID)
    if err != nil { return failed(err) }
    if !found {
        plan, err := planner.ResolveAndRender(ctx, pod, claim)
        if err != nil { return failed(err) }
        devices, err = deviceState.PrepareWithManagedPlan(ctx, claim, plan)
        if err != nil { return failed(err) }
        // Existing host/CDI flow; persist Pending before API side effects.
        if err := podManager.Set(pod.UID, claim.UID, devices); err != nil {
            // No shadow/Pod write has occurred. Clean only this new claim.
            cleanupErr := rollbackNewClaimDevices(claim.UID, devices)
            return failed(joinErrors(err, cleanupErr))
        }
    }
    if hasDeletingOrReleaseCompleteBinding(devices) { return failed(ErrReleaseInProgress) }
    if err := EnsureManagedBindings(ctx, pod.Identity()); err != nil {
        return failed(err) // retain checkpoint and prepared hardware
    }
    // Ensure persists Ready after the Pod handoff for completed bindings.
    return prepared(devices)
}

Integrate this with the actual signatures in dra_hook.go and devicestate; do not mechanically paste it or add a second Set that conflicts with the existing one. The critical insertion point for external API work is after the existing successful Set. The claim and Pod operation locks are acquired once at this boundary, in that order; Ensure/Release internals must not recursively acquire either lock. Worker entry points acquire them in the same order before calling the shared locked helper.

EnsureManagedBindings takes a fresh checkpoint snapshot of Pending/Ready claims for this Pod, ensures every referenced shadow and UID, and makes one complete Pod registry/selection merge. It must not publish references to another claim’s not-yet-created shadows. The already-prepared path must run it before returning. Keep the original immediate return only for ordinary nonmanaged records. On an enabled node, an old checkpoint without a managed binding is a migration condition: require a drained activation, not a guessed template for a possibly running VF.

8.3 Independent repair and finalizer reconciliation

Start a small node-local worker after checkpoint load and before managed readiness. At startup and on a bounded periodic interval:

• Pending/Ready: verify identities and repair healthy bindings.
• Deleting: resume authorized local cleanup and then finalization.
• ReleaseComplete: finish API/finalizer cleanup only; never restore/rebind hardware or repeat local CDI cleanup.

The two release phases record an earlier NodeUnprepare authorization. Pod/NAD deletion events, Pod NotFound and a missing Node Lease do not create that authorization. Watch events can accelerate reconciliation; periodic reads of known bindings cover missed events and a driver-only restart.

For a matching live Pod and still-valid allocation, ensure missing healthy shadows and owned Pod entries from frozen bytes. Do not reread edited templates/policies to alter a prepared binding. When a shadow is truly NotFound, first persist Pending with its recorded shadow UID cleared. Then create or recover the deterministic name with ownerReference/finalizer, verify full identity/content, persist the observed UID, and update the Pod registry before Ready. A release with an empty recorded UID may recover it only from an exact identity/hash match and then use that UID for API mutations. A nonempty recorded UID differing from an existing object’s UID remains a conflict.

A shadow with deletionTimestamp is not missing and must not be recreated. Retain its finalizer and frozen configuration, stop forward repair, emit ShadowDeletionPending and await authorized release. A DELETE of a shadow while the Pod is live must not mark the driver binding Deleting or release its VF. Deletion cannot be cancelled in place; the supported recovery is normal workload termination/recreation. Do not try to clear deletionTimestamp or add a finalizer after deletion has started. K8 K14

For an exact owned shadow missing its protection finalizer while not deleting, restore the key using a UID/resourceVersion-checked metadata patch before returning managed readiness. If deletion starts during that patch, retain the checkpoint, fail preparation with ShadowDeletionPending, and never assume protection was restored. A deleting object that already lost the key requires visible intervention; the driver cannot re-add it.

An existing shadow with wrong ownership/content is an explicit failure. Do not recommend force-removing its finalizer while the VF is active to trigger recreation. Inspect the conflict and use normal workload release or the fenced-node procedure.

Stop forward repair for a terminating/deleted Pod, changed reservation/allocation, claim UID replacement, node UID mismatch, Deleting, or ReleaseComplete. These observations alone do not authorize hardware release. A Pod deletion notification enqueues the binding; normal kubelet unprepare performs the release transition.

Limit of the deliberately simple handoff: if someone removes every driver-owned Pod annotation/registry and CNI ADD races the repair worker, Multus has no allocation lookup from which to infer that a network is missing. Preserve driver-owned metadata until release. The registry detects partial deletion and the worker repairs accidental removal, but neither proves that all metadata once existed. An optional admission policy protecting those keys provides stronger enforcement without adding DRA lookup to Multus.

8.4 Unprepare and finalizer removal

Find the checkpoint by exact claim UID and verify its recorded Pod identity. Under the claim-then-Pod locks, transition Pending/Ready to Deleting and persist it before cleanup. Keep an existing Deleting phase as-is. Never downgrade ReleaseComplete to Deleting: resume its API-only finalization path below. Deleting means release was authorized, not merely that Kubernetes marked the NAD for deletion.

Perform this sequence; every API operation is idempotent and retains recoverable state on errors:

1. Resolve each exact owned shadow UID from the checkpoint and verify its binding provenance/config hash. For an empty UID left by a lost Create response, GET the deterministic name and verify namespace, complete binding identity, managed metadata and frozen config before adopting and persisting its UID. During release, an absent Pod ownerReference is allowed: orphan deletion propagation can legitimately remove it. Do not require the owner Pod or its reference to still exist. A present Pod ownerReference must match the recorded Pod UID; a conflicting owner, replacement UID or content mismatch is an error, not permission to edit another object. This release exception does not weaken the owner-reference requirement for healthy prepare/ADD.
2. Request shadow deletion using its UID precondition. If GC already set deletionTimestamp, continue. If it is NotFound, continue with local cleanup anyway. Do not wait for the NAD to disappear while this driver’s finalizer is present.
3. Remove only this claim’s exact owned Pod selections/registry entries from the same Pod UID, when that Pod exists.
4. Use existing device-state unprepare to restore the original host driver and delete claim CDI/device-info artifacts. The driver does not execute CNI; the normal kubelet/runtime teardown path is the supported authority.
5. Preserve/rebuild Pod-global CDI while any other locally unreleased claim needs it; delete it after the last such claim. ReleaseComplete records retained for API work are no longer active CDI consumers.
6. Persist ReleaseComplete for the claim’s managed bindings. If persistence fails, retain Deleting and replay idempotent local cleanup on retry. Do not remove protection before this checkpoint succeeds.
7. With ReleaseComplete durable, ensure deletion was requested for the exact shadow and remove only sriovnetwork.k8snetworkplumbingwg.io/attachment-protection. Preserve every unrelated finalizer. Perform this through a UID/resourceVersion-checked JSON Patch on the NAD’s main resource endpoint.
8. A NotFound response, or a live matching-UID object with deletionTimestamp set and this driver’s finalizer absent, completes the driver’s finalizer obligation. A lost patch response is resolved by re-reading. Do not wait indefinitely for unrelated finalizers to disappear or repeat hardware cleanup because another controller still holds the NAD.
9. Remove the checkpoint claim only after its owned finalizer obligations and local cleanup are complete. Return unprepare success after that durable deletion. A new Pod using the same claim UID must wait until this point.

A ReleaseComplete retry performs only steps 1–3 as still needed for API consistency and steps 7–9; it never repeats steps 4–5. State retained solely for finalizer completion cannot be used by the prepare fast path or included in new Pod network selections.

On finalizer JSON Patch test failure, re-read the live object. A changed UID is a replacement conflict and must not be touched. With the same UID and a changed resourceVersion, rebuild the patch from the latest finalizer list and retry within a bounded backoff. Kubernetes can return HTTP 422 for a failed JSON Patch test, so RetryOnConflict alone is insufficient. Do not retry a genuinely invalid patch endlessly. K12

A missing Pod/claim makes its API cleanup a no-op, not the entire cleanup a no-op. Do not wait for Pod NotFound or phase Succeeded/Failed before unprepare/finalizer removal. In the reviewed normal path, kubelet KillPod calls StopPodSandbox, then kubelet unprepares resources before writing terminal Pod status. Waiting for final Pod deletion can prevent that lifecycle from completing. Qualify both intended runtimes with teardown tests. K6 K13

Live reads and finalizers do not provide a lock against arbitrary external concurrent ADD and release. This implementation supports the tested kubelet/runtime lifecycle. Force-deleted Pods, lost nodes and missing runtime evidence require conservative recovery rather than a Pod-404 shortcut.

8.5 Owner deletion, orphans, and edge cases

A finalizer is not a timeout lease. Do not remove it automatically after a fixed age, Pod NotFound, stale Node Lease or node-name reuse. Fencing must prevent the old node/runtime from continuing to use the VF/network; any required IPAM recovery remains part of the administrative procedure.

At startup and periodically, list NADs across namespaces with selectors for the driver-managed label and current node UID, then correlate full binding identity with the checkpoint. The default installation grants NAD list cluster-wide; no Namespace list or workload namespace configuration is needed. Listing only namespaces already present in checkpoints would miss orphans whose checkpoint was lost. Labels narrow discovery, while verified identities determine ownership. Watch is an optional accelerator. Never act on another node UID. A protected object without a matching checkpoint is reported as an orphan, not treated as safe to finalize.

Normal cleanup, when only this driver’s finalizer exists, leaves no shadow. Uninstall/downgrade instructions must drain and release managed workloads, resolve owned finalizers, and then remove the driver and its credentials. Deleting the finalizer controller first leaves objects stuck Terminating.

9. Pod annotation merge and ownership

The user-facing annotation remains the standard k8s.v1.cni.cncf.io/networks. Do not ask a workload author to add a second annotation. The driver maintains a compact internal registry under proposed key sriovnetwork.k8snetworkplumbingwg.io/managed-networks.

Use JSON version 1 for the registry, with entries keyed by full attachmentID. Each entry contains shadow namespace/name/UID, interface, claim UID, and rendered-config SHA-256. Use explicit field names and a shared set of contract fixtures in both repositories. Do not introduce a runtime dependency from Multus to the driver repository. A registry is a record of entries this driver owns, not a complete inventory of every DRA allocation in the Pod.

A generated selection uses only standard Multus fields:

{"name":"dra-sriov-<32-hex-digest>","namespace":"ai-workloads","interface":"dra0"}

The placeholder above is explanatory, not a literal valid generated name. Do not put the PCI address into the Pod selection’s deviceID field as a substitute for embedding it in the NAD. The reviewed sriov-cni requires the plugin configuration’s deviceID; Multus’s runtime-capability path is a different mechanism. M4 C1

Implement the merge as a pure helper with these rules:

1. Treat an absent networks annotation as an empty list. Parse valid JSON arrays and existing comma notation, including namespace and interface shorthand. Match current Multus namespace defaults. Reject malformed input rather than replacing it.
2. Preserve unrelated JSON objects, their unknown fields, and their relative order. Using only the typed NetworkSelectionElement and re-marshalling it can lose fields unknown to this version; keep raw object maps alongside validated known fields.
3. Preserve valid manual ips, mac, routes, bandwidth and other options on unrelated selections. Managed entries are generated exclusively by the driver; unexpected edits to their name/interface/options produce an ownership conflict, not an attempt to reuse arbitrary tenant parameters.
4. Recognize existing owned entries using the registry plus checkpoint identity and exact selection tuple. Prefix matching alone does not establish ownership. Refuse a generated-name collision without matching provenance.
5. Append missing managed entries once in deterministic order. Multiple VFs using the same source NAD still need separate shadows and unique interface names. Do not deduplicate by source NAD name.
6. A user-written selection of the same source NAD as an automatic binding is ambiguous and should fail with ManualSelectionConflictsWithManagedTemplate, naming that entry and explaining the migration. Do not silently create a second attachment or rewrite a manual request into a different interface.
7. Drop only this claim’s exact owned selections during release. Preserve manual entries and other claims’ entries. If ownership data is malformed or has an unknown version, fail with an actionable error rather than guessing what to delete.
8. Validate the total annotation size using Kubernetes’s metadata validation limits, including all existing annotations. Do not publish a partial list when a very large VF count exceeds that limit.
9. Return the existing value unchanged when semantically identical; repeated repair should not continuously rewrite a running Pod.

Use JSON Patch with test operations for the Pod UID and resourceVersion, followed by the two annotation changes in the same patch, or an equivalently proven optimistic-concurrency mechanism. Escape JSON Pointer keys correctly. If annotations is nil, create the map under the same version precondition. On conflict/test failure, fetch the live Pod, verify identity and reservation, and recompute from its current annotations. Do not reuse the old complete annotations map.

A bounded conflict retry returns an error to kubelet/worker; it does not replace another writer’s data. Use request contexts and backoff. Admission rejecting the mutation is an actionable deployment/configuration failure.

When the registry exists, Multus can require that every recorded managed selection appears exactly once and resolves to the recorded shadow UID/hash. This is a metadata consistency check on objects it already reads. If every driver-owned annotation is removed, the limitation in section 8.3 applies.

10. Minimal Multus implementation

10.1 Preserve the existing static-device path

In getKubernetesDelegate, the resource-client and DRA-client lookups are inside the resourceName-annotation branch. With that key absent, LoadDelegateNetConf receives no lookup-derived deviceID, and the static deviceID already in the raw NAD JSON remains intact. Do not add new driver-specific allocation code to this function. M2

Add a regression test in which the resource-client factory fails if called and fake ResourceClaim/ResourceSlice API reactors fail if accessed. A valid managed shadow must still execute. Scope the assertion to this path: another explicitly configured default/manual NAD with resourceName may legitimately use the old lookup code.

10.2 Live reads for ADD

Add LiveNetworkConfigOnAdd bool with JSON key liveNetworkConfigOnAdd,omitempty to pkg/types/types.go:NetConf. After GetK8sClient in CmdAdd, before GetPod, derive a request-local client view:

func (c *ClientInfo) WithLiveReads() *ClientInfo {
    if c == nil {
        return nil
    }
    copy := *c
    copy.PodInformer = nil
    copy.NetDefInformer = nil
    return &copy
}

The inspected ClientInfo contains shared client interfaces/event helpers and the two informer interfaces; it has no mutex or per-request mutable resource map that makes this copy unsafe. Recheck that assumption on rebase. Existing Pod/NAD methods use live API GETs when informers are nil. GetK8sClient preserves an already supplied client, so delegate loading retains this request-local view. Never mutate the daemon’s shared ClientInfo. M1 M8

Requirements:

• Use ordinary live GET semantics; do not request resourceVersion: "0" and then assume a freshness barrier.
• Fail ADD on a failed required Pod/NAD read. Do not fall back to a stale successful cache entry after the live path fails.
• Keep UID checks and existing namespace-isolation checks.
• Bound API requests by the existing CNI request context/time budget. If a current NAD helper lacks context, add a context-aware variant for ADD and thread that context.
• In thick mode, cover the earlier missing-K8S_POD_UID fallback: server.go:podUID currently reads the shared cache before CmdAdd. For ADD with this setting enabled, use a bounded live Pod query or consistently pass the request-local live view.
• Make that fallback command-aware. The same runtime-argument helper runs before DEL. DEL with saved delegates, an omitted runtime Pod UID and an already deleted Pod must reach cached cleanup without a mandatory successful Pod GET. Preserve an empty UID where the cached path supports it; do not apply an ADD-only identity requirement to DEL. Test this exact case through the thick server, not only by calling CmdDel directly.
• Keep the saved-config DEL path independent of these live reads.

The daemon loader retains raw configuration, and its overlay applies daemon keys over shim keys. Set the flag in the actual daemon-config.json. Add the field to pkg/server/config/generator.go:MultusConf so generated shim configuration retains it; test round-trip and daemon override behavior. A ControllerNetConf field is not necessary merely to propagate the existing raw overlay. Thin configuration may include the same key; thin mode already reads live when no informers exist. M6 M7

This setting adds API traffic for Pod/NAD reads on enabled ADDs. Measure request counts and startup latency in a scale test; do not optimize away the freshness property before the race test exists.

10.3 Metadata-only checks before delegate execution

Include a small generated-NAD consistency guard for the supported managed deployment. It performs no DRA or allocation lookup. Given the already loaded Pod, registry, NAD and rendered CNI bytes:

• Recognize the versioned driver binding annotation, reject unknown versions, and verify the same Pod UID, node name and intended namespace.
• Verify the selected shadow UID/hash and exactly one matching selection if a registry entry exists.
• Require the selected managed shadow to be in its bound Pod namespace. Preserve ordinary Multus namespace isolation for selected NADs. The driver may read an explicitly configured source NAD in another namespace under its API credentials; do not impose a new source-provenance namespace allowlist/check in Multus.
• Require resourceName-key absence, a valid inline static SR-IOV configuration, and consistent embedded PCI identity.
• Reject duplicate managed selections and duplicate resolved interface names before any delegate ADD. Resolve names with Multus’s actual getIfname behavior, including configured default/additional networks, rather than guessing from only the Pod annotation.
• Reject an unexpected managed selection override such as alternate deviceID, interface or options that disagrees with the registry.
• Leave ordinary NAD behavior unchanged.

Hash the exact spec.config bytes written by the driver and stored in its checkpoint, before Multus adds runtime fields. Specify lowercase hexadecimal SHA-256 and UTF-8 bytes in cross-repository fixtures. Formatting edits of a managed shadow are unsupported and should be detected.

Verify the same-namespace Pod ownerReference UID as well as the binding annotation’s Pod UID. Require the driver’s protection finalizer on healthy managed ADDs. A nonnil NAD deletionTimestamp must reject a new managed ADD before any delegate executes; stock Multus otherwise still reads and loads a finalizer-held NAD. M1 M2 Kubernetes annotations are not inherently immutable. The driver enforces immutable content by verifying existing objects, and deployments can enforce write restrictions/admission. The guard is not a signature or independent proof that a VF was allocated.

Run these ownership/readiness/deletion checks only for ADD. Do not require the Pod or shadow to exist during cached DEL. A DEL fallback that reads the retained original shadow must not reject it solely because it is Terminating. Cleanup must remain possible after Pod deletion and during NAD finalization. Keep the operation context explicit when sharing delegate-loading helpers.

10.4 Device-info is optional

The static CNI path supplies connectivity and ordinary network-status interface/IP/MAC reporting. Multus’s internal DelegateNetConf.DeviceID can remain empty while the raw injected JSON works. Some PCI/RDMA device-info enrichment and auxiliary CNIDeviceInfoFile behavior will then be absent. M4 M5

Do not reintroduce resourceName solely to recover that enrichment. If required for a consumer, add a small optional follow-up: the driver writes discovered PCI/PF/RDMA metadata into its generated descriptor, and Multus uses that already-read descriptor for network-status or its existing runtime-file path. Make file creation/retry/delete idempotent; the NAD-client’s existing save helper rejects an existing file. Use an actually discovered RDMA name, not the boolean rdmaCapable value. This is not required to deliver the three requested UX improvements.

11. CNI teardown, retries, and persistence

Reuse Multus’s existing delegate cache. It saves raw effective delegates before ADD and normally loads them for DEL in reverse order. sriov-cni separately saves the original VF state using container/interface identity. IPAM DEL consumes the delegated stdin configuration, so saving exact rendered bytes is essential. M5 C2

The feature must include tests proving:

• The driver never invokes sriov-cni ADD/DEL in MULTUS mode.
• A delegate/IPAM failure produces the expected cleanup and kubelet retry behavior.
• DEL still uses the original rail/IPAM config after source, shadow or Pod deletion.
• Driver and Multus process restarts preserve the mounted host state directories.
• Sandbox recreation without a new NodePrepare still has the same valid frozen shadows/selections.
• CNI network name, container ID and interface name remain consistent for the applicable retry/DEL invocation.

Do not rebuild a missing old sandbox record from a mutable current source NAD. The existing Multus fallback on a missing cache may read current API configuration; this is an existing limitation, not a complete cache-loss recovery mechanism. Required deployment validation includes persistent Multus and sriov-cni cache mounts and an explicit cache-loss fault test showing the failure/cleanup behavior.

A second redundant delegate journal, tombstones, or an attachment lease between driver and Multus may improve severe node-disk-loss and arbitrary concurrent-invocation recovery. Keep those out of the baseline implementation. A shadow alone cannot reconstruct every original VF/IPAM state after all local CNI caches are lost.

12. Operator integration and permission model

12.1 Implement against the correct foundation

At the inspected master commit, sriov-network-operator does not contain merged DRA provider support. PR #1151 is open and unmerged and contains the intended foundation; the earlier PR #1061 is closed without merge. Recheck the PR state before implementation. Use #1151 or its merged successor as an explicit dependency for the operator-managed track. Do not claim that setting a feature-gate string on existing master installs missing code. O1 O3

For standalone DRA installation, an operator can still be relevant to hardware provisioning, but avoid its legacy device plugin advertising the same VFs. The first supported provider transition uses drained target nodes and no active allocations. The pending integration starts a replacement provider before removing the old DaemonSet and has no complete active-allocation migration. This feature must not attempt live DP-to-DRA allocation transfer. O6

12.2 Node-policy custom attributes and SriovNetwork references

This extension is part of the implementation target. The operator-managed UX is: configure hardware and custom DRA attributes in SriovNetworkNodePolicy, configure the rail’s L2/L3/IPAM settings in SriovNetwork, and let the operator generate DeviceAttributes/SriovResourcePolicy. Users do not hand-edit generated objects or create a second matching resourceName attribute.

Neither the inspected master nor PR #1151 node-policy API has a DRA/custom-attributes field. The pending controller currently creates one DeviceAttributes per resource pool and replaces its Spec on reconciliation. Extend the API and generator rather than asking users to patch those objects. O4 O11

12.2.1 Proposed API and exact input keys

Add optional spec.dra with an attributes map. Use these proposed Go shapes in api/v1/sriovnetworknodepolicy_types.go; add the validation markers and generated schema specified below:

// New optional field on SriovNetworkNodePolicySpec:
DRA *SriovNetworkNodePolicyDRAConfig `json:"dra,omitempty"`

type SriovNetworkNodePolicyDRAConfig struct {
    Attributes map[string]DRAAttributeValue `json:"attributes,omitempty"`
}

// Exactly one scalar member is required; use pointers to preserve zero/false.
// Make each value atomic for server-side apply.
type DRAAttributeValue struct {
    IntValue     *int64  `json:"int,omitempty"`
    BoolValue    *bool   `json:"bool,omitempty"`
    StringValue  *string `json:"string,omitempty"`
    VersionValue *string `json:"version,omitempty"`
}

Use an explicit operator-owned scalar value type, then convert/deep-copy it into resourceapi.DeviceAttribute. Do not silently expose newly added upstream list-value fields through a dependency upgrade. The initial API supports int, bool, string and version only. This is an additive namespaced node-policy API extension; it introduces no new CRD kind.

The two operator reference keys are either both absent or both present in the same policy. They are configuration inputs consumed by the operator, not attributes published to ResourceSlices. Replace them with the complete existing driver-facing netAttachDefName/netAttachDefNamespace pair. Do not introduce alternate nadName spellings or teach the driver to GET SriovNetwork.

An attributes-only policy without a SriovNetwork reference is allowed. In managed MULTUS mode its VF still needs a template through the existing valid VfConfig path. An absent spec.dra keeps the previous operator policy behavior. An explicit empty spec.dra: {} uses the new per-policy generator with only operator-generated base attributes. Store and validate the optional block even when the DRA provider is disabled, but do not apply its attributes to the legacy device-plugin configuration. Report DRAConfigurationReady=False/DRAInactive for such a staged configuration. Do not use a field on the default bootstrap policy to bypass its special handling; reject a nonnil dra block there.

12.2.2 Admission and controller validation

Implement structural schema validation plus semantic webhook/controller validation. Repeat essential checks in reconciliation because a disabled webhook or an older stored object must not produce unsafe output.

• Each value has exactly one of int, bool, string or version. Enforce the union with CRD CEL/structural validation and repeat it in Go. Boolean false and integer zero count as present. Reject unknown/list fields; make value objects atomic under SSA.
• Require fully qualified DRA keys: a DNS subdomain prefix of at most 63 characters, a slash, and a C identifier suffix of at most 32. Kubernetes annotation-key validation alone is insufficient. Strings and semver 2.0.0 values are at most 64 characters; validate version syntax. These are the pinned DRA limits. K4
• Apply maxProperties=32 as an input bound, but do not advertise 32 custom attributes as usable capacity. The final discovered-plus-generated-plus-custom attribute/capacity total must fit 32. Check the actual merged device budget in the driver before publication/preparation; reject rather than dropping user keys. K4
• Validate the paired SriovNetwork keys as strings and Kubernetes object names/namespaces. Do not default the CR namespace to the node policy, Pod or eventual NAD namespace. Names longer than 64 cannot traverse this scalar-attribute path; fail clearly. The existing direct-driver typed VfConfig path remains available for longer NAD names.
• Reject discovery-owned keys from the driver’s ReservedAttributes, including PCI/device identity. Reject custom k8s.cni.cncf.io/resourceName because the operator generates any compatibility value itself. Treat driver netAttachDefName/netAttachDefNamespace as generated output on this operator API: reject manual values rather than combining them with a SriovNetwork reference or silently overwriting them. Direct-driver DeviceAttributes retain their own documented input contract.
• Reserve the two operator reference keys for their defined meaning; an ordinary custom key cannot shadow them. Keep these configuration keys separate from the driver’s discovery ReservedAttributes list.
• Do not require a reference target to exist during API admission: applying a multi-object manifest must not depend on object order. Existence, readiness, API authorization and NAD rendering are reconcile-time checks with conditions/events.

12.2.3 Resolve the network CR into a ready source NAD

For each referenced policy, use a live reader to GET the exact SriovNetwork. Reject/defer a missing, deleting, unauthorized or invalid CR. Validate the existing operator rule: spec.networkNamespace may be set only when the SriovNetwork lives in the operator namespace. O2

Resolve the target by the operator’s existing rendering rule:

sourceNAD.name      = sriovNetwork.metadata.name
sourceNAD.namespace = sriovNetwork.spec.networkNamespace, if nonempty
                      otherwise sriovNetwork.metadata.namespace

The CR namespace and NAD namespace are distinct values. The node-policy controller must use the actual generated target, never assume its own namespace. Reuse RenderNetAttDef or factor a shared resolver with tests so the two controllers cannot drift. O12

Before publishing a usable per-policy attribute object, require a nondeleting source NAD, current SriovNetwork Ready=True with observedGeneration equal to metadata.generation, and a source NAD matching the expected rendered spec and operator ownership marker. Use the existing renderer/ownership helper; do not hardcode its marker format or accept a stale Ready condition from an earlier generation. A status update without a generation change must wake reconciliation. A same-namespace source ownerReference, if present, must match the current SriovNetwork UID. The existing network controller remains the sole writer/owner of the source NAD. The node-policy controller reads it; the DRA driver later reads it and creates its own Pod-owned shadows.

Build a complete desired attribute map from: operator-generated compatibility attributes, validated ordinary custom attributes, and the resolved NAD pair. Consume the operator reference keys instead of exporting them. Do not publish a partial pair, keep an old network value after a failed resolution, or overwrite the source with a PCI address. A NAD read or readiness check can fail between controllers; section 12.2.5 defines the explicit unavailable state.

Existing SriovNetwork still requires spec.resourceName and renders its legacy NAD annotation. Retain that API for this change. Neither its value nor its annotation participates in automatic attachment selection, and the resolver must not require equality with the node-policy resourceName. The driver removes the legacy annotation from each shadow. Removing the legacy-required SriovNetwork field later is a separate backwards-compatible API change, not a prerequisite here. O1 O9

12.2.4 Generate isolated per-policy attributes

The current per-pool object cannot distinguish two rail mappings in the same resource pool. Build one validated resolution plan per reconciliation and share it between attribute generation and node-config generation; do not independently resolve the same policy twice into inconsistent desired state. For every policy with a nonnil dra block, generate one complete DeviceAttributes in the operator/driver policy namespace:

• Deterministic name: dra-policy-<full-policy-UID>-attrs. Policy recreation gets a different object; no dependence on resourceVersion or map order.
• Labels: the existing operator generated-by label plus sriovnetwork.openshift.io/node-policy-uid: <policy-UID>. Do not put the existing sriovnetwork.openshift.io/resource-pool label on these objects. Otherwise an unchanged legacy pool selector would accidentally merge every rail’s custom map.
• Owner: preserve the integration’s existing SriovOperatorConfig controller ownership and ownership/adoption checks. Track the desired set by full object name/policy UID and clean obsolete managed objects explicitly. Do not add a second controller owner or tie these policy attributes to a workload Pod.
• Spec: one complete map containing the generated compatibility resourceName, ordinary custom attributes and resolved NAD pair when applicable. Never split the pair across multiple objects.
• Corresponding generated SriovResourcePolicy config: preserve the policy’s translated hardware filters and set deviceAttributesSelector to the policy-UID label plus the operator generated-by label. Select this complete object directly; do not AND unrelated pool/extra-object selectors and expect a union.

For policies without a dra block, retain the existing pool object and selector, generated only for resource pools still needed by legacy policies. Preserve independently owned DeviceAttributes and existing compatibility behavior there. The new operator-facing path requires no administrator-owned overlay objects. A policy’s extra attributes do not become global pool attributes merely because two policies share spec.resourceName.

Create/update a ready desired per-policy object before publishing the corresponding selector. Withhold/delete an unready object’s stale version while retaining the selector/filter that makes its devices unavailable. Clean obsolete generated attributes after the desired configs no longer reference them. Change the current cleanup code from checking only desired pool labels to comparing the complete desired managed-object name set; it must not delete the new UID-labeled objects every reconcile. Preserve the current do-not-adopt-unmarked-objects safeguards. O4

Shared resourceName is supported for disjoint PF/VF sets with different custom attributes and SriovNetwork references. Keep the existing generated per-resource DeviceClass compatibility path. The topology example uses a broad eligible SR-IOV DeviceClass and pairs each interface request with a GPU request using resource.kubernetes.io/pcieRoot. It does not select netAttachDefName or a rail. Each allocated VF’s per-policy mapping supplies the correct source NAD/IPAM after scheduling. Do not add a per-policy DeviceClass unless independently needed.

Reject conflicting managed mappings for overlapping actual devices on a node. Check translated filters against current node inventory; compare effective custom/reference maps, not just resourceName. Existing hardware-policy priority does not make a supplementary DRA metadata overlay: the driver chooses the first matching config and does not merge configs. Require disjoint filters for the documented rail example. If an unsupported/ambiguous selector cannot be translated, do not emit a match-all config or silently fall through to another network. Retain the existing unsupported-filter restrictions and diagnose the node/policies involved. For a detected overlap or targeting error whose affected VFs cannot be isolated safely, apply an empty configs list to that node’s existing generated SriovResourcePolicy and report ConflictingDRAMapping/UnsupportedManagedSelector; do not leave its old usable config in place after returning an error. A config with empty ResourceFilters is match-all, whereas an empty configs list advertises no devices. The operator-managed installation must prohibit independent overlapping SriovResourcePolicies for this inventory. Keep healthy nodes reconciling and prepared checkpoints intact. D5 O7

12.2.5 Dependency failures and driver eligibility

A missing/unready/deleting SriovNetwork, unavailable target namespace/NAD, wrong owner/content or invalid attribute map produces DRAConfigurationReady=False with the precise reason. Keep reconciling independent healthy policies. Do not return on the first bad reference before invalidating its old generated object and processing other desired entries. API writes can fail; retain/retry those invalidations and never report readiness until they succeed.

For an unresolved reference, withhold/delete only that policy’s managed DeviceAttributes. Keep its exact generated hardware filter and policy-UID selector in SriovResourcePolicy. Do not leave the stale NAD pair, replace it with an empty-but-matching attribute object, or remove the config and allow a broader policy to take its devices.

Make this small generic driver correction in managed mode:

1. Resolve an explicit selector to both the matched-object count and its validated merged attributes. A nil selector permits the existing uniform VfConfig fallback. A nonnil selector with zero matches, invalid semantic references or conflicting pairs is unresolved. A matching object with an empty map is distinct from no matching object and can be valid.
2. When a config first matches a hardware device, record that device in a separate claimed set even if attribute resolution is invalid. Invalid matches go to a blocked set and cannot fall through to later configs. Return only eligible devices to UpdatePolicyDevices. This preserves first-match semantics while making unresolved configuration unavailable.
3. Keep all discovered devices in the underlying allocatable inventory. Update the advertised policy membership and clear withdrawn policy attributes; do not delete host identity needed by prepared checkpoints or teardown. Snapshot eligibility and attributes atomically under the discovery/state lock.
4. Before first managed prepare, reject a device absent from eligible policy membership before applying a VfConfig fallback or changing hardware. A stale scheduler allocation therefore fails visibly instead of reaching another rail. Cached Pending/Ready paths and unprepare use frozen state and bypass this current-eligibility gate.
5. Fix the existing clearPolicyAttributes change-reporting bug: deleting an advertised membership whose old attribute-key set is empty must still trigger ResourceSlice republication. Compare advertised membership before mutation or return a change for that deletion. D2 D5

Scope the unresolved-selector behavior to the automatic mode unless a separate compatibility-reviewed change extends it. Reuse the driver’s existing policy/attribute API; no operator CRD dependency, new Multus lookup or new on-wire readiness attribute is required.

This is unavailable-state enforcement after reconciliation observes the dependency failure. Separate API objects and informer caches do not provide instantaneous revocation or an atomic scheduling-time mapping. Keep physical rail mappings stable while claims are allocated; drain/recreate allocations for topology changes. Prepared bindings continue from frozen source config even if a policy, SriovNetwork or source NAD later changes/disappears. Never let operator cleanup delete active shadows or remove their Pod-release finalizers.

12.2.6 Watches, cache boundaries and status

Add SriovNetwork and source-NAD create/update/delete triggers to node-policy DRA reconciliation, including status-only readiness updates and namespace changes. Filter to referenced sources and relevant content/ownership/deletion changes, inspecting both old and new objects so removal of an owner marker is still observed; a Pod-shadow NAD event must not trigger a whole-inventory rebuild. Keep periodic reconciliation as missed-event recovery.

The existing node-policy reconciler uses a manager cache restricted to the operator namespace, while network/NAD controllers use mgrGlobal. Adding ordinary Watches to the restricted manager will miss workload-namespace NADs. In main.go, pass a live APIReader for reference GETs and the global cache for event sources. Register raw source.Kind watches from that global cache into the existing node-policy work queue, or an equivalent tested cross-manager event bridge. Do not silently expand all policy caches or reuse the startup-only channel that is closed immediately. Recheck the concrete controller-runtime source.Kind signature at the implementation dependency version. O13 O15

For this extension, embed the existing ConditionStatus in SriovNetworkNodePolicyStatus, add its GetConditions accessor, and wire the existing status patcher. Use condition type DRAConfigurationReady with observedGeneration. True means its custom attributes/reference have been validated and the intended generated objects/configs have been applied; it is not proof of a Pod’s connectivity. False/Unknown reasons distinguish DRAInactive, InvalidDRAAttribute, SriovNetworkNotFound, SriovNetworkNotReady, SourceNADNotReady, SourceAccessDenied, ConflictingDRAMapping and DRAConfigurationSyncFailed. Avoid status-only update loops and never set a network condition through the wrong controller. O11 O17 O18

The operator reads SriovNetwork and source NADs using its existing network-management privileges; verify the effective global read/watch grants and condition-status patch permission in both installation paths. The DRA driver’s new permissions remain Pod/NAD permissions only. It receives no SriovNetwork read permission.

12.3 Minimal new RBAC

Retain existing DRA allocation/publication permissions. The default installation supports workload Pods and configured source NADs across namespaces without a driver namespace list:

Use a ClusterRole/ClusterRoleBinding for these default Pod/NAD grants. Reads/writes still target concrete object namespaces. Orphan inspection uses an all-namespace NAD list filtered by managed/node-UID labels; it does not enumerate Namespace objects or infer scope from RoleBindings. A namespace RoleBinding alone is insufficient for that list. The existing integration’s operator-namespace Pod role and NAD-get-only rule must be updated in both chart and static manifests. O5 O8

Example additional grant for a standalone ServiceAccount named dra-driver-sriov in namespace dra-driver-sriov follows. Substitute the actual rendered ServiceAccount for an operator deployment. This is the feature’s additional Pod/NAD permission set, not a replacement for the existing DRA permissions.

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: dra-sriov-managed-networks
rules:
  - apiGroups: [""]
    resources: ["pods"]
    verbs: ["get", "patch"]
  - apiGroups: ["k8s.cni.cncf.io"]
    resources: ["network-attachment-definitions"]
    verbs: ["get", "list", "create", "patch", "delete"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: dra-sriov-managed-networks
subjects:
  - kind: ServiceAccount
    name: dra-driver-sriov
    namespace: dra-driver-sriov
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: dra-sriov-managed-networks

Patch finalizers on the main NetworkAttachmentDefinition endpoint; the NAD CRD has no dedicated /finalizers endpoint. With blockOwnerDeletion false, the Pod owner reference does not require pods/finalizers update permission. Do not use an owner-reference helper that silently sets it true. No Pod finalizer, Pod delete or Pod update permission is added. K10 K11

The driver must verify Pod/node/claim identities and its ownership before mutations; broad namespace access does not change those checks. An API Forbidden response is surfaced as an access error. Do not add a second namespace-policy mechanism inside the driver or treat a generated name/label as authorization.

12.4 Operator deliverables

Wire the driver opt-in, default cross-namespace Pod/NAD RBAC and matched images into operator reconciliation. Ensure Multus’s actual live-read setting is deployed by whichever component owns Multus; the SR-IOV operator must not silently take ownership of another operator’s Multus configuration.

Add conditions/events for incompatible Multus configuration, missing template grants, missing NAD, conflicting rail mappings and provider overlap. Preserve default legacy behavior with the opt-in disabled.

Deliver the spec.dra.attributes API, SriovNetwork reference resolution, per-policy generated maps, dependency watches/status, unavailable-device behavior and migration rules in section 12.2 as required operator functionality. Regenerate all node-policy CRD copies and install-time schemas so the API server does not prune the new fields. Do not document this as an optional future extension.

13. Exact code change checklist

Repository-relative paths below are implementation targets at the reviewed revisions. Rebase and read the local AGENTS.md/CONTRIBUTING instructions first.

13.1 dra-driver-sriov

Do not add the writable NAD-reference keys to the existing ReservedAttributes list. That list identifies driver-discovered fields whose policy overrides are skipped. Putting the new configuration keys there would make the feature’s DeviceAttributes disappear. Use a separate semantic-validation set. D3 D5

Suggested small new package pkg/multus:

Keep these helpers testable with interfaces for API clients and checkpoint storage. Do not place API requests inside a global device-map lock.

Two exact hooks need special care: run EnsureManagedBindings in the existing prepared return at dra_hook.go:133–139 and after the successful Set at 155–166. The outer global-CDI failures at 62 and 74 currently call rollbackPreparedClaims after that point. Managed checkpointed records must be retained there; destructive rollback would undo already published bindings. Use frozen prepared PCI data for global CDI, group by Pod UID, and retry errors forward. D1

13.2 multus-cni

No new pkg/draclient resolver or claim-status format is part of this implementation. Existing legacy DRA behavior should continue unchanged.

13.3 sriov-network-operator

On the DRA integration branch or its merged successor:

Fix the existing empty-filter translation while retaining a clear distinction between a deliberate all-devices selector, a translated empty node-policy config list, and an unconvertible selector. Do not copy the current PR’s silent skip behavior into the new managed path when that would expose a lower mapping.

Run make generate and make manifests after editing the Go API. Regenerate deepcopies, any generated client/apply configuration outputs owned by the repository, config/crd/bases/sriovnetwork.openshift.io_sriovnetworknodepolicies.yaml and its deployment/sriov-network-operator-chart/crds/ copy. The inspected manifests target copies generated bases into the chart. Inspect both schemas for spec.dra and status.conditions; server dry-run must prove the fields survive admission. Do not hand-update one CRD copy only. O16

Update controllers/sriovnetworknodepolicy_dra_test.go, controllers/sriovnetworknodepolicy_controller_test.go, controllers/sriovoperatorconfig_controller_test.go, API/webhook/rendering tests and test/conformance/tests/test_dra_driver.go. Add a controller test file where one does not already exist. Source NAD resourceName remains required by its current API; the new explicit reference path does not require changing that legacy field.

No sriov-cni code change is required for this initial feature; use its existing deviceID API.

14. Complete target configuration examples

These examples demonstrate the implementation contract. They do not activate automatic attachment on the reviewed unmodified releases. Use exact tested image digests, target Kubernetes APIs and deployment flags from the implementation’s compatibility matrix.

14.1 One NAD for a DeviceClass: no custom reference attributes

Create a normal inline sriov NAD in ai-workloads named rail-a-net, without resourceName or deviceID. A matching SriovResourcePolicy must advertise the intended VFs: discovery alone does not make them allocatable. For this uniform-network pattern, omit deviceAttributesSelector; custom reference attributes are unnecessary. The example policy below advertises the intended PF on labeled nodes. Replace its physical identifiers and do not install an overlapping policy with conflicting per-rail references. Create the namespaces, provision VFs, and install the required components/permissions first. Put the NAD reference once into a DeviceClass:

apiVersion: sriovnetwork.k8snetworkplumbingwg.io/v1alpha1
kind: SriovResourcePolicy
metadata:
  name: single-network-policy
  namespace: dra-driver-sriov
spec:
  nodeSelector:
    nodeSelectorTerms:
      - matchExpressions:
          - key: networking.example.com/dual-rail
            operator: In
            values: ["true"]
  configs:
    - resourceFilters:
        - vendors: ["15b3"]
          pfNames: ["ens1f0"]
          linkType: eth
---
apiVersion: resource.k8s.io/v1
kind: DeviceClass
metadata:
  name: sriov-one-network
spec:
  selectors:
    - cel:
        expression: device.driver == "sriovnetwork.k8snetworkplumbingwg.io"
  config:
    - opaque:
        driver: sriovnetwork.k8snetworkplumbingwg.io
        parameters:
          apiVersion: sriovnetwork.k8snetworkplumbingwg.io/v1alpha1
          kind: VfConfig
          netAttachDefName: rail-a-net
          netAttachDefNamespace: ai-workloads
---
apiVersion: resource.k8s.io/v1
kind: ResourceClaimTemplate
metadata:
  name: one-network-vf
  namespace: ai-workloads
spec:
  spec:
    devices:
      requests:
        - name: vf
          exactly:
            deviceClassName: sriov-one-network
            allocationMode: ExactCount
            count: 1

A Pod uses the claim template through the standard DRA resourceClaims fields, as in section 14.2. It does not repeat a network selection or configure a resource-name match. Narrow the DeviceClass selector to the intended PF/pool/rail when the cluster contains multiple physical networks. A broad driver-only class is appropriate only when all eligible VFs use that template. Do not apply this fallback with conflicting authoritative per-VF references.

14.2 Two GPUs and two interfaces: PCIe-root matching with automatic rail IPAM

The workload requests two GPU/NIC pairs. Kubernetes DRA scheduling selects compatible devices using their discovered resource.kubernetes.io/pcieRoot values. After allocation, the SR-IOV driver resolves the selected VF’s administrator-provided NAD reference and creates its shadow with that rail’s IPAM configuration. Neither the workload claim nor its DeviceClasses contains a rail name, a source NAD name, or a resourceName selector. The administrator’s DeviceAttributes/SriovResourcePolicy objects below still map physical PF/VF inventory to network templates; those mappings are not repeated in workload YAML.

Use one ResourceClaimTemplate containing four count-one requests: gpu-0, nic-0, gpu-1 and nic-1. Scope one matchAttribute constraint to each GPU/NIC pair. An unscoped constraint applies to every request; two count-two requests under one constraint also require all four devices to share one root. Neither represents two independent pairs. K4

The example uses dedicated illustrative DeviceClasses named nvidia-gpu-pcie and sriov-pcie-netdevice. The GPU class selects NVIDIA full GPUs (gpu.nvidia.com, type gpu); both classes require a present, nonempty scalar PCIe-root value without fixing its value. Both drivers already publish the standard attribute. NVIDIA omits it on discovery failure; the reviewed SR-IOV driver can publish an empty string, so a presence check alone is insufficient. Do not add or override pcieRoot through custom DeviceAttributes: it is discovered hardware topology. Use exclusive full-GPU allocation, with no GPU sharing or MIG substitution, for the two-physical-GPU example. D3 G1 G2 G3 K15

For the two-rail acceptance fixture, prepare a node with exactly two eligible full GPUs on different roots and an eligible rail PF under each root:

These roots and GPU labels describe the fixture; they are not selectors. The scheduler may assign either physical row to request pair 0 or pair 1. On other inventories, the two pairwise equalities do not by themselves require different roots, PFs or rails between pairs. An optional explicit distinct-root constraint is shown after the manifest. PCIe-root equality also does not prove closest-switch locality or GPUDirect support.

Copy the following block to multus-auto-gpu-nic-pairs.yaml. Provision the GPUs/VFs, install the NVIDIA DRA full-GPU driver with working CDI/runtime integration, install/configure the required Multus, sriov-cni, regular Whereabouts and enhanced SR-IOV driver, and replace PF names/subnets/node labels for the intended environment. Pin compatible images and verify actual ResourceSlice attributes. Do not combine this direct policy with an overlapping operator-generated policy.

# TWO GPU/NIC PAIRS -- match each interface to its GPU by discovered PCIe root.
# Automatic NAD/IPAM selection requires the implementation described here;
# applying this manifest to the reviewed unmodified SR-IOV driver is insufficient.
#
# Existing API kinds: DeviceAttributes, SriovResourcePolicy, DeviceClass,
# ResourceClaimTemplate, Pod, NetworkAttachmentDefinition, Namespace.
# Administrator mappings below associate physical VFs with network templates.
# Workload DeviceClasses/claims select no NAD names, resourceName values or rails.
# DRA scheduling chooses each GPU/NIC pair; the SR-IOV driver uses each allocated
# VF's attributes to choose its source NAD/IPAM and inject its allocated deviceID.
# netAttachDefNamespace is optional; omission below means the Pod namespace.
# No user-written Pod networks annotation is needed.
#
# Prerequisites / physical substitutions:
# - Install DRA-capable Kubernetes, Multus with the required freshness/protection
#   changes, the enhanced SR-IOV DRA driver in MULTUS auto-attachment mode, the
#   sriov CNI binary, and regular Whereabouts with its datastore/RBAC config.
# - Install NVIDIA DRA full-GPU allocation and its CDI/runtime prerequisites.
#   Use two exclusive physical GPUs, not MIG slices or shared GPU allocations.
# - Verify both drivers publish nonempty resource.kubernetes.io/pcieRoot strings.
#   For the two-rail fixture, each of the two eligible GPUs has a different root
#   and an eligible rail PF/VF under that root; do not configure root values here.
# - Configure the existing policy watch namespace as dra-driver-sriov.
#   There is no additional workload/source namespace flag or allowlist.
# - Provision usable kernel/netdevice VFs on BOTH physical rails on each target
#   node. This file does not create VFs, switch/VLAN configuration, or routing.
# - Replace ens1f0/ens2f0 below with each selected node's actual PF names, or use
#   separate node-specific policies if PF naming differs across nodes.
# - The vendor filter 15b3 selects NVIDIA; change/remove it for other hardware.
# - Label only those prepared nodes networking.example.com/dual-rail=true.
#   Change the label domain/key here and in the Pod if your cluster uses another.
# - Replace documentation CIDRs 192.0.2.0/24 and 198.51.100.0/24 with the real
#   rail subnets; adjust ranges/exclusions to avoid infrastructure addresses.
# - Keep CNI name and Whereabouts network_name stable for every allocation on
#   the same logical rail; do not rewrite either to an ephemeral shadow name.
# - Use a mutually exclusive policy set: no other SriovResourcePolicy may match
#   these same VFs first. Do not combine this direct-policy example with an
#   operator-generated overlapping policy; use the operator-specific mapping
#   option from the document instead when the operator owns DRA policies.
# - Install the documented driver ClusterRole/ClusterRoleBinding for source
#   reads, Pod patching, shadow NADs and cross-namespace orphan inspection.
# - Pin a tested diagnostic image with networking and GPU tools. The toolbox
#   below is used by the SR-IOV demos; verify GPU diagnostics for your GPU stack.
# - Run kubectl apply --dry-run=server before applying on the intended cluster.
---
apiVersion: v1
kind: Namespace
metadata:
  name: dra-driver-sriov
---
apiVersion: v1
kind: Namespace
metadata:
  name: ai-workloads
---
apiVersion: k8s.cni.cncf.io/v1
kind: NetworkAttachmentDefinition
metadata:
  name: rail-a-net
  namespace: ai-workloads
spec:
  # A template: no static deviceID and no resourceName annotation.
  # The enhanced integration supplies the actual allocated VF PCI address.
  config: |-
    {
      "cniVersion": "1.0.0",
      "name": "ai-workloads-rail-a",
      "type": "sriov",
      "ipam": {
        "type": "whereabouts",
        "network_name": "ai-workloads-rail-a",
        "range": "192.0.2.0/24",
        "range_start": "192.0.2.10",
        "range_end": "192.0.2.200"
      }
    }
---
apiVersion: k8s.cni.cncf.io/v1
kind: NetworkAttachmentDefinition
metadata:
  name: rail-b-net
  namespace: ai-workloads
spec:
  config: |-
    {
      "cniVersion": "1.0.0",
      "name": "ai-workloads-rail-b",
      "type": "sriov",
      "ipam": {
        "type": "whereabouts",
        "network_name": "ai-workloads-rail-b",
        "range": "198.51.100.0/24",
        "range_start": "198.51.100.10",
        "range_end": "198.51.100.200"
      }
    }
---
apiVersion: sriovnetwork.k8snetworkplumbingwg.io/v1alpha1
kind: DeviceAttributes
metadata:
  name: rail-a-network-attributes
  namespace: dra-driver-sriov
  labels:
    networking.example.com/template: rail-a-net
spec:
  attributes:
    sriovnetwork.k8snetworkplumbingwg.io/netAttachDefName:
      string: rail-a-net
---
apiVersion: sriovnetwork.k8snetworkplumbingwg.io/v1alpha1
kind: DeviceAttributes
metadata:
  name: rail-b-network-attributes
  namespace: dra-driver-sriov
  labels:
    networking.example.com/template: rail-b-net
spec:
  attributes:
    sriovnetwork.k8snetworkplumbingwg.io/netAttachDefName:
      string: rail-b-net
---
apiVersion: sriovnetwork.k8snetworkplumbingwg.io/v1alpha1
kind: SriovResourcePolicy
metadata:
  name: dual-rail-netdevice-policy
  namespace: dra-driver-sriov
spec:
  # This CR uses core/v1 NodeSelector, not a flat matchLabels map.
  nodeSelector:
    nodeSelectorTerms:
      - matchExpressions:
          - key: networking.example.com/dual-rail
            operator: In
            values: ["true"]
  configs:
    - deviceAttributesSelector:
        matchLabels:
          networking.example.com/template: rail-a-net
      resourceFilters:
        - vendors: ["15b3"]
          pfNames: ["ens1f0"]
          linkType: eth
    - deviceAttributesSelector:
        matchLabels:
          networking.example.com/template: rail-b-net
      resourceFilters:
        - vendors: ["15b3"]
          pfNames: ["ens2f0"]
          linkType: eth
---
apiVersion: resource.k8s.io/v1
kind: DeviceClass
metadata:
  name: sriov-pcie-netdevice
spec:
  selectors:
    - cel:
        # Select eligible SR-IOV inventory with usable topology, not a named rail.
        # The administrator's policy above limits this inventory to netdevice VFs.
        expression: >-
          device.driver == "sriovnetwork.k8snetworkplumbingwg.io" &&
          "resource.kubernetes.io" in device.attributes &&
          "pcieRoot" in device.attributes["resource.kubernetes.io"] &&
          device.attributes["resource.kubernetes.io"].pcieRoot != ""
---
apiVersion: resource.k8s.io/v1
kind: DeviceClass
metadata:
  name: nvidia-gpu-pcie
spec:
  selectors:
    - cel:
        # Full physical GPUs with discovered topology; no GPU index/root pinning.
        expression: >-
          device.driver == "gpu.nvidia.com" &&
          "gpu.nvidia.com" in device.attributes &&
          "type" in device.attributes["gpu.nvidia.com"] &&
          device.attributes["gpu.nvidia.com"].type == "gpu" &&
          "resource.kubernetes.io" in device.attributes &&
          "pcieRoot" in device.attributes["resource.kubernetes.io"] &&
          device.attributes["resource.kubernetes.io"].pcieRoot != ""
---
apiVersion: resource.k8s.io/v1
kind: ResourceClaimTemplate
metadata:
  name: gpu-nic-pairs
  namespace: ai-workloads
spec:
  spec:
    devices:
      requests:
        - name: gpu-0
          exactly:
            deviceClassName: nvidia-gpu-pcie
            allocationMode: ExactCount
            count: 1
        - name: nic-0
          exactly:
            deviceClassName: sriov-pcie-netdevice
            allocationMode: ExactCount
            count: 1
        - name: gpu-1
          exactly:
            deviceClassName: nvidia-gpu-pcie
            allocationMode: ExactCount
            count: 1
        - name: nic-1
          exactly:
            deviceClassName: sriov-pcie-netdevice
            allocationMode: ExactCount
            count: 1
      constraints:
        - requests: [gpu-0, nic-0]
          matchAttribute: resource.kubernetes.io/pcieRoot
        - requests: [gpu-1, nic-1]
          matchAttribute: resource.kubernetes.io/pcieRoot
---
apiVersion: v1
kind: Pod
metadata:
  name: gpu-sriov-pairs-demo
  namespace: ai-workloads
  labels:
    app: gpu-sriov-pairs-demo
  # No user-authored k8s.v1.cni.cncf.io/networks annotation.
spec:
  nodeSelector:
    networking.example.com/dual-rail: "true"
  terminationGracePeriodSeconds: 5
  containers:
    - name: workload
      image: quay.io/schseba/toolbox:latest
      command: ["/bin/bash", "-c", "sleep infinity"]
      resources:
        claims:
          - name: devices
  resourceClaims:
    - name: devices
      resourceClaimTemplateName: gpu-nic-pairs

Optional strict root diversity. If both pairs must use different PCIe roots on arbitrary eligible inventory, append this third entry to the same constraints list. It does not name a rail or NAD; the VF’s administrator mapping still supplies IPAM. distinctAttribute is part of DRAConsumableCapacity, beta and enabled by default in Kubernetes 1.36; verify that gate is enabled on the target control plane. Its use on exclusive devices does not request shared devices. Root diversity implies rail diversity only when the administrator’s physical topology/mappings have that relationship. K4 K16

# Additional entry under spec.spec.devices.constraints, alongside both matches:
- requests: [gpu-0, gpu-1]
  distinctAttribute: resource.kubernetes.io/pcieRoot

Prepare contract for this example. The one claim contains four allocation results across two drivers. Filter by the SR-IOV driver name before managed validation, local-device lookup, NAD resolution, interface numbering, checkpoint/CDI work and cleanup. Produce exactly two SR-IOV bindings/shadows, using the attributes of the allocated nic-0 and nic-1 devices. The NVIDIA driver prepares the two GPUs; its allocation/status/CDI state remains its responsibility. Neither the SR-IOV driver nor Multus reselects GPUs or implements a second topology allocator. Preserve the existing foreign-result/config filtering and status merge. D2 D7 D14

Request names identify logical requests, not CUDA ordinals or physical rail names. In this single-claim example, section 6.5’s ordering maps nic-0/nic-1 to dra0/dra1; gpu-0 does not imply CUDA device 0. Verify pair identity using each result’s request and (driver, pool, device), and then the driver’s VF/interface binding. Do not zip allocation arrays or infer GPU/interface pairing from enumeration order.

14.3 Two rails through SriovNetworkNodePolicy and SriovNetwork

This is the preferred operator-facing example after implementing the new API. Use it instead of the raw NAD/DeviceAttributes/SriovResourcePolicy objects in section 14.2. Retain that section’s sriov-pcie-netdevice and nvidia-gpu-pcie DeviceClasses, gpu-nic-pairs ResourceClaimTemplate and Pod. They request two GPU/NIC pairs matched by PCIe root without selecting any NAD or rail. The operator-generated per-VF attributes determine the network/IPAM after allocation. No Pod networks annotation is added by the user.

Use the operator’s actual installation namespace in place of sriov-network-operator. Create ai-workloads, provision/discover the intended NVIDIA PFs, label the intended nodes, configure the DRA provider/automatic mode and install the documented driver RBAC, and deploy the required Multus build first. These policies request 16 VFs per PF; choose a supported count for the hardware and observe the operator’s drain procedure. The CIDRs are documentation examples. Do not install this example alongside the overlapping direct-driver policies.

Both node policies deliberately use the same resourceName: sriov_vfs. Per-policy attribute selection keeps each physical PF’s network mapping separate. The scheduler matches each allocated VF to its GPU by PCIe root; the allocated VF’s mapping then selects rail IPAM. With the two-root fixture in section 14.2, this exercises both rails without naming either in the claim. The SriovNetwork resourceName values below retain the current legacy-required field; matching those values is not part of this automatic attachment contract.

apiVersion: sriovnetwork.openshift.io/v1
kind: SriovNetwork
metadata:
  name: rail-a-net
  namespace: sriov-network-operator
spec:
  resourceName: sriov_vfs
  networkNamespace: ai-workloads
  ipam: |-
    {
      "type": "whereabouts",
      "network_name": "ai-workloads-rail-a",
      "range": "192.0.2.0/24",
      "range_start": "192.0.2.10",
      "range_end": "192.0.2.200"
    }
---
apiVersion: sriovnetwork.openshift.io/v1
kind: SriovNetwork
metadata:
  name: rail-b-net
  namespace: sriov-network-operator
spec:
  resourceName: sriov_vfs
  networkNamespace: ai-workloads
  ipam: |-
    {
      "type": "whereabouts",
      "network_name": "ai-workloads-rail-b",
      "range": "198.51.100.0/24",
      "range_start": "198.51.100.10",
      "range_end": "198.51.100.200"
    }
---
apiVersion: sriovnetwork.openshift.io/v1
kind: SriovNetworkNodePolicy
metadata:
  name: rail-a-policy
  namespace: sriov-network-operator
spec:
  resourceName: sriov_vfs
  nodeSelector:
    networking.example.com/dual-rail: "true"
  priority: 99
  numVfs: 16
  nicSelector:
    vendor: "15b3"
    pfNames: ["ens1f0"]
  deviceType: netdevice
  linkType: eth
  dra:
    attributes:
      sriovnetwork.openshift.io/sriovNetworkName:
        string: rail-a-net
      sriovnetwork.openshift.io/sriovNetworkNamespace:
        string: sriov-network-operator
---
apiVersion: sriovnetwork.openshift.io/v1
kind: SriovNetworkNodePolicy
metadata:
  name: rail-b-policy
  namespace: sriov-network-operator
spec:
  resourceName: sriov_vfs
  nodeSelector:
    networking.example.com/dual-rail: "true"
  priority: 99
  numVfs: 16
  nicSelector:
    vendor: "15b3"
    pfNames: ["ens2f0"]
  deviceType: netdevice
  linkType: eth
  dra:
    attributes:
      sriovnetwork.openshift.io/sriovNetworkName:
        string: rail-b-net
      sriovnetwork.openshift.io/sriovNetworkNamespace:
        string: sriov-network-operator

Expected operator output, shown for inspection rather than a manifest users apply:

The operator-generated attribute object for rail A contains the following application attributes plus its automatic compatibility resourceName attribute:

spec:
  attributes:
    sriovnetwork.k8snetworkplumbingwg.io/netAttachDefName:
      string: rail-a-net
    sriovnetwork.k8snetworkplumbingwg.io/netAttachDefNamespace:
      string: ai-workloads

The operator-only sriovNetworkName/sriovNetworkNamespace input keys are absent from that output. The corresponding config selects the exact policy-UID attribute object, so rail B cannot merge into rail A despite the shared resourceName. Verify both node-policy DRAConfigurationReady conditions and source-network Ready conditions before starting the workload. A status-only Ready transition must recover automatically without reapplying the node policy.

An administrator may still use independently owned DeviceAttributes with old pool selectors for existing legacy/direct installations, but this new operator workflow does not require that overlay. Do not label arbitrary extra objects with the internal node-policy UID selector.

14.4 Expected runtime artifacts and verification

For each allocated VF, expect one shadow NAD in the Pod namespace, with the exact Pod ownerReference, blockOwnerDeletion false, and the driver protection finalizer. The GPU/NIC example has four total allocation results but exactly two SR-IOV shadows and two generated network selections; GPU results create no NADs. The workload YAML contains no networks annotation, NAD/rail selector or resourceName matching attribute.

For each entry, inspect and correlate:

1. ResourceClaim allocation: request plus driver, pool and device. Confirm two distinct full-GPU results and two distinct VF results.
2. For gpu-0/nic-0 and gpu-1/nic-1 independently, compare nonempty published PCIe-root values and trusted hardware topology. Then identify the allocated VF’s authoritative PCI/PF and source mapping; either logical pair may use either rail.
3. Shadow’s bound Pod/claim/node identities and source reference.
4. Static deviceID in the correct sriov plugin object.
5. Preserved CNI network name and rail IPAM config.
6. Actual Pod interface and address reported by CNI.

The network-status name will normally be the runtime shadow’s namespaced NAD name because that is what Multus selected. This is a real UX tradeoff. Keep source NAD/rail/interface in the internal registry and provide an inspection command or diagnostic output that resolves it. Do not relabel network-status as the source NAD without a separately tested compatibility decision.

Example read-only checks after deployment:

kubectl -n ai-workloads get pod gpu-sriov-pairs-demo -o yaml
kubectl -n ai-workloads get resourceclaims -o yaml
kubectl -n ai-workloads get network-attachment-definitions -o yaml
kubectl -n ai-workloads exec gpu-sriov-pairs-demo -- ip -j address
kubectl -n ai-workloads exec gpu-sriov-pairs-demo -- ip -j route

Inspect GPU visibility with an image/runtime-appropriate diagnostic, such as nvidia-smi -L when available. Compare the actual GPU and VF PCI/root association from trusted node discovery and the driver binding record, not merely the presence of two interfaces or two GPUs. Pairwise equality does not require both pairs to use distinct roots; validate that separately when using the optional distinct constraint. For NVIDIA RoCE, run a separate RDMA/fabric test using actual discovered RDMA devices and the intended peer topology. L3 IP assignment alone does not prove rail isolation, RDMA device exposure, switch configuration or GPU communication performance.

Avoid adding default routes automatically to both rails. Preserve the administrator’s per-NAD routes/gateway. Applications needing source-specific routing or explicit interface/rail selection must still receive that configuration.

15. Existing bugs and corner cases to address

These are findings from the inspected code, not claims of reproduced production incidents. Separate preexisting defects from new design requirements in PR descriptions.

References: D1 D2 D4 D5 D6 D7 D8 M1 M2 M3 O4 O7 N1 I1.

Do not report already-fixed older issues as current: the reviewed PodManager.DeleteClaim preserves other claims in its map, and the current claim-status update retries a fresh merge and preserves other drivers. The remaining global-CDI deletion bug is in the device-state cleanup path, not that map deletion.

Allocation-to-prepare mapping window: ordinary allocation results identify devices but do not freeze arbitrary device attributes or NAD content. This design snapshots current authoritative configuration during first prepare, not at scheduling time. Keep physical rail labels and template mappings stable while claims are allocated; drain/recreate allocations for topology changes. Prepared bindings continue using their frozen snapshot. A guarantee of scheduling-time template identity would require an additional versioned allocation contract and is outside this simpler path.

16. Implementation order and acceptance gates

Make the feature reviewable in a small sequence. Do not ship the driver opt-in before the compatible Multus deployment is available.

Each PR should state the concrete behavior change, compatibility boundary, and tests actually run. The node-policy attribute extension is required here. A replacement network API, extended-resource support, arbitrary bonding, full hardware journaling or RDMA metadata parity remain separately scoped.

16.1 Proposed helper contracts

Resolve implementation types to the repository’s conventions, but keep the responsibilities:

• ResolvePodForClaim(ctx, claim) -> VerifiedPod: live identity/reservation/reference validation.
• SnapshotEligibleDevice(allocationIdentity) -> device/attributes/eligible: atomic fresh-prepare input; preserve discovery and cached bindings when eligibility is withdrawn.
• ResolveTemplate(deviceSnapshot, effectiveConfig, podNamespace) -> NamespacedName: deterministic reference precedence and Pod-namespace defaulting; fetch the selected source under normal API authorization.
• Operator BuildDRAPolicyPlan(ctx, policies) -> ready/unready policy plans: validate inputs, resolve SriovNetwork/NAD once, generate complete desired maps/selectors and diagnostics without dropping bad configs.
• Operator ResolveSriovNetworkReference(ctx, ref) -> sourceNAD identity/readiness: use live global reader and existing rendering rules; consume operator-only inputs.
• RenderShadowConfig(sourceNAD, allocatedPCI) -> RenderedConfig: pure output containing exact bytes, effective logical name, plugin index/type and SHA-256.
• BuildBindingIdentity(node, pod, claim, allocation) -> attachmentID, shadowName: pure stable hash.
• MergeManagedSelections(pod, desiredBindings, priorOwnedRegistry) -> patch/no-op: preserve unrelated fields and enforce ownership.
• EnsureShadow(ctx, binding) -> observedUID: create-or-verify with Pod ownerReference/finalizer, reject Terminating, never arbitrary overwrite.
• EnsureManagedBindings(ctx, podIdentity) -> error: snapshot Pending/Ready bindings, ensure healthy protected shadows, persist UIDs, patch registry/selections, mark completed claims ready.
• ReleaseManagedClaim(ctx, claimIdentity) -> error: Deleting, exact Delete request/selection removal, existing host/CDI cleanup, durable ReleaseComplete, owned-finalizer removal, checkpoint deletion last.
• FinalizeReleasedShadow(ctx, binding) -> error: API-only Delete/finalizer completion with UID/resourceVersion checks; preserve foreign keys, handle lost patch responses, never touch hardware.

Use a small fakeable checkpoint interface and typed errors. Expected reconciliation errors should not trigger the driver’s fatal-background-error cancellation path. Reserve process-fatal errors for genuinely unusable local state such as corrupt checkpoints or impossible node identity.

16.2 Completion criteria

The feature is complete only when a user can submit a Pod containing standard DRA claims and obtain the correct secondary interfaces without a user-written networks annotation, without a source resourceName annotation, and without a user-written resourceName attribute.

The GPU/NIC example must allocate two physical GPUs and two VFs, satisfy each pair’s PCIe-root equality, and assign each VF an address from its own administrator-mapped rail subnet without a workload NAD/rail selector. The two-root fixture exercises both subnets. Restart/retry/DEL behavior must retain those allocation-to-template associations. Source NADs must remain unchanged, Multus must perform zero DRA/resource allocation lookups for managed shadows, and normal cleanup must leave no shadows or IPAM leaks.

17. Verification plan

17.1 Unit tests: driver

Use table-driven tests for semantic cases and fault injection for lifecycle boundaries. These are meaningful correctness tests, not snapshots of every private helper line.

The policy semantic validator must block publication/use of an invalid managed name/optional-namespace reference with a clear condition/event; it must not remove the reference and accidentally fall back to a different network.

17.2 Deterministic failure-injection matrix

Inject failures at the actual side-effect boundaries, restart the driver where indicated, and compare checkpoint, Pod, NADs and host actions.

Cover API Forbidden, NotFound, timeout and conflict separately where they imply different recovery. Avoid a mock that returns only “some error” for all paths.

17.3 Unit/integration tests: Multus

The decisive regression test is a successful stale informer read, not merely NotFound:

1. Put the old Pod without managed entries in its informer.
2. Put the patched Pod and newly created static shadow in the fake/live API.
3. Leave the shadow absent from the NAD informer.
4. Enable liveNetworkConfigOnAdd.
5. Invoke ADD through the actual loading path.
6. Assert the intended sriov delegate is invoked with the allocated PCI and rail IPAM, or the operation fails visibly. It must never succeed primary-only because the cache was old.
7. Assert no resource-client factory, ResourceClaim GET/LIST, or ResourceSlice GET/LIST was used for the shadow.
8. Assert the shared daemon client still has its original informer pointers.

Also test the missing runtime Pod UID path, daemon/shim flag round-trip, enabled and disabled concurrent requests, API failure without stale fallback, generated-NAD UID/hash/owner/finalizer/namespace guard, duplicate interfaces, unrelated legacy NAD compatibility, and cached DEL after all relevant API objects disappear. A managed NAD with deletionTimestamp must be rejected for ADD before any delegate runs; cached DEL and a DEL fallback reading the original retained NAD must still work during finalization.

Use fake CNI plugins that record stdin and command order for control-flow tests. Compile shared binding/registry fixtures in both repositories to prevent field-name/hash drift. Do not infer real VF functionality from a fake plugin.

17.4 API integration and operator tests

Use envtest or a suitable Kubernetes test cluster for admission, resourceVersion conflicts and permission checks. Fake client success alone does not prove RBAC or real patch semantics.

• Server dry-run all documented YAML and compile/evaluate CEL selectors on the target version.
• For the GPU/NIC example, assert four count-one requests and exactly two pair-scoped matches. DeviceClasses/claims contain no NAD/rail/root-value selectors. Check root-present/nonempty filters, full-GPU type selection, both request-order permutations and incompatible/missing-root inventory. The optional distinct constraint must survive admission when its gate is enabled; do not accept a silently dropped constraint.
• Exercise concurrent NVIDIA/SR-IOV status updates and prepare/release in either driver order. Preserve GPU opaque config/status/CDI and create only two NIC shadows from each allocated VF’s own template.
• Verify the installed driver ServiceAccount can patch Pods/create shadows in multiple Pod namespaces with the default ClusterRole/ClusterRoleBinding. Separately simulate API Forbidden to verify clear access errors and retained retry state; no driver namespace allowlist is involved.
• Verify omitted and empty source namespace resolve to the Pod namespace, including when DeviceAttributes lives elsewhere. An explicit shared source namespace is read under normal API authorization; the generated shadow remains Pod-local. Multus applies existing namespace isolation to the actual selected NAD, with no new source-provenance policy.
• Verify operator-created source NAD is unchanged across prepare and operator reconcile.
• Verify the installed base/chart node-policy CRD stores spec.dra.attributes and status.conditions rather than pruning them. Check typed zero/false values, scalar union, unsupported list values, qualified-key syntax, semver, >64-character values, partial refs, reserved keys and final merged budget.
• Apply a node policy before its SriovNetwork and before the target NAD namespace; verify non-ready conditions then automatic recovery after both dependencies appear. A dependency need not exist at admission.
• Resolve source NAD namespace both with and without networkNamespace; reject a non-operator-namespace SriovNetwork attempting redirection. Do not confuse reference namespace with target namespace.
• Require Ready for the current SriovNetwork generation and matching rendered NAD content/ownership. Test stale observedGeneration, missing/deleting NAD, changed config, removed owner marker and CR/NAD UID replacement.
• Two policies with the same resourceName and disjoint PFs must produce separate per-policy maps/selectors and different NAD refs. Neither new object has the old pool label. Legacy selectors must not match them, and legacy cleanup must not delete them.
• Change/delete/recreate a policy with a new UID; verify desired-name cleanup and no adoption of an unowned same-name attribute object. Preserve independently owned legacy/direct DeviceAttributes.
• Mixed healthy/unready policies must still reconcile healthy entries and invalidate the stale entry. Keep the invalid policy config/filter present; no partial-map publication or lower-config fallback. Simulate API failures during invalidation and require readiness to remain false/unknown until applied.
• Verify node-policy source GETs/events work across the namespaced/global manager boundary. Status-only Ready updates must trigger recovery; old/new owner-marker removal events must trigger invalidation; generated Pod-shadow events must not rebuild policy inventory.
• A valid-but-different SriovNetwork.resourceName must not affect explicit reference resolution or static-device attachment. Assert no equality-based resourceName linkage is introduced.
• Test disjoint shared-pool rails and conflicting overlapping filters. Unsafe overlap/translation must withdraw that node’s generated configs instead of keeping an old mapping; empty configs means no advertised devices, while an individual empty ResourceFilters remains match-all.
• Test nil dra block legacy behavior, empty dra block per-policy behavior, attributes-only policies with uniform VfConfig, provider-disabled staging and re-enablement. New behavior must not modify legacy device-plugin config.
• Verify new unprepared claims fail eligibility checks after invalidation, including claims allocated before withdrawal. Pending/Ready checkpoints continue with frozen config, and resource unprepare/finalizers still complete.
• Test the operator’s empty-filter translation fix and the driver’s empty-key advertisement-removal regression together with ResourceSlice observations.
• Verify chart and static manifests render the automatic-attachment opt-in, default cross-namespace Pod/NAD permissions and persistent paths, and contain no workload/source namespace flags or allowlists.
• With a real garbage collector, test both foreground and background Pod deletion: exact ownerReference, Pod not blocked by the NAD, and shadow retained until the driver removes its key. Envtest alone does not run the garbage-collection controller.
• Verify finalizer metadata is present in the initial NAD Create, not a later patch; admission succeeds without Pod delete or pods/finalizers permissions.
• Test namespace termination: existing finalizers remain patchable, no new shadow creation is attempted, and cleanup credentials/controllers stay available.
• Verify orphan propagation actually removes the Pod ownerReference, then confirm unprepare still explicitly deletes/finalizes the shadow using its recorded UID and binding identity. Cover empty-UID lost-Create recovery in this state. Retained completed Pods must not retain released shadows.
• Verify the cross-namespace orphan scan finds a protected object in a Pod namespace absent from local checkpoints, reports it without removing its finalizer, and ignores other node UIDs.
• With auto attachment disabled, existing manual DRA, standalone mode and legacy device-plugin behavior remain unchanged.

17.5 Hardware and runtime E2E acceptance

Run the minimal path on a node with real usable VFs, then the two-GPU/two-NIC topology fixture and a two-node/two-rail IPAM setup with the intended NVIDIA GPUs/NICs. Reuse test/e2e/framework/alignment.go’s ExpectResourceClaimRequestsSharePCIeRoot once for gpu-0/nic-0 and once for gpu-1/nic-1; never pass all four when testing pairwise alignment. Assert request/result counts and physical GPU uniqueness separately. Synthetic GPU fixtures prove scheduling control flow, not NVIDIA hardware compatibility. Use both supported runtime versions if claiming containerd and CRI-O support. D15

For host-local, run a same-node multi-shadow regression with a source lacking config.name and prove distinct addresses. Across nodes use disjoint ranges or use the Whereabouts example; do not accidentally describe a same-subnet host-local test as a supported coordinated allocator.

For RDMA, additionally validate discovered RDMA device exposure, intended GID/MTU/routing and communication on each physical rail with the actual workload tooling. That is a hardware qualification milestone; ordinary IP ping and mocked VFs are not substitutes.

17.6 Repository commands and recorded evidence

At the inspected heads, driver make test configures envtest assets, operator make test runs generation/lint/manifests/envtest, and Multus make test invokes its privileged test script. Run them in appropriate build/test environments. Do not run a cluster creation or privileged network suite against an unspecified production environment.

Useful focused commands after implementing the corresponding packages:

# dra-driver-sriov checkout, with the repository's envtest assets configured
go test -race ./pkg/driver ./pkg/devicestate ./pkg/podmanager ./pkg/types ./pkg/multus
make test

# multus-cni checkout; the repository's full suite needs its documented privileges
go test ./pkg/k8sclient ./pkg/types ./pkg/server/config
make test

# sriov-network-operator integration checkout
make test

The driver’s existing isolated virtual-cluster entry points are:

make deploy-single-node-virtual-cluster-multus
make deploy-multi-node-virtual-cluster-multus
make e2e-workloads E2E_LABEL_FILTER='Multus'

Choose one cluster deployment size for the intended test; these are alternatives, not three commands to run blindly. Add the new test label/fixtures to the suite so the documented filter actually covers automatic mode.

For each completed gate, retain commit/image/runtime/Kubernetes versions, test commands, logs, actual claim-to-PCI-to-shadow mapping, Pod network-status, IPAM lease observations, and post-teardown inventories. Report failures and skipped hardware tests plainly.

What this investigation validated: source paths and call order at the pinned revisions; static-deviceID feasibility; pending operator PR contents/state; local YAML/embedded-JSON parsing and reference shapes. It did not implement these features, compile new CEL against an API server, run repository suites for unmade changes, attach a real VF, or execute a live-cluster E2E test.

18. Operations, rollout, and troubleshooting

18.1 Safe first rollout

1. Pin the compatible driver, Multus, sriov-cni and IPAM builds, plus target Kubernetes/runtime versions.
2. Install the Multus live-read/guard build and verify its effective daemon configuration on every target node.
3. Prepare disjoint hardware selectors and rail templates. For operator-managed nodes, use SriovNetwork plus spec.dra.attributes on node policies; verify current DRAConfigurationReady and source Ready conditions. Direct installations may use DeviceAttributes. Validate source/workload namespaces and forbid independent overlapping policies.
4. Install the documented ClusterRole/ClusterRoleBinding for the actual driver ServiceAccount. Verify Pod patching, configured source reads, shadow writes/finalization and cross-namespace NAD listing under its effective privileges.
5. Drain intended test nodes and release old/manual allocations before switching providers or enabling managed mode. Ensure one provider owns each VF inventory. New shadows must be created with protection; do not enable this version against old automatic bindings lacking a tested ownership/finalizer migration.
6. Enable the driver opt-in and verify checkpoint recovery readiness.
7. Run the one-NAD smoke test, then two-rail/multi-node acceptance and teardown.
8. Expand gradually while measuring API traffic, failed prepares and orphan counts.

Disabling the flag does not make existing shadows/annotations/finalizers safe to abandon. Drain/release managed workloads first; verify no managed checkpoint records or owned protection finalizers remain; then disable, downgrade or uninstall. Keep the driver finalizer worker and its cleanup RBAC available until cleanup finishes. Uninstalling them first can leave NADs and workload namespaces stuck Terminating. Do not ship an uninstall hook that strips finalizers without release/fencing evidence. The old binary may ignore unknown binding fields and is not a safe cleanup implementation.

18.2 Actionable errors

Use structured logs/events with Pod/claim UID, driver/pool/device, source reference and binding ID. Avoid dumping complete CNI configs or IPAM credential material.

At minimum expose counts of bindings by phase, prepare/repair/cleanup failures by bounded reason, and repair latency. Keep Pod/claim/binding IDs in logs rather than high-cardinality metric labels. A bounded periodic worker should rate-limit API errors and emit transitions instead of endlessly repeating the same event.

18.3 Object growth and orphan handling

The API object cost is approximately one shadow per prepared VF, plus Pod metadata. Record expected scale explicitly. A thousand Pods with eight VFs means roughly eight thousand live shadows; benchmark informer/API/checkpoint behavior at the intended size rather than assuming objects are free.

Pod ownership triggers ordinary cascading deletion; NodeUnprepare explicitly requests deletion and discharges the driver finalizer after local cleanup. A missing node requires inspection/fencing first. Provide an admin command that lists managed shadows by node UID and correlates checkpoint/claim/Pod identities, with dry-run output. Distinguish active, Deleting, ReleaseComplete and orphaned records. Use UID-precondition DELETE and UID/resourceVersion-checked removal of only the owned finalizer. Report foreign finalizers separately. Do not use a broad kubectl delete nad --all or blanket finalizers=[] recipe.

Protect driver-owned metadata through organizational/admission policy if workloads or GitOps controllers routinely overwrite annotations. Exempt legitimate driver changes and release cleanup. Without that protection, document the full-metadata-removal limitation rather than making extra allocation lookups an implicit fallback.

19. Better options and additional UX improvements

19.1 Choices recommended for this implementation

• Uniform network: use one existing DeviceClass VfConfig NAD reference. This removes custom mapping attributes from the direct-driver common case.
• Operator-managed rails: put the SriovNetwork reference on each node policy; add generic custom scheduling attributes only when useful. Let the operator create the driver mapping. The user does not maintain generated DeviceAttributes or match resource-name attributes.
• Automatic rail/IPAM selection: administrators map each eligible VF to a NAD name and optional namespace. Workloads request GPU/NIC pairs with standard PCIe-root constraints; after scheduling, each allocated VF’s mapping chooses its rail template. No workload NAD-name, resourceName or rail selector is required.
• Frozen private shadows with Pod ownership and NAD protection: use the existing static-CNI path, let GC request dependent deletion, and let authorized driver cleanup remove the finalizer. Multus remains independent of allocation details.
• Generic live ADD reads: fix the consistency boundary once; avoid DRA-specific cache conditions.
• ResourceClaimTemplate by default: one fresh exclusive claim per workload avoids the most confusing reuse/sharing cases.
• One request per device in each topology pair: four count-one requests and two pair-scoped PCIe-root constraints express two GPU/NIC pairs. Use a distinct-root constraint only when cross-pair root diversity is also required; do not encode rail names to achieve locality.
• Regular coordinated IPAM: keep subnet allocation correct across nodes, with stable logical network identities.

19.2 Useful follow-ups, in priority order

1. Preflight/inspection command. Given a DeviceClass/claim/Pod, show matching VF/rail, source NAD, interface, permission gaps, conflicting manual entries and effective CNI/IPAM identity before users debug a stuck sandbox. The driver already owns this information; no Multus allocation lookup is necessary.
2. Readable diagnostics for shadows. Report source NAD and rail next to generated names in events/inspection output. Offer stable explicit per-request interface configuration where count-one requests make it unambiguous.
3. Typed operator reference follow-up. The spec.dra.attributes implementation in section 12.2 already provides per-policy SriovNetwork mappings and shared-pool rail selection. A future typed reference field could improve editor validation/autocompletion; it must translate to the same driver keys and preserve per-policy selectors.
4. Template readiness validation. For explicit-namespace and operator-resolved references, surface missing/malformed/unauthorized source NADs in policy/operator conditions before scheduling reaches NodePrepare. For Pod-relative references, validate syntax at policy reconciliation, then resolve and check the actual source after the consuming Pod is known. Do not change an already frozen binding during reconciliation.
5. Optional PCI/RDMA device-info parity. Populate diagnostics from driver-discovered metadata already carried by the shadow, with no resourceName lookup.
6. Conditional operator NAD rendering. If demand justifies it, add an explicit DRA-template mode that omits legacy resourceName and changes CRD validation conditionally. Preserve the default DP API behavior.
7. Extended-resource claim UX. Add its separate resolver and kubelet cache/lifecycle tests before advertising it; it may reduce workload YAML further.
8. Larger topology and recovery features. Arbitrary bonding/multi-VF conflists, fast Whereabouts slices, sharing, live network changes and redundant CNI journals need their own designs and hardware tests.

19.3 Alternatives considered

The selected design has a deliberate cost: generated NADs, Pod mutation, and a small consistency patch in Multus. In exchange, every CNI invocation receives an ordinary frozen SR-IOV configuration with the allocated PCI already present, and workload authors configure only their DRA resource needs.

20. Source index

Links below point to inspected commits, the explicitly identified PR discussion, and official documentation where noted. They let an implementation agent verify each claim against source even after default branches move. Proposed behavior and names elsewhere in this document are design decisions; source links establish current behavior and the reasons for changing it.

Driver

• D1 — Prepare hooks, checkpoint fast path, outer rollback, and unprepare

• D2 — Device preparation, policy snapshots, advertisement, and CDI cleanup

• D3 — Hardware discovery and built-in VF attributes

• D4 — VfConfig fields and incomplete Override

• D5 — Policy precedence, attribute merge, and selector implementation

• D6 — Root-only injector, PreparedDevice, and existing checkpoint

• D7 — Existing request configuration resolution

• D8 — UID/conflict-safe claim status retries

• D9 — Checkpoint storage, locking, rollback, and per-claim deletion

• D10 — DeviceAttributes and SriovResourcePolicy CRD API

• D11 — Empty versus default driver binding behavior

• D12 — Discovery-owned ReservedAttributes and driver identity

• D13 — Discovered PCI/PF/RDMA metadata and auxiliary files

• D14 — Claim device status data and merge helpers

• D15 — Existing E2E PCIe-root alignment helper

Multus

• M1 — ClientInfo, informer-backed Pod/NAD reads
• M2 — resourceName branch, static delegate loading, and Pod networks
• M3 — Existing resourceName-based DRA discovery path
• M4 — Raw delegate loading, PCI injection, conflist and runtime capabilities
• M5 — ADD order, delegate cache, network-status, DEL fallback and teardown
• M6 — Thick configuration overlay, runtime UID fallback, daemon loader
• M7 — Generated Multus configuration fields and marshalling
• M8 — Preservation of the supplied client

Operator

• O1 — Master SriovNetwork required resourceName and CNI/IPAM fields
• O2 — Operator source-NAD namespace and reconciliation ownership
• O3 — Open operator DRA integration PR #1151; state checked 22 October 2026
• O4 — Pending per-pool attributes, reconciliation, policy translation
• O5 — Pending driver ClusterRole: NAD get and existing DRA permissions
• O6 — Pending provider transition and rollout
• O7 — Pending empty-filter translation mismatch
• O8 — Pending Pod role limited to operator namespace
• O9 — Master NAD template, resourceName annotation and logical name
• O10 — Pending operator driver deployment rendering
• O11 — Pending node-policy API/status before this extension
• O12 — SriovNetwork renderer and NAD namespace/defaulting
• O13 — Node-policy reconciliation order, startup trigger and watch list
• O14 — Existing node-policy validation and resource-name collision handling
• O15 — Namespaced/global managers and controller wiring
• O16 — Operator API/CRD generation and Helm copy targets
• O17 — Reusable operator condition schema
• O18 — Operator condition patcher and GetConditions contract

Kubernetes

• K1 — PrepareDynamicResources before createPodSandbox
• K2 — DRA RPC claim identity, with no Pod argument
• K3 — Kubelet prepared-claim cache and multi-Pod reference handling
• K4 — DRA attribute limits, consumer identity, config and allocation semantics
• K5 — Name, IsForPod, reservations, and ConfigForResult
• K6 — Unprepare occurs before terminal Pod status update
• K7 — Checkpoint restore resets the in-memory prepared flag
• K8 — Kubernetes finalizer behavior and deletion restrictions
• K9 — Kubernetes cascading deletion and blockOwnerDeletion
• K10 — Owner-reference admission and owner-finalizer authorization
• K11 — CRD main/status/scale routing and main-resource PATCH
• K12 — JSON Patch test failure HTTP 422 behavior
• K13 — Normal KillPod calls StopPodSandbox before resource unprepare
• K14 — Metadata validation forbids new finalizers after deletion starts

GPU topology example

• G1 — NVIDIA full-GPU DeviceClass predicate
• G2 — NVIDIA standard PCIe-root discovery and failure behavior
• G3 — NVIDIA publication of discovered topology attributes
• K15 — Official standard device-attribute contract
• K16 — Official DRA feature documentation, including distinctAttribute

CNI and IPAM

• C1 — SR-IOV CNI deviceID parsing and validation
• C2 — SR-IOV CNI ADD, IPAM, local VF-state cache, and DEL
• N1 — Inline CNI loading and missing-name defaulting
• I1 — host-local allocation directory keyed by logical CNI name
• I2 — Whereabouts allocation-domain naming
• I3 — Fast node-slice NAD ownership and configuration interactions
• I4 — Whereabouts storage namespace and overlap-reservation identity
• I5 — Whereabouts external configuration merge