DRA Composable Networking with gRPC NetworkPlugins

Status: Proposed architecture for review; not an implemented or upstream Kubernetes API.
Revision: 2026-09-23 — align discovery capability names and MAC lease v2 types; retain the typed-reference, IPAM/MAC, workload-context, and stable-identity extension of the four-hook design.
Companions: Network Device Discovery and ResourceSlice Design and Framework and Plugin SDK.
Terminology: NetworkPlugins are the small execution components defined here. Existing CNI implementations may supply reusable code, but their binary protocol, configuration schemas, and result semantics are not this protocol.
Reading the examples: Kubernetes resource.k8s.io/v1 objects use upstream APIs. networking.dra.io/* resources, plugin schemas, protocol messages, labels, and state machines are project proposals. They require implementation and installation before use. Example driver/API domains retain the original draft’s spelling; the project must select domains it controls before release.

Executive Decision

Build one logical network DRA authority that publishes devices, describes compatibility and capacity, and coordinates their safe use. Keep actual network realization in small gRPC NetworkPlugins orchestrated as a typed directed acyclic graph (DAG).

The network authority does not replace kube-scheduler: it publishes constraints through ResourceSlices and validates the resulting allocations. It also does not replace GPU drivers, fabric controllers, IPAM services, or the primary Pod network. Those systems have explicit integration boundaries.

This revision uses structured inputs.<port>.from references exclusively. Literal config and verified WorkloadContext remain separate. IPAM and MAC plugins return typed lease outputs that VLAN or other explicitly capable steps can consume. Stable network identity belongs to a verified workload subject; it is not inferred from arbitrary labels or a sandbox name. Section 14 defines the reservation and handoff model.

The execution API has exactly four mutating lifecycle hooks:

Root-only preparation means every allocation root, not one distinguished root for the entire graph. A derived operation never receives NodePrepare or NodeUnprepare.

Two additional driver responsibilities are mandatory but are not extra NetworkPlugin lifecycle hooks: a container-start readiness gate through the runtime integration, and continuous recovery of journaled work. Read-only plugin inspection and health APIs are also separate from the four lifecycle operations.

This is a resource-aware, recoverable orchestration design—not an assertion that a sequence of network mutations is an atomic transaction.

1. Problem, Goals, and Boundaries

1.1 Problem

Complex workloads need more than a device name. Examples include two allocated VFs forming an Ethernet bond, VLAN branches with separate address allocation and routing, independent RDMA rails colocated with GPUs, and VFIO devices delivered to userspace packet processing. Platform teams need to encapsulate those arrangements without delegating arbitrary host-network administration to application Pods.

The original proposal’s useful separation remains: publish and allocate the roots, then realize a topology on the selected node. The replacement changes the execution contract so multiple inputs, resource ownership, retries, and teardown have explicit semantics.

1.2 Goals

• One authoritative inventory and conflict model for the network resources managed by this project.
• Declarative, reusable topologies with administrator-approved plugins and immutable execution revisions.
• Scheduler-visible allocation of physical devices or explicitly modeled shared-parent capacity.
• Small independently testable NetworkPlugins for device attachment, bonding, VLAN creation, tuning, IP allocation, MAC allocation, and application of their typed leases.
• Predictable behavior across retry, partial failure, sandbox replacement, plugin upgrade, coordinator restart, and node reboot.
• A mixed-driver ResourceClaim where appropriate, without the network driver taking over GPU preparation.
• A path from a secondary-network implementation to more sophisticated SDN and hardware-offload integrations.

1.3 Non-goals for the initial implementation

The initial implementation does not replace the primary Pod-network provider, add a new Kubernetes scheduler, provide cross-node gang scheduling, automatically configure the physical fabric, or support arbitrary tenant-supplied executables. It does not promise that NIC/GPU proximity guarantees GPUDirect RDMA, that a Linux bond is an RDMA bond, or that scheduler capacity reservations configure hardware QoS.

It does not support unrestricted multi-Pod reuse of one network claim, hot modification of a live topology, or unmodeled host resources hidden behind derived steps. Stable workload reservations are included in the design; KubeVirt live handoff is an opt-in integration profile requiring a tested binding, migration controller and actual dataplane fencing. It is not enabled merely by requesting a stable MAC.

Upstream boundary: the design uses existing DRA objects and kubelet plugin RPCs, plus project CRDs and node/runtime integration. No additional field on upstream ResourceClaim is proposed. However, a runtime that cannot meet the NRI readiness and namespace-lifetime contract is not supported merely because it supports DRA. Some environments may need runtime changes or a different adapter. U1 U2 U5

2. Architecture and Ownership

2.1 Control plane and node plane

Platform administrator
  |-- DeviceExposurePolicy ---> inventory/policy compiler ---> ResourceSlices
  |-- NetworkPlugin registry -> approved contracts and deployment
  `-- NetworkTopology --------> immutable revision + generated DeviceClasses
                                                           |
Application ResourceClaimTemplate / ResourceClaim ----------+
                                                           v
                                                   kube-scheduler
                                                   allocates roots
                                                           |
                                                           v
Node network DRA agent <---- kubelet DRA Prepare / Unprepare
  |-- claim resolver and durable ownership/journal store
  |-- typed DAG compiler/executor and namespace-handle manager
  |-- NRI adapter + per-sandbox readiness gate
  `-- gRPC clients over local Unix sockets
           |-- sriov / host-device / macvlan / bridge / vfio roots
           |-- bond / vlan / tuning operations
           `-- ipam / mac / address / optional guest-binding and SDN integrations

“Central” describes authority, not a single global execution bottleneck. Controllers may be replicated with leader election for writes. Each node has one active execution coordinator, fenced against a previous instance. Link operations execute locally; no cluster-global service is inserted between every pair of local graph steps.

Plugins may run as separate node daemons, or several may share a trusted process and common SDK. The public contract remains the same. Process boundaries are an operational choice, not the definition of composability.

2.2 Responsibility matrix

All plugins remain trusted node software. A gRPC boundary improves modularity and auditing; it does not make a privileged plugin unable to bypass host security. Least-privilege execution is a separate hardening concern.

2.3 One owner per physical effect

The DRA inventory owns allocation identity. A NetworkPlugin owns specific realized effects. A bond plugin owns the bond and its member relationships, not the allocated VFs themselves. An address plugin owns the addresses and routes it installed, not every route in the namespace. The primary network owns its own interface, default route, and resolver integration unless a separately approved profile explicitly delegates them.

Two independent DRA drivers must not advertise the same physical network function as independent available inventory. Compatibility groups are not a cluster-global lock across driver identities. Shared hardware ownership needs one authority or explicit partitioning. U3

3. Core Objects, Identities, and Lifetimes

3.1 Project objects

The status object is the required consumer-facing observation contract, but successful API publication is not a prerequisite for the initial L2 data path. The execution journal is durable node-local state, not an API-server status field. Section 15 defines the status fields and integration boundaries. Full structural CRD schemas and controller code remain implementation deliverables; the examples here are resource shapes, not installable CRD definitions.

3.2 Identity hierarchy

Use separate identities rather than one podUID key:

Root allocation identity:
  claim UID + allocation fingerprint + driver/pool/device/shareID + root step

Preparation identity:
  node UID + root allocation identity + preparation incarnation

Topology instance identity:
  claim UID + allocation fingerprint + topology revision UID

Sandbox attachment identity:
  node UID + boot ID + Pod UID + runtime sandbox ID
  + topology instance identity + attachment attempt incarnation

Step operation identity:
  preparation or attachment scope + step name + hook + desired generation

Stable network identity (distinct from a DRA allocation):
  cluster/namespace boundary + immutable network UID + verified workload subject
  + stable interface key + identity generation

Lease holder identity:
  network identity UID + component + attachment identity

An allocation fingerprint is a canonical digest of the relevant allocated results and authoritative configuration, excluding mutable status observations and unrelated drivers’ status. A shared allocation uses its actual shareID; an empty share ID is not a wildcard. Physical identity is obtained from the allocation-aware inventory, not reconstructed solely from a possibly reused interface name or PCI address.

Discovery Sections 4.1.1–4.1.5 define cold PCI-first discovery, including VFIO-bound and unbound functions, persistent presence epochs, optional netdevice observations, and typed host-dependency protection. Before mutation or historical cleanup, revalidate the committed identity and the relevant owner/dependency evidence. Unknown or changed ownership blocks that mutation and retains cleanup debt; it does not authorize cleanup against a replacement device. Discovery cases F12-01–F12-18 are acceptance criteria, not executed hardware results.

A container restart within the same sandbox reuses networking. A new sandbox for the same Pod UID is a new attachment. A retry of a single RPC is not a new attachment attempt. A new holder may reuse a retained reservation after the previous endpoint is removed or fenced. Node/sandbox and allocation identity must not be embedded into a stable VM address key. The immutable context snapshot is attached to the operation, not to the long-lived address identity.

3.3 Initial consumer contract

The first profile is attachmentPolicy: SinglePod: one consuming Pod per topology claim and no simultaneous active sandboxes using the same network allocation. A ResourceClaimTemplate is the normal application path. The node agent atomically binds its prepared instance to the authorized Pod/sandbox, and rejects a second consumer or overlapping replacement until cleanup is resolved.

This is a project restriction. Upstream claims can be reused; additional consumers do not automatically reserve additional network capacity. Multiple containers in one authorized Pod share the sandbox network. Container-level device access remains controlled by DRA request references and CDI. U2 U3

For an approved live migration, source and target use separate DRA claims and separate attachment holders that reference the same retained network identity. The per-claim SinglePod rule is not relaxed. Section 14 defines how a standby target is distinguished from a second active user.

Admission catches obvious claim reuse, but a count observed at admission is not a race-free ownership lock. The node agent’s durable compare-and-set and sandbox identity checks remain necessary.

The MVP supports one network topology instance in one network claim per Pod; that claim can also contain independently managed GPU requests. Multiple independent network claims per Pod are a later extension requiring Pod-wide interface-name, route, and readiness validation. The identity model already distinguishes their instances.

4. The Four-Hook Lifecycle Contract

4.1 NodePrepare: root preparation, no Pod namespace

The node agent resolves the exact allocated claim, validates the whole plan, verifies required plugin contracts, then invokes NodePrepare on each allocation root. Derived plugins can be inspected for readiness but do not receive preparation hooks.

A root may validate hardware identity, hold a local reservation, snapshot restorable settings, apply an explicitly authorized device-binding change, and prepare container device access. It must not require a Pod UID or Pod network namespace. The kubelet DRA request identifies claims, not a particular Pod sandbox. U1

For shared parents, preparation holds an entitlement and validates a compatible parent configuration. It must not reinitialize a shared PF for every claim or assume the last locally prepared claim is the last scheduler allocation. Shared parent state needs centrally coordinated, persistent references and a stable configuration contract.

A successful response includes versioned preparation state, optional typed preparation outputs, and any CDI device IDs. The node agent validates and persists the entire response before reporting DRA preparation success. CDI entries must already exist and be scoped to the allocated roots and requests.

4.2 ConnectNetwork: roots and derived steps

The NRI adapter receives the runtime’s sandbox-create event, resolves the Pod’s actual claim references, verifies authorization and prepared state, and acquires a persistent namespace handle. It freezes the effective, per-plugin projected WorkloadContext and resolves approved stable identities. It then runs every required step in topological order. Reservation operations may precede root connection, but only after all roots have completed claim-scoped preparation.

Roots connect the prepared resource or provide a checked no-op for access-only resources. Derived steps use typed input handles, create or modify only declared objects, and return named outputs. Root ConnectNetwork does not repeat claim-scoped preparation.

For example, an SR-IOV root moves the assigned VF into the sandbox; a macvlan root keeps its parent on the host and creates one child covered by the allocated capacity. A VFIO root may only confirm that prepared userspace access belongs to this authorized attachment and return an access handle. None is forced to fabricate a kernel interface.

The ordinary attachment becomes Ready only after all required steps and their success records are durable, all required IP/MAC components belong to the same authorized endpoint bundle, and required final invariants hold. A migration-only staged connection reports Staged, never ordinary readiness; it requires the explicitly certified adapter described in Section 14. Ready records completion of the attachment contract; it is not a permanent guarantee of link health or external fabric reachability. Continuous health reporting and drift detection are separate from initial readiness, and recovery must not overwrite a live external writer. No new CDI requirement may first appear in a derived connect step: container access that kubelet needs must be known and prepared through roots.

4.3 RemoveNetwork: reverse the attachment, not the claim

The normal trigger is NRI RemovePodSandbox. The same hook is also used to compensate a failed connect, resume cleanup after restart, or handle a missed event once the coordinator has authoritative evidence that the sandbox is no longer usable.

Removal first fences the attachment against further connect work. It then cleans the failed or uncertain step and its dependencies in safe reverse graph order. A successful response means the step’s attachment effects are removed or verified absent, not merely that cleanup was queued. For IPAM/MAC this means releasing the target holder after its consumers are gone; retaining the stable reservation under policy is a valid completed cleanup, not a leak.

A root removal reverses sandbox attachment: it can return a VF to its prepared host state, delete a per-attachment macvlan, or remove an attachment-scoped access association. It does not free the DRA allocation, delete claim-scoped CDI state, or undo a still-needed driver binding.

The namespace may be missing. Plugins must support recovery from recorded ownership and observed state without treating an empty namespace as permission to operate on the host by default.

4.4 NodeUnprepare: roots only, after safe attachment cleanup

The node agent checks for active or uncertain attachments before dispatching root NodeUnprepare. If a sandbox may still be running, preparation remains held and the call returns a retryable per-claim failure. If the runtime proves it stopped, the agent may finish the same journaled RemoveNetwork workflow first, including when the NRI removal event was missed.

Once attachment cleanup is complete, roots undo their preparation effects and release their local preparation references. The agent retires CDI state only when no relevant container or attachment can use it. An unprepare acknowledgment is not a write to scheduler allocation state and is not evidence that unrelated allocations have ended.

Do not block forever waiting for a future RemovePodSandbox callback: runtime and kubelet teardown may have different ordering. Reconcile observable runtime state, return bounded retryable errors, and retain the journal until safe.

4.5 Hook support matrix

An accidental prepare/unprepare call to a derived-only plugin returns FAILED_PRECONDITION. This catches executor bugs rather than silently accepting an invalid phase.

5. Typed DAG and Result Model

5.1 Explicit step kinds and independent reservation producers

kind: Allocation identifies a step requiring one DRA scheduler allocation. It receives root-only preparation hooks. Its locked contract may permit Connect-only reservation inputs, such as a MAC or IP lease; it cannot depend on a derived interface or require Pod metadata during NodePrepare.

kind: Operation identifies an authorized operation without a new DRA device allocation. Most operations consume outputs of allocated roots or their descendants. A contract with operationRole: Reservation may instead produce a lease without an interface input. It declares authorizationRoots: [<root-step>, ...] and must feed an authorized consumer in the same topology.

inputs.<port>.from creates a data/execution edge automatically. Optional dependOn creates an ordering-only edge. authorizationRoots does not create an execution edge: an IPAM step can be authorized by a macvlan allocation and supply the macvlan’s connection input without a graph cycle. It still cannot run until all roots are prepared and its consumer/context are authorized.

Only Allocation steps generate DeviceClasses. An Operation with no inputs is not automatically an allocation. Validate the entire graph: every resource effect needs root authority, independent providers must be approved reservation roles, and unused or unauthorized reservation outputs are rejected. Compile downstream hardware requirements into root preparation independently of later lease values.

5.2 Named input and output ports

# Project step fragment: two typed interface inputs, not merged previous results.
name: bond0
kind: Operation
pluginRef:
  name: bond
  contractVersion: v1alpha1
inputs:
  members:
    - from: {step: vf0, output: interface}
    - from: {step: vf1, output: interface}
interfaceName: bond0
config:
  mode: active-backup
  monitorIntervalMs: 100
  mtu: 9000

Ports have declared type and cardinality. References resolve complete typed values; arbitrary string interpolation is not supported. Interface names and PCI addresses shown in observations are descriptive properties, not bearer authority.

The plugin receives named input bindings independently. There is no prevResult, inherited interface-array ordering, index rewriting, or automatic concatenation. A diamond graph reuses the same stable resource handle rather than cloning its ancestors into each result.

5.3 Core resource types

Plugins may add versioned types through approved contracts. The executor is generic over these types, but must know their authority and identity rules. A descriptor that says merely “arbitrary JSON in/out” is insufficient for the baseline.

A link description includes logical handle ID, namespace identity, current name, observed ifindex, MAC, parent relationship when applicable, attachment identity, and allocation-root provenance. An ifindex is an observation, not a permanent identity across namespace moves or link recreation.

5.4 Ownership and mutation access

An output declares whether it creates a resource or aliases an input after mutation. A tuning step returns the same interface handle ID with updated observations; it does not claim a new interface. Its undo record records only the settings it changed. A bond returns its own bond handle, with references to its member handles.

Every destructive or mutating input has an access mode in the plugin contract, such as Read, CreateChild, ConfigureLink, or AttachMember. The compiler and node coordinator reject simultaneous incompatible writers. Returning a handle does not transfer unlimited ownership of all ancestors. Address/MAC ports declare UseLease; their consumer may apply the authorized value to the designated endpoint but cannot release the reservation. Holder handles are attachment-specific and cannot be copied from a source to a migration destination.

Reject duplicate physical bond members, a link attached to two bonds, a VFIO access handle wired into a kernel VLAN, cycles, missing ports, incompatible cardinality, and a reference to an unrelated topology instance. Removing a downstream mutation must not restore an old value over a newer legitimate writer; the initial executor serializes such writers and rejects overlapping external ownership.

5.5 Roots cover effects, not just graph reachability

Derived operations are not assumed free. The compiler derives an effects budget for each allocated root. Any host-side child creation, shared-parent use, hardware context, or other constrained quantity must be covered by the root’s reserved capacity and permitted mode.

One allocated macvlan unit authorizes one child in the initial profile, not an arbitrary number of derived children. A macvlan persona cannot authorize an ipvlan operation on that host parent. Fan-out whose quantitative or compatibility effects cannot be represented is rejected, partitioned into additional roots, or deferred to an explicitly supported feature profile.

A valid graph is therefore one with valid types, ownership, and resource effects—not merely a graph with no cycles.

6. Plugin Contracts and gRPC Transport

6.1 Registry and immutable plugin lock

A NetworkPlugin resource describes an administrator-approved implementation, its artifact, contract schema, supported step kinds, input/output ports, effects, and state-format compatibility. The topology controller resolves a friendly pluginRef to immutable artifact and contract digests when it compiles a revision. Node endpoints are obtained from trusted local registration, never from tenant configuration.

# Proposed registry shape; the artifact placeholder is not a published image.
apiVersion: networking.dra.io/v1alpha1
kind: NetworkPlugin
metadata:
  name: bond
spec:
  contractVersion: v1alpha1
  artifact:
    image: registry.example.com/networkplugins/bond@sha256:REPLACE_WITH_REAL_DIGEST
  stepKinds: [Operation]
  inputs:
    members:
      type: KernelInterface/v1
      minItems: 2
      maxItems: 8
      access: AttachMember
  outputs:
    interface:
      type: KernelInterface/v1
      cardinality: 1
      ownership: Created
  configSchema:
    type: object
    additionalProperties: false
    required: [mode, monitorIntervalMs, mtu]
    properties:
      mode: {type: string, enum: [active-backup, "802.3ad"]}
      monitorIntervalMs: {type: integer, minimum: 1}
      lacpRate: {type: string, enum: [slow, fast]}
      xmitHashPolicy: {type: string, enum: [layer2, layer2+3, layer3+4]}
      mtu: {type: integer, minimum: 68, maximum: 65535}
  stateFormat:
    writeVersion: "1"
    readableVersions: ["1"]

The eight-member bound is a project support limit, not a claimed Linux kernel maximum. Cross-field rules reject LACP-only options for active-backup mode. Hardware/plugin-specific MTU and offload limitations are validated in addition to generic bounds. Plugin descriptors are mandatory; an unknown plugin does not bypass validation.

6.2 Deliberately new configuration contracts

The new bond plugin accepts integer monitorIntervalMs and implements the declared LACP controls. The VLAN plugin remains layer-2-only when no lease ports are connected. Its v1alpha2 contract optionally accepts addresses: AddressLease/v2 and mac: MACAddressLease/v2, applying them through shared SDK effects. Independent IPAM and MAC plugins own allocation; VLAN never invokes them implicitly. A separate address operation is an alternative application path, not a second owner of the same addresses. Tuning remains limited to its declared settings.

These are contracts to implement and test, not claims that unchanged CNI binaries accept these fields. Ports of existing implementations must preserve licensing and notices and replace process/environment assumptions, result accumulation, and cleanup handling with the protocol here. Shelling out to a legacy binary may be a transitional adapter, but the adapter—not the central DAG engine—must own all ABI conversion and recovery guarantees.

6.3 Transport and state ownership

Use persistent local gRPC connections over Unix-domain sockets. Authenticate the node coordinator and registered plugin identity; restrict socket directories and disallow arbitrary filesystem paths, symlink traversal, or user-selected endpoints. Pin the artifact and contract digests, and perform bounded health/identity checks before dispatch.

Config is canonical, schema-validated literal JSON in a versioned envelope. Pod labels and annotations arrive only in a separately validated, projected WorkloadContext; template substitution is not an execution feature. Resource handles have validated typed descriptions and provenance. The plugin’s opaque state is separately versioned, bounded, and stored as sensitive node-local data. It must not be treated as an executable script or echoed into workload-visible status.

A plugin and the central agent each persist their own part of a transaction: the coordinator’s intent/result journal and the plugin’s durable side-effect/undo journal. A successful RPC response must be reproducible after a plugin restart. Lost replies must be recoverable by operation ID, including when no opaque state reached the caller.

6.4 Exactly four mutating RPCs

service NetworkPlugin {
  rpc NodePrepare(NodePrepareRequest) returns (NodePrepareResponse);
  rpc NodeUnprepare(NodeUnprepareRequest) returns (CleanupResponse);
  rpc ConnectNetwork(ConnectNetworkRequest) returns (ConnectNetworkResponse);
  rpc RemoveNetwork(RemoveNetworkRequest) returns (CleanupResponse);
}

The canonical draft message definitions are checked in at proto/networkplugin.proto; Appendix A is its generated Markdown mirror. Run make network-docs-check to verify the mirror and generate/test protocol bindings, or make network-docs-sync after editing the canonical source. Generation instructions and limits are in tools/network-docs/README.md. The inspection service has only GetInfo and InspectOperation; it cannot mutate networking and is not a fifth lifecycle hook. Appendix A appends context and intent fields while preserving existing field numbers. Changed IPAM/MAC/consumer semantics use locked v1alpha2 plugin contracts and explicit capabilities; the protobuf draft package alone is not a compatibility guarantee. An old endpoint that silently ignores new fields must be rejected before use. Standard health checking may also be exposed.

6.5 RPC execution rules

Use explicit deadlines, bounded retry with jitter, and admission limits per plugin. A gRPC deadline does not undo a completed kernel operation, and cancellation requires server-side cooperation. Transport errors therefore leave the effect state uncertain unless the plugin durably proves otherwise. U8

Do not enable hedged concurrent mutation calls. Repeat the same operation ID and semantic digest when retrying; the plugin joins or reports the same operation rather than running it twice. A different semantic payload under the same ID is INVALID_ARGUMENT.

Suggested error classes are INVALID_ARGUMENT for contract violations, PERMISSION_DENIED for invalid authority, FAILED_PRECONDITION for incompatible state or stale prerequisites, RESOURCE_EXHAUSTED for a verified exhausted plugin/backend resource, and UNAVAILABLE/DEADLINE_EXCEEDED for retryable transport conditions. A status code alone is never proof that no side effect occurred.

CleanupResponse with gRPC OK means cleanup has completed. A plugin with deferred or uncertain work returns an error with recovery details and keeps its journal. The coordinator never interprets “request accepted” as “resource free.”

7. NetworkTopology Compilation and Revisioning

7.1 Schema rules

A topology declares runtimeProfile, attachmentPolicy, steps, optional execution limits, workloadContext projection, identityBindings, and exported named outputs. Identity bindings and all allowed metadata-driven choices are administrator-approved parts of the immutable revision. Each step has an explicit kind, plugin reference, strict config, and named inputs. Allocation steps additionally declare sharing expectations and a device selector. Reservation Operation steps declare authorizationRoots; supported root reservation inputs are resolved at Connect time only. Optional capacityRequests are compile-time requirements that generated claim examples and admission must enforce; they are not a hidden DeviceClass allocation mechanism.

The compiler validates all plugin schemas and input types before generating classes. It derives edges from input references, adds dependOn edges, computes the closure of required plugins and effects, and produces a deterministic execution and reverse-cleanup plan.

7.2 Immutable revisions

Changing NetworkTopology.spec creates a new NetworkTopologyRevision; it does not alter old plans. A revision includes the normalized topology, compiler version, plugin artifact/contract lock, exported ports, context projection/identity-policy definitions, resolved network/pool policy identities and full content digest. Per-Pod metadata and resolved subject identities are captured separately at runtime; they do not mutate that revision. It is immutable by admission and authorization, and the node verifies its digest before use.

Generated DeviceClass names include a revision identifier. Their selectors and class-origin root binding are immutable by project policy. A revision reference includes its name, Kubernetes UID, and full content digest. The name alone is not sufficient, and a generation number without retained content cannot reconstruct an old plan.

Hash the normalized topology, compiler version, and plugin lock. Exclude generated class names, Kubernetes metadata/status, and the digest field itself. Define canonical JSON serialization and version the canonicalization algorithm. Short names/labels are collision-checked against the full digest; two different digests must never silently share a readiness label or revision name.

7.3 Retention and garbage collection

Do not delete an old revision or plugin artifact solely because no Pod is currently running. A pending claim can still reference an old class, an allocated claim can be prepared later, and a node can still need old cleanup code.

Retain revisions while any relevant DeviceClass, pending or allocated claim, claim template selected for continued use, node preparation, or cleanup journal can reference them. The initial policy favors explicit retirement and administrator-managed retention over aggressive garbage collection. Garbage collection must distinguish safe retirement from ordinary object-owner cascading deletion.

A driver checkpoint contains enough resolved plan and configuration to finish an authorized teardown without fetching the newest topology or DeviceClass. Unknown or corrupted state fails closed and requires recovery, not latest-spec substitution.

7.4 Whole-plan node readiness

A node with an SR-IOV root plugin but without the bond or address plugin cannot execute the plan. Root supportedNetworkPlugins attributes do not establish readiness of every derived operation.

The initial controller publishes a trusted plan-revision readiness label on nodes only after verifying all required plugin revisions, context/lease capabilities, identity-backend access, runtime integration, feature profile, and local prerequisites. Backend reachability does not guarantee future free addresses. Pod admission injects or validates required node affinity for that label. Required affinity must be conjoined with every existing OR term, not added as another alternative term. The node agent rechecks readiness before preparation and connection.

The sample label is networking.dra.io/plan-<digest-prefix>=ready. It is a proposed administrative contract, not a Kubernetes-protected label by default. Restrict node-label writes and use an appropriate protected namespace/admission policy in the deployment. Losing readiness prevents new placement; it does not erase active device declarations or automatically evict a running Pod.

Do not create duplicate hardware devices per topology just to encode software readiness: that risks advertising the same exclusive device or shared capacity more than once. A future scheduler integration may replace the label scheme if revision cardinality becomes a scaling problem. Neither labels nor an API schema replace actual runtime conformance tests.

8. Full Example: Bonded Ethernet with Two VLAN Networks

The original VF → bond → VLAN → tuning example is retained and extended with independent IPAM and MAC providers. This is an Ethernet topology, not a claim of bonded RDMA or working VM migration.

vf0 --\                    +--> data-vlan --> tune-data --> export data
        +--> bond0 ----------+       ^    ^
vf1 --/                     |       |    +---- mac-data (lease)
                            |       +--------- ipam-data (lease)
                            |
                            +--> mgmt-vlan --> tune-mgmt --> export management
                                    ^    ^
                                    |    +---- mac-mgmt (lease)
                                    +--------- ipam-mgmt (lease)

Reservation steps can execute independently of link creation after all allocated roots are prepared. Their authorizationRoots are not arrows in this DAG. Each VLAN consumes parent, IP lease and MAC lease through separate typed ports. The primary network and default route remain preserved.

8.1 NetworkTopology

The node/fabric must support the selected Ethernet bond mode. LACP requires corresponding switch configuration. This explicit administrator-approved example permits VF trust and disables source-MAC spoof checking so VLAN endpoint MACs can differ from VF MACs; it is not a recommended default or sufficient validation for every NIC. A tested hardware enforcement profile must authorize only the intended traffic, and unsupported settings must fail rather than be silently relaxed. Two PF names do not prove independent card failure domains. MTU is propagated from each child to the owned parent path without changing an unallocated host parent.

ai-data and ai-management resolve to approved network definitions; the IP and MAC pools under each belong to that network’s declared uniqueness domains. These are project/backend references, not built-in objects. The controller pins their identities in the revision. The initial PodUID policy preserves values across replacement sandboxes of the same Pod; Section 14 shows other continuity strategies.

apiVersion: networking.dra.io/v1alpha1
kind: NetworkTopology
metadata:
  name: bonded-secondary
spec:
  runtimeProfile: nri-secondary-v1
  attachmentPolicy: SinglePod
  execution:
    maxParallelSteps: 1
    primaryNetworkPolicy: Preserve
  steps:
  - name: vf0
    kind: Allocation
    pluginRef:
      name: sriov
      contractVersion: v1alpha1
    allocation:
      sharing: Exclusive
      selector:
        cel: device.attributes["dra.networking"].type == "vf" && device.attributes["dra.networking"].pfName == "enp3s0f0"
    interfaceName: net1
    config:
      mtu: 9000
      vf:
        vlan: 0
        spoofCheck: false
        trust: true
      rdmaAccess: false
  - name: vf1
    kind: Allocation
    pluginRef:
      name: sriov
      contractVersion: v1alpha1
    allocation:
      sharing: Exclusive
      selector:
        cel: device.attributes["dra.networking"].type == "vf" && device.attributes["dra.networking"].pfName == "enp3s0f1"
    interfaceName: net2
    config:
      mtu: 9000
      vf:
        vlan: 0
        spoofCheck: false
        trust: true
      rdmaAccess: false
  - name: bond0
    kind: Operation
    pluginRef:
      name: bond
      contractVersion: v1alpha1
    inputs:
      members:
      - from:
          step: vf0
          output: interface
      - from:
          step: vf1
          output: interface
    interfaceName: bond0
    config:
      mode: 802.3ad
      monitorIntervalMs: 100
      lacpRate: fast
      xmitHashPolicy: layer3+4
      mtu: 9000
  - name: ipam-data
    kind: Operation
    pluginRef:
      name: ipam
      contractVersion: v1alpha2
    authorizationRoots:
    - vf0
    - vf1
    config:
      identityBinding: data
      poolRef: ai-data-v4
      addressFamilies:
      - IPv4
  - name: mac-data
    kind: Operation
    pluginRef:
      name: mac
      contractVersion: v1alpha2
    authorizationRoots:
    - vf0
    - vf1
    config:
      identityBinding: data
      poolRef: ai-data-macs
  - name: data-vlan
    kind: Operation
    pluginRef:
      name: vlan
      contractVersion: v1alpha2
    inputs:
      parent:
        from:
          step: bond0
          output: interface
      addresses:
        from:
          step: ipam-data
          output: lease
      mac:
        from:
          step: mac-data
          output: lease
    interfaceName: data0
    config:
      vlanId: 100
      mtu: 9000
      addressApplication: PodInterface
      defaultRoutePolicy: Preserve
  - name: tune-data
    kind: Operation
    pluginRef:
      name: tuning
      contractVersion: v1alpha1
    inputs:
      interface:
        from:
          step: data-vlan
          output: interface
    config:
      mtu: 9000
  - name: ipam-mgmt
    kind: Operation
    pluginRef:
      name: ipam
      contractVersion: v1alpha2
    authorizationRoots:
    - vf0
    - vf1
    config:
      identityBinding: management
      poolRef: ai-mgmt-v4
      addressFamilies:
      - IPv4
  - name: mac-mgmt
    kind: Operation
    pluginRef:
      name: mac
      contractVersion: v1alpha2
    authorizationRoots:
    - vf0
    - vf1
    config:
      identityBinding: management
      poolRef: ai-mgmt-macs
  - name: mgmt-vlan
    kind: Operation
    pluginRef:
      name: vlan
      contractVersion: v1alpha2
    inputs:
      parent:
        from:
          step: bond0
          output: interface
      addresses:
        from:
          step: ipam-mgmt
          output: lease
      mac:
        from:
          step: mac-mgmt
          output: lease
    interfaceName: mgmt0
    config:
      vlanId: 200
      mtu: 1500
      addressApplication: PodInterface
      defaultRoutePolicy: Preserve
  - name: tune-mgmt
    kind: Operation
    pluginRef:
      name: tuning
      contractVersion: v1alpha1
    inputs:
      interface:
        from:
          step: mgmt-vlan
          output: interface
    config:
      mtu: 1500
  exports:
    data:
      from:
        step: tune-data
        output: interface
    management:
      from:
        step: tune-mgmt
        output: interface
  workloadContext:
    pod:
      labels:
      - app.kubernetes.io/name
      annotations:
      - networking.example.com/address-profile
    resolveOwners:
    - ReplicaSet
    - Deployment
    - StatefulSet
  identityBindings:
  - name: data
    networkRef:
      name: ai-data
    interfaceKey: data0
    subject:
      strategy: PodUID
    retention:
      mode: SubjectLifetime
    activation: Exclusive
  - name: management
    networkRef:
      name: ai-management
    interfaceKey: mgmt0
    subject:
      strategy: PodUID
    retention:
      mode: SubjectLifetime
    activation: Exclusive

Both VLAN steps consume leases directly. The optional tuning steps still alias their input interface; they do not allocate an address or return a newly owned link. No concrete IP or MAC is assigned in this reusable topology.

8.2 Generated DeviceClasses

The following generated classes show a synthetic revision used only for this document’s fixtures. UID and artifact values are illustrative, not references to installed resources. A live controller substitutes the created revision’s actual UID and digests for built, approved artifacts. These are new synthetic revision values for this extension, illustrating binding consistency only. They are not a calculated digest of a complete compiled revision; a complete built-artifact revision is not supplied.

apiVersion: resource.k8s.io/v1
kind: DeviceClass
metadata:
  name: bonded-secondary-c52c45f5d34c-vf0
spec:
  selectors:
  - cel:
      expression: device.driver == "dra.networking" && device.allowMultipleAllocations == false && (device.attributes["dra.networking"].type
        == "vf" && device.attributes["dra.networking"].pfName == "enp3s0f0") && device.attributes["dra.networking"].supportedNetworkPlugins.includes("sriov")
  config:
  - opaque:
      driver: dra.networking
      parameters:
        apiVersion: networking.dra.io/v1alpha1
        kind: TopologyRootConfiguration
        revisionRef:
          name: bonded-secondary-c52c45f5d34c
          uid: 22222222-2222-4222-8222-222222222222
          planDigest: sha256:c52c45f5d34cfe1bbbf7e211660c9043f268056ac0c12507c4ffb8535c22dfda
        rootStep: vf0
        instancePolicy: SinglePod
---
apiVersion: resource.k8s.io/v1
kind: DeviceClass
metadata:
  name: bonded-secondary-c52c45f5d34c-vf1
spec:
  selectors:
  - cel:
      expression: device.driver == "dra.networking" && device.allowMultipleAllocations == false && (device.attributes["dra.networking"].type
        == "vf" && device.attributes["dra.networking"].pfName == "enp3s0f1") && device.attributes["dra.networking"].supportedNetworkPlugins.includes("sriov")
  config:
  - opaque:
      driver: dra.networking
      parameters:
        apiVersion: networking.dra.io/v1alpha1
        kind: TopologyRootConfiguration
        revisionRef:
          name: bonded-secondary-c52c45f5d34c
          uid: 22222222-2222-4222-8222-222222222222
          planDigest: sha256:c52c45f5d34cfe1bbbf7e211660c9043f268056ac0c12507c4ffb8535c22dfda
        rootStep: vf1
        instancePolicy: SinglePod

Class-origin configuration maps the selected device to rootStep; the claim’s request name need not equal that step name. Both generated selectors enforce the root plugin and sharing contract in addition to the topology’s device selector.

8.3 Application ResourceClaimTemplate

apiVersion: resource.k8s.io/v1
kind: ResourceClaimTemplate
metadata:
  name: bonded-network
  namespace: default
spec:
  spec:
    devices:
      requests:
      - name: primary-nic
        exactly:
          deviceClassName: bonded-secondary-c52c45f5d34c-vf0
          allocationMode: ExactCount
          count: 1
      - name: backup-nic
        exactly:
          deviceClassName: bonded-secondary-c52c45f5d34c-vf1
          allocationMode: ExactCount
          count: 1

The intentionally different request names, primary-nic and backup-nic, demonstrate that mapping does not depend on naming coincidence. The exclusive roots need no local capacity request. A shared-root example appears in Section 16.

The same claim can include resources managed by another DRA driver. This example requests the two VFs plus a GPU and constrains all three devices to the same PCIe root:

apiVersion: resource.k8s.io/v1
kind: ResourceClaimTemplate
metadata:
  name: bonded-network-and-gpu
  namespace: default
spec:
  spec:
    devices:
      requests:
      - name: primary-nic
        exactly:
          deviceClassName: bonded-secondary-c52c45f5d34c-vf0
          allocationMode: ExactCount
          count: 1
      - name: backup-nic
        exactly:
          deviceClassName: bonded-secondary-c52c45f5d34c-vf1
          allocationMode: ExactCount
          count: 1
      - name: gpu
        exactly:
          deviceClassName: site-gpu-class
          allocationMode: ExactCount
          count: 1
      constraints:
      - requests:
        - primary-nic
        - backup-nic
        - gpu
        matchAttribute: device.k8s.io/pcieRoot

The network topology still owns only the two VF roots; the GPU DRA driver owns GPU preparation and device access. The scheduler applies the cross-driver constraint because all three requests are part of one claim. site-gpu-class is a placeholder for an administrator-provided GPU DeviceClass. Use this constraint only when every participating driver publishes device.k8s.io/pcieRoot with interoperable values. Matching a PCIe root improves locality, but does not by itself guarantee GPUDirect RDMA support or performance.

8.4 Pod and required runtime participation

This shows the post-admission runtime requirement and affinity. A production admission controller injects and protects these fields for every Pod that consumes this topology, including generated claims. The image is only a placeholder to demonstrate the Pod/claim relationship; it is not a recommended application image.

apiVersion: v1
kind: Pod
metadata:
  name: bonded-workload
  namespace: default
  labels:
    app.kubernetes.io/name: bonded-workload
  annotations:
    required-plugins.noderesource.dev: '["network-dra"]'
    networking.example.com/address-profile: standard
spec:
  affinity:
    nodeAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
        nodeSelectorTerms:
        - matchExpressions:
          - key: networking.dra.io/plan-c52c45f5d34c
            operator: In
            values:
            - ready
  resourceClaims:
  - name: network
    resourceClaimTemplateName: bonded-network
  containers:
  - name: workload
    image: registry.k8s.io/pause:3.10
    resources:
      claims:
      - name: network

NRI documents the required-plugins.noderesource.dev annotation for required plugin participation. Its behavior depends on the runtime’s enabled validator and supported integration; the annotation alone is not a fail-closed guarantee. U5

8.5 Runtime walkthrough

1. Kubernetes allocates the two roots. Preparation resolves the pinned plan, validates all plugin and identity-backend prerequisites, and dispatches NodePrepare to the VF roots only.
2. The sandbox adapter verifies the Pod/claim association, pins the namespace, snapshots approved labels/annotations, resolves the two network identity bindings and persists the context digest.
3. IPAM and MAC providers acquire distinct per-binding IP/MAC components and attachment holders. Retries reuse the same identities. The roots connect and the bond is built; provider calls may also be scheduled before those Connect calls.
4. For each branch, VLAN validates the complete lease bundle and parent, creates its child, applies the approved MAC and IP configuration, and permits traffic only for the active holder. Tuning changes only its declared settings.
5. The agent records Ready only after all required effects and results are durable. Every affected init/application container passes the mandatory runtime readiness gate.
6. Teardown removes tuning and VLAN effects before IPAM/MAC holders are released. The bond is removed after both VLANs; VF roots detach only after their consumers are gone. Backend reservations may remain retained under the subject policy.
7. NodeUnprepare reverses root preparation once all attachment/device-access cleanup permits it. It does not erase a workload’s retained IP/MAC identity.

Scheduling does not reserve external pool space. IPAM/MAC exhaustion or unavailable identity services fail the affected step without secretly changing the allocated devices. A complete staged-migration flow is a different, capability-gated runtime profile described in Section 14.

9. Claim Resolution, Admission, and Configuration Trust

9.1 Correct kubelet boundary

NodePrepareResources carries claim identifiers. The driver obtains the current ResourceClaim, verifies the requested UID, and uses status.allocation.devices.results and status.allocation.devices.config. It does not pretend the wire request includes an AllocationResult or Pod UID. Each requested claim receives a per-claim result or error following the upstream protocol. U1

The driver filters results and config for its own driver domain. Use upstream resourceclaim.ConfigForResult or equivalent tested request/subrequest applicability logic, then apply project policy for trusted configuration sources. Do not process every opaque entry as though it applies to every allocated device. U4

9.2 Trusted binding versus user parameters

Only generated, class-origin TopologyRootConfiguration may choose the revision and root. Claim-origin config can supply only explicitly permitted typed parameters defined by that revision. It cannot override the plugin implementation, host device, preparation policy, namespace, root identity, or topology revision. Upstream allocation config distinguishes FromClass and FromClaim; preserve that distinction. U3

All roots of an instance must resolve to the same revision UID/digest and instance policy. Reject multiple different root bindings for one allocation, missing or duplicate roots, an incompatible device persona, and an unexpected extra result managed by this driver.

9.3 Admission covers clean claims and templates

The normal user claim has no inline topology reference. Admission therefore examines the referenced generated DeviceClasses and their protected root bindings. Inspect inline config too, but do not use it as the only trigger.

For the baseline, require one ExactCount: 1 device request per network root, reject All and firstAvailable for topology-bound roots, and reject duplicate topology instances in one claim. Administrative-access claims and allocation results that skip required node preparation/unpreparation are outside this profile and must not bypass the lifecycle checks. Validate mandatory capacity requests and required claim constraints where a topology profile defines them. Other drivers’ requests are allowed and left to their own semantics.

Pod admission resolves ResourceClaimTemplate references or directly referenced claims, enforces the single-consumer profile, runtime requirement, and plan readiness affinity, and rejects unsupported bypasses such as hostNetwork, arbitrary nodeName placement, or an unapproved scheduler for this profile. It protects required-plugin annotations against removal and disallows tenant use of runtime bypass annotations.

For identity-aware plans, validate context projections, allowed subject strategies, network/pool authorization, reservation-provider roles, all required lease types and retention compatibility. ReplicaSet singleton/slot modes and VM identities require a trusted resolver, not just a matching Pod label. A KubeVirt target must be associated with an authorized migration and its own claim.

Admission is for early feedback and policy enforcement. The node independently repeats security-sensitive validation; cache races, mutable external state, and in-flight scheduler activity do not disappear because a webhook ran.

9.4 Availability and outages

A missing revision, unavailable relevant admission dependency, or unknown binding prevents new work rather than falling back to executing raw configuration. Scope admission carefully so the failure of this networking service does not block unrelated workloads with no reference to it.

Prepared, already-authorized work may use its verified cached revision when the API server is unavailable, subject to the installation’s authorization policy. Starting a new attachment without enough evidence to verify its consumer is not permitted. Cleanup must not depend on fetching a live claim that may already be deleted.

10. Preparation, Attachment, and Cleanup State Machines

10.1 Durable states

Claim/root preparation:
  Unprepared -> Preparing -> Prepared -> Unpreparing -> Unprepared
                   |                         |
                   +------> RecoveryRequired <+

Sandbox attachment:
  Absent -> Connecting -> Ready -> Removing -> Removed
                 |                    |
                 +-> CleanupPending <-+
                          |
                          +-> Removed or Quarantined

The optional certified handoff path adds Connecting(Stage) -> Staged -> Connecting(Activate) -> Ready; it is not available to ordinary legacy plugins. All generations share one attachment cleanup scope.

The absent/removed states retain tombstones long enough to reject delayed operations. A new preparation or attachment after complete teardown gets a new incarnation identity. State transitions do not erase historical evidence before cleanup is proven.

10.2 Central journal contents

Before any side effect, persist claim/allocation identity, pinned publication contract, revision digest and resolved plan, plugin lock, root bindings, expected inputs and grants, assigned interface names, namespace identity, effective WorkloadContext and digest, resolved identity/pool contracts, operation ID, desired generation, and intent state.

Persist exact preparation and connection responses separately. Record partial/uncertain calls, undo dependencies, cleanup receipts, last error classification, attempts, and ownership fences. API-visible status is a sanitized projection, not this source of truth.

The store must survive agent/container restart on local durable storage. Atomic record updates, durability barriers, corruption detection, schema migrations, and bounded storage are part of the implementation. An in-memory map plus periodic checkpoint is insufficient for the side-effect boundary.

10.3 Plugin write-ahead journal

A plugin persists its own intent and undo information before changing kernel or backend state. It records successful completion before returning. It must recover an operation by scope/operation ID even when the coordinator never received a cleanup token.

For a link move, record the stable device identity and original/target namespaces before moving it. For a newly created object, use a collision-resistant deterministic ownership identity and validate any preexisting object before adopting it. For IPAM/MAC, the external backend atomically reserves the component for the stable network identity and adds the exact attachment holder. The local journal saves its idempotency key before the call and its receipt afterward. A lost response is recovered by that key, not by allocating another address. Stable reservation, holder and activation state are separate; a node journal alone cannot enforce cross-node uniqueness.

The coordinator’s journal alone does not close the window between a plugin performing a mutation and returning its response. Both sides participate in recovery.

10.4 Fencing and timeouts

There is one active coordinator per node, protected by local process ownership and a persisted coordinator epoch. Plugins serialize conflicting operations and validate that epoch. A higher epoch fences old clients but is not by itself proof that an old in-flight operation stopped; takeover waits for, cancels and reconciles, or quarantines those operations before conflicting new work is permitted.

Within a preparation or attachment-step scope, desired generations are monotonic. Creation at generation 1 and removal at generation 2 cannot be reordered into resurrection: once removal intent is durably accepted, a delayed generation-1 create is rejected. Cleanup joins or fences the older worker under the same resource lock. A new retry attempt cannot lower the generation.

Do not run RemoveNetwork concurrently against an unresolved ConnectNetwork on the same resource unless the plugin’s state machine explicitly coordinates that transition. gRPC cancellation is a signal, not a distributed lock. U8

Semantic digests cover hook, scope, desired generation, plan and plugin lock, allocation, resolved config, input identities, and assigned outputs. They exclude transport deadlines, trace IDs, refreshed authorization tokens, and coordinator epoch changes that do not change the intended effect. Namespace identity participates; a remounted path for the same verified namespace is a locator update, not permission to target another namespace.

10.5 Failure and rollback rules

Rollback is a saga of compensating operations. A failed topology may leave a temporarily reserved device or quarantined lease until cleanup finishes. Prefer a visible unavailable resource to silent reuse with uncertain ownership.

11. Namespace Lifetime and Runtime Integration

11.1 Sandbox creation contract

The runtime adapter must demonstrate that its create event supplies an accessible network namespace, that primary-network work has reached the agreed phase, and that affected application containers cannot run before the topology readiness gate. Do not equate an event name with a universal ordering guarantee.

NRI provides sandbox and container lifecycle integration. The exact runtime build and adapter must be tested for namespace paths, callback errors, retries, handler subscription, primary network ordering, and behavior after reconnect. The inspected containerd adapter can issue stop/remove callbacks after a failed sandbox-create callback, which makes duplicate/partial cleanup normal cases to handle. U6

11.2 Why a persistent namespace handle is required

The inspected containerd code explicitly handles the fact that the normal sandbox network namespace can be torn down before RemovePodSandbox. Therefore removal must not rely on a runtime-provided path remaining valid. Linux permits a bind mount or open descriptor to keep a namespace alive. U7 U9

Before any attachment mutation, the node agent creates a driver-owned bind-mounted handle to the verified sandbox namespace and journals its identity. The bind mount must live in a mount namespace with the required lifetime and be visible at an approved path to the relevant plugins; a mount private to a disposable agent container is not a persistent host handle. Installation must test mount propagation, SELinux/user-namespace constraints, and restart behavior.

A descriptor held only by a process is useful during a call but does not survive that process dying. A bind-mounted handle can survive a process restart within the same host boot. It does not survive a node reboot as a usable reference to the old namespace.

gRPC does not transfer Linux file descriptors by sending their integer values. The baseline passes a verified driver-issued handle and approved shared path; plugins open and verify the namespace themselves. A separate authenticated Unix FD broker using descriptor passing is a possible later transport optimization, not another network lifecycle hook.

Record node boot ID, nsfs device/inode, Pod UID, and sandbox ID. Reject stale or symlink-substituted targets, reject the host namespace for ordinary workloads, and never identify a namespace by inode alone across reboots.

11.3 Removal timing and pin ownership

Keep the namespace pinned until all attachment effects that need it are removed and address ownership is safe to release. Then remove the driver pin. Failure to remove the pin is observable cleanup debt, not a reason to discard its ownership record.

A pin extends namespace lifetime; it does not preserve objects the primary network explicitly deletes, nor prove a runtime will accept the integration. Plugins must tolerate externally removed primary-network objects without removing unowned state. Managed physical devices are tracked by identity if they are returned to the host by namespace destruction.

If removal is delayed and the namespace still exists only through the pin, the agent must still know that no workload can use it before releasing leases or moving devices. A missing Pod API object is not sufficient evidence of runtime quiescence.

11.4 Fail-closed container readiness

Successful DRA preparation does not prove the NRI adapter participated. The deployment requires a runtime mechanism that rejects container creation when network-dra was not consulted, and the consulted handler checks durable Ready state for the current sandbox and all required network instances. This applies to init and application containers, including replacement containers. U5

The handler belongs to the central driver; it is not a fifth plugin hook. It does not perform a second topology connection during CreateContainer. When the topology is not ready, it returns an error and leaves the existing attachment transaction to recover or clean up.

NRI validation of OCI adjustments is not a transaction around kernel networking or IPAM. A later container-create failure requires ordinary attachment lifecycle handling, not an assumption that external network changes were rolled back automatically. Required-plugin bypass annotations are restricted to narrowly scoped platform bootstrap workloads, not tenant Pods.

11.5 Runtime profile acceptance

The initial nri-secondary-v1 profile is a certification contract, not a claim that every runtime with NRI implements it. Test create ordering, required-plugin enforcement, namespace pinning, duplicate notifications, failed sandbox creation, remove-after-namespace-deletion, runtime restart, and missed callbacks on each supported containerd/CRI-O build.

An adapter may use an earlier, verified-quiesced stop event to trigger the same RemoveNetwork workflow when required for a runtime. That is an explicitly selected profile variation, not a new plugin hook or an assumption that every stop notification means all containers are gone. The baseline normal trigger remains sandbox removal.

If a runtime cannot meet the contract, do not mark nodes ready. An upstream runtime integration change or an alternative sandbox lifecycle adapter may be necessary. Never silently replace the required hook with best-effort execution after application startup.

12. Safe Teardown and Recovery

12.1 Dependency-aware cleanup

Cleanup is reverse dependency order, not reverse YAML order. Attempted or uncertain steps are eligible without a success response. Removal fences all Connect/Activate generations for that attachment before reclaiming effects.

For the full example:

tune-data -> data-vlan --+--> release ipam-data holder
                        +--> release mac-data holder
                        +------------------------------+ 
                                                       +--> bond0 -> VF detachment
tune-mgmt -> mgmt-vlan --+------------------------------+
                        +--> release ipam-mgmt holder
                        +--> release mac-mgmt holder

After a VLAN’s addresses/MAC use and link are verifiably removed, its reservation providers and hardware ancestors may clean up independently subject to their other dependents. Releasing a holder is not deleting a retained reservation. A failed VLAN cleanup blocks both its IP/MAC release and unsafe ancestor removal. A source holder’s cleanup cannot remove the target holder or active-owner epoch after migration.

Receipt IDs must be authenticated and matched against the original attachment, holder, operation and generation records. They are not user-supplied proof. Retention/GC happens separately after all holders and uncertain uses are resolved.

12.2 Missing namespace and partial absence

A namespace may be gone after reboot, or the pin may be unavailable due to an installation failure. Each plugin reconciles its own effects:

• A VLAN/bond may be absent; do not delete an unrelated host link with the same name.
• A physical device may have returned to the initial network namespace; identify it through its tracked hardware identity and restore only owned state.
• IP/MAC reservations and holders can still exist externally. Release only the old attachment’s holder after ownership and absence/fencing of its use are established; keep subject-retained reservations and other holders.
• Backend SDN/OVS objects may outlive the namespace; remove them by their recorded object IDs and owner tags.

AlreadyAbsent is a verified outcome, not a blanket response to every lookup error. Permission denied, API unavailable, or journal corruption is not absence.

12.3 Reconnect and reboot

At startup the node agent reconciles its journal with runtime sandboxes, allocation-aware inventory, current claims where available, and plugin inspection state. It does not automatically recreate every previously ready network.

Same-boot healthy attachments can be revalidated without repeating side effects. New sandbox incarnations need new attachment identities and cannot reuse an old network namespace handle. After reboot, old kernel namespace handles are invalid; external leases and device-binding state still require recovery checks. Boot IDs distinguish observations, not authorization to free every external resource.

Unexpected plugin disappearance prevents new work and marks relevant attachments for recovery. Existing data-plane links are not deliberately removed merely because the management process is temporarily down. No plugin or coordinator TTL releases hardware or IP addresses automatically while workload use remains possible.

12.4 Discovery integration during recovery

The discovery agent retains the active publication contract while a device is allocated, prepared, attached, or involved in uncertain cleanup. A VF moved out of the host namespace or rebound to VFIO does not become a new allocatable device.

If scheduler allocation state has disappeared but cleanup is incomplete, maintain a node-local ownership/quarantine guard and prevent conflicting preparation. Publication should also communicate unavailability where supported, but a taint or asynchronous inventory update is not the only safety barrier. The node agent must reject a racing new allocation that aliases uncleared hardware.

For KEP 5963, the companion design targets an inspected implementation that builds compatibility from live published devices. Preserve group and counter-set association identity through active use; do not assume a claim snapshot already provides that protection. U10

13. Resource Accounting, Compatibility, and Host Safety

13.1 Scheduler-visible roots

Each Allocation step maps to one DRA allocation result. Shared roots use the result’s actual consumed capacity and share ID. The compiler’s effects must fit within those reservations. Plugin-local bookkeeping cannot grant more device capacity than the scheduler allocated.

Compatibility remains the companion document’s mechanism: shared counter-set associations carry compatibilityGroups; local capacity limits repeated allocations. The non-binding members counter supplies a valid association, not a per-attachment lock. Optional KEP 5941 shared budgets are a separate feature profile and are not required to execute the baseline DAG. U3 U10

13.2 Publication and execution contracts agree

The exposure policy uses exposure.supportedNetworkPlugins; discovery publishes those logical names as the dra.networking/supportedNetworkPlugins attribute. A generated root DeviceClass performs exact membership matching for the selected pluginRef.name and checks the sharing/persona requirements. These are project fields/attributes; upstream list values use strings. Artifact names, service names and legacy CNI binary names do not substitute for logical NetworkPlugin identities.

A plugin name alone cannot describe every conflict. For example, configurations of one shared-parent plugin may require incompatible host settings. Such distinctions belong in separately exposed personas, explicit contract attributes and compatibility groups, or a validated policy restricting configuration. They cannot be hidden only in opaque topology config, because the scheduler does not evaluate that config. U3

The compiler verifies that root selection guarantees the effects it authorizes; the node rechecks actual device identity and the pinned inventory contract. If a requested effect cannot be expressed by the advertised resource model, reject or make it a separately scoped feature rather than claiming scheduler enforcement.

13.3 Node locks and cross-plugin effects

The coordinator computes lock keys from physical identities and compatibility domains, not plugin names. Acquire multiple keys in deterministic order. Lock reservations cover all steps that can affect those resources and persist through uncertain cleanup.

An expiring coordination lease is not permission to start a second conflicting hardware operation. Plugins honor authenticated grants and fencing, while the coordinator resolves takeover and outstanding workers. Shared resources distinguish read/create-child access from exclusive reconfiguration.

A root operation that changes a PF-global setting, binding, or e-switch mode may affect sibling VFs even if the root allocation itself is exclusive only for one VF. Permit it only when the resource contract covers the entire effect or the setting is a preconfigured invariant. No plugin may reset a PF merely to make a VF request succeed.

13.4 Host-owned resources

Management links, the primary-network uplink, host bridges and their uplinks, and externally managed device users must be explicitly protected or integrated into the authority model. Hiding a busy VF from ResourceSlices does not make its PF safe for passthrough.

The baseline keeps host topology provisioning separate from per-Pod attachment. A shared bridge plugin attaches a workload to an administrator-managed bridge; it does not create a new physical uplink or move an unallocated bridge member. More dynamic host ownership needs an explicit reservation/provisioning API.

14. Workload Context, IPAM, MAC Reservations, and Stable Identity

This section extends the typed NetworkPlugin design. The context envelope, identity bindings, lease records and migration protocol below are project contracts to implement, not features automatically provided by DRA, CNI, NRI, or KubeVirt. The four public mutating hooks remain unchanged.

14.1 First-class reservation outputs, without templates

ipam and mac are logical NetworkPlugins with stepKinds: [Operation] and a descriptor operationRole: Reservation. They can run before a link exists. ConnectNetwork reserves or reuses their identity component and creates an attachment-specific holder; RemoveNetwork releases that holder after all consumers are cleaned. They never receive root preparation hooks.

A reservation operation can have no data inputs. This does not make it a DRA Allocation root. Its authorizationRoots field names the already allocated roots that authorize its use in this topology. Those references are authorization associations, not connection dependency edges. The whole plan is validated and all DRA roots are prepared before any reservation call. An unused reservation, a reservation with no authorized roots, or a consumer outside their approved plan is rejected.

The canonical wiring is:

# NetworkTopology step fragment. All from references create execution edges.
- name: ipam-data
  kind: Operation
  pluginRef: {name: ipam, contractVersion: v1alpha2}
  authorizationRoots: [vf0, vf1]
  config:
    identityBinding: data
    poolRef: tenant-data-v4
    addressFamilies: [IPv4]

- name: mac-data
  kind: Operation
  pluginRef: {name: mac, contractVersion: v1alpha2}
  authorizationRoots: [vf0, vf1]
  config:
    identityBinding: data
    poolRef: tenant-data-macs

- name: data-vlan
  kind: Operation
  pluginRef: {name: vlan, contractVersion: v1alpha2}
  inputs:
    parent:
      from: {step: bond0, output: interface}
    addresses:
      from: {step: ipam-data, output: lease}
    mac:
      from: {step: mac-data, output: lease}
  interfaceName: data0
  config:
    vlanId: 100
    mtu: 1500
    addressApplication: PodInterface
    defaultRoutePolicy: Preserve

config contains literals; inputs carries complete typed values. There is no string interpolation, index lookup in accumulated results, or implicit plugin invocation from a scalar value. The compiler resolves these ports before dispatch and rejects wrong types, cycles, and mismatched identity bindings.

14.2 Lease types and the allocation/application boundary

The following are illustrative typed descriptions, carried inside two ResourceHandles, not raw string values to substitute into config:

{
  "addressLease": {
    "type": "AddressLease/v2",
    "identityUID": "endpoint-data-42",
    "reservationUID": "ip-reservation-17",
    "holderID": "attachment-b-ip-holder",
    "networkUID": "tenant-data-network-uid",
    "collisionDomainUID": "tenant-data-routing-uid",
    "activationEpoch": 7,
    "addresses": [{"address": "10.100.0.25/24", "gateway": "10.100.0.1"}]
  },
  "macLease": {
    "type": "MACAddressLease/v2",
    "identityUID": "endpoint-data-42",
    "reservationUID": "mac-reservation-9",
    "holderID": "attachment-b-mac-holder",
    "networkUID": "tenant-data-network-uid",
    "collisionDomainUID": "tenant-data-l2-uid",
    "activationEpoch": 7,
    "address": "02:00:00:00:10:25"
  }
}

The active-use grant and backend record—not the illustrative epoch number in these descriptions—authorize activation. The IP collision domain and MAC collision domain need not be the same UID; both must match the approved network definition. These values are example outputs for one identity, not defaults applied to all workloads.

The lease handle ID is per holder and attachment, even when two holders reference the same retained reservation. Stable reservation IDs live in the typed description. Never reuse a source sandbox’s handle in the destination sandbox; obtain a new authorized handle against the same backend identity.

The IPAM plugin reserves addresses; the MAC plugin reserves a MAC. Neither creates a link or installs an address. A consuming VLAN plugin may now apply both directly using the shared SDK effect kit. It does not call IPAM internally. A separate address plugin remains supported for users who prefer separate link and address steps, but the compiler permits only one address owner per interface/address tuple. Do not enable both paths for the same lease on the same interface.

An absent addresses input means layer-2-only operation. An absent mac input means the link plugin’s approved native/default MAC behavior. Missing inputs required by a locked contract fail validation; the executor never silently allocates replacements.

14.3 Pod information passed to plugins

The node coordinator obtains the actual Pod through the supported runtime/API integration, verifies its UID, sandbox association, node and claim authorization, and constructs a versioned WorkloadContext. It is supplied at ConnectNetwork and retained for RemoveNetwork. Kubelet DRA preparation still has no guaranteed Pod context; NodePrepare must remain independent of Pod metadata. U1

# Proposed NetworkTopology.spec fields. These form part of the immutable revision.
workloadContext:
  pod:
    labels: [app.kubernetes.io/name, networking.example.com/replica-slot]
    annotations: [networking.example.com/address-profile]
  resolveOwners: [ReplicaSet, Deployment, StatefulSet, VirtualMachineInstance, VirtualMachine]
identityBindings:
  - name: data
    networkRef: {name: tenant-data}
    interfaceKey: data0
    subject: {strategy: PodUID}
    retention: {mode: SubjectLifetime}
    activation: Exclusive

networkRef resolves to an administrator-approved network definition with an immutable network UID and explicit IP routing/MAC collision domains. This is a project/backend configuration reference, not a claim that Kubernetes supplies a built-in object with that name. Network and pool names are resolved and pinned before use; a reused name must not adopt an old reservation.

The envelope includes cluster identity, namespace name and UID, Pod name/UID/resourceVersion/service account, projected labels and annotations, verified controller relationships, observation identity/digest, and resolved network-identity bindings. Include only metadata requested by the topology and allowed by the plugin’s locked metadata policy. Apply count/size limits and redact confidential annotations; do not forward the whole Pod, Secrets, arbitrary environment variables, or service-account tokens.

Per-plugin descriptor policy is explicit, for example:

# NetworkPlugin.spec fragment for ipam v1alpha2; not a complete CRD.
stepKinds: [Operation]
operationRole: Reservation
workloadContext:
  required: true
  pod:
    labels: [app.kubernetes.io/name]
    annotations: [networking.example.com/address-profile]
  verifiedIdentityRequired: true
outputs:
  lease:
    type: AddressLease/v2
    cardinality: 1
    ownership: Held

The MAC descriptor similarly exports MACAddressLease/v2 and can request less metadata. A lease-aware VLAN descriptor declares parent: KernelInterface/v1 with CreateChild access, optional addresses: AddressLease/v2 and mac: MACAddressLease/v2 with UseLease access. It need not receive unrelated workload labels. Descriptor ownership: Held means an attachment holder of an independently retained reservation; it is not a newly created kernel resource.

The initial context-size policy caps the canonical envelope at 32 KiB, projected labels and annotations at 32 entries each, an annotation value at 2 KiB, verified owner traversal at depth eight, and identity bindings at 16. These are project limits to validate against real workloads, not upstream API limits. Reject an oversized required projection with a diagnostic; do not silently truncate identity-bearing data. Hash canonical effective content while excluding the digest field itself and ephemeral authorization-token bytes. The projection policy and resolved identity policy remain digest inputs.

The SDK makes metadata available as read-only data for pool/profile selection and diagnostics. For example, an approved address-profile annotation can select an entry in the plugin’s administrator-defined profile map. It cannot provide a new pool endpoint, credential, host path, arbitrary IP to steal, or a MAC outside the authorized pool. Plugin-specific derived choices are included in the persisted semantic request before backend effects.

Read from the API does not mean trusted for identity. Pod labels and annotations may be tenant-controlled. Owner references must be resolved to exact objects and checked against authorization and the controller’s real relationship; a claimed owner UID alone is insufficient. Protected slot values and VM ownership require the corresponding trusted resolver/admission contract. Kubernetes records controller ownership via owner references, but labels alone do not establish that relationship. I2 I3

Snapshot the effective context once per attachment transaction before dispatch. A later metadata edit cannot change the meaning of a retry, switch its pool, or rename its stable owner. Cleanup uses the saved context even when the Pod was deleted. Before a new attachment or migration, resolve current authorization anew. Security revocation is an explicit coordinated operation, not an in-place mutation of an old request’s metadata digest.

If the required owner graph or identity proof cannot be verified, return a bounded failure. Do not fall back from VM identity to launcher-Pod identity or from a protected slot to a random label.

14.4 Stable subject strategies

A stable address is a promise about a defined subject, not about every object that happens to share an app label.

StatefulSet ordinals are an upstream stable workload identity. ReplicaSets instead manage interchangeable replicas; this design must supply any per-replica slot assignment it requires. I1 I2

For multiple replicas, do not allocate one IP or MAC from just the ReplicaSet UID. Use WorkloadSlot, giving each slot a different identity, or use PodUID when replacement continuity is not needed. A label such as networking.example.com/replica-slot is only a transport hint: a trusted slot record must bind that slot to the current Pod. Without such a controller, reject the slot profile. Choosing the first free slot on each node or hashing all replicas’ common labels is not sufficient.

For continuity through a Deployment rollout, key a protected slot to the verified Deployment UID, not the old ReplicaSet UID. A singleton Deployment using a stable address must either use an explicitly coordinated handoff or wait until the old holder is fenced; an ordinary surge rollout cannot activate duplicate addresses. Across owner deletion and recreation, UID identity intentionally changes. Retention across object recreation requires an explicitly transferred administrator-owned identity record, not name-based reuse.

A reservation key is conceptually:

cluster/tenant boundary + namespace UID + immutable network UID
+ canonical verified subject + stable interfaceKey + identity generation

It excludes node, Pod sandbox, DRA device/share identity, topology revision, and step name when the selected subject intentionally survives them. Changing a VLAN routing domain, network definition, pool policy or incompatible interface contract still requires explicit rebinding or drain; excluding a topology hash is not permission to silently move an address between networks.

The IPv4/IPv6 and MAC components for one binding share a backend identityUID and logical endpoint identity. Backend uniqueness also uses the actual collision domain: two namespaces attached to the same L2 segment must not allocate the same unicast MAC independently. Overlapping IP pools in one routing domain need coordinated uniqueness or configuration rejection.

14.5 Durable reservation, holder, and activation state

Use three separate backend records/concepts:

1. Reservation: long-lived assignment of an IP component or MAC to a canonical identity. It may outlive every Pod sandbox.
2. Holder: an attachment’s right to stage/use that component, keyed by attachment identity, node, claim and operation. Repeating an RPC returns the same holder, not a new reference-count increment.
3. Activation record: who may currently use the logical endpoint, with a monotonic fencing epoch shared by its IP and MAC components.

Node-local checkpoints cannot provide cross-node uniqueness. IPAM and MAC services must use a durable, authenticated backend with atomic owner/address uniqueness, idempotency keys, and compare-and-set or transactions. A backend partition must not fall back to a node-local allocator. Candidate hashing for repeatability is allowed only with collision detection and committed reservations; hashing a VM name is not allocation.

New MAC allocations are canonical 48-bit, locally administered unicast values from an approved pool, excluding all-zero/broadcast and reserved values. Importing a preassigned guest MAC uses an authorized reservation/import path with the same uniqueness checks. Do not claim ownership of a vendor OUI. If a guest already has a configured MAC, the KubeVirt adapter must reconcile it with the reservation instead of silently changing one side. I5

For IPAM, reserve all required families or fail the binding as incomplete. An IPv4 reservation and a MAC reservation may complete in different steps; there is no implicit distributed transaction. The identity service tracks component completion, and the consumer requires the complete matching bundle before activation. Retry partial acquisition by the same canonical key. Compensation drops only the failed attempt’s holder and applies the configured retention policy; never delete another holder’s component. The reservation/activation backend can be a common service even when IPAM and MAC have separate logical endpoints.

For an ordinary Exclusive attachment, the coordinator/identity authority commits the initial activation epoch only when the complete required component bundle and this attachment’s holder set are durable, and no usable previous holder remains. A consumer’s SDK use effect validates and records use against that decision; an IPAM or MAC provider cannot independently transfer the endpoint. A provider may finish its step before the endpoint is active: its successful result means its reservation/holder is ready, not that the workload can transmit. Whole-topology readiness still requires the consuming link/binding to complete.

RemoveNetwork drops a holder only after every interface/guest consumer of it is detached or verifiably fenced. It does not automatically free the reservation:

A VM policy normally uses SubjectLifetime or ExplicitRelease. IPAM and MAC must use compatible retention for the same endpoint. Quotas and orphan reconciliation apply to retained reservations. Owner disappearance, missed heartbeats, lease TTL expiry or deletion of a Kubernetes Pod is not proof that an old node or VM stopped transmitting. A reaper may flag/quarantine uncertainty; it must not reassign until the deployment’s fencing policy establishes safe reuse. Backend retention/GC is control-plane work, not a new NetworkPlugin mutation hook.

Renewals, where needed, are idempotent internal lease maintenance tied to the persisted holder. They cannot reassign identity, change activation ownership or free reservations. Cleanup removes consumers before releasing holders; a renewal racing holder deletion must fail its holder/epoch compare-and-set.

14.6 Applying lease outputs in VLAN and other steps

A vlan contract with the two optional lease ports uses this effect sequence for ordinary Pod attachment:

Validate parent authority, namespace, identity bundle and use grant.
Create the VLAN down, using the approved MAC at creation when supported.
Set/verify its MAC while still isolated; never change the parent's MAC.
Install only addresses/routes covered by the AddressLease and this step's policy.
Activate the child only when its identity holder is authorized Active.
Complete DAD/required local validation and commit outputs.

The SDK supplies these owned effects; VLAN does not duplicate an IP allocator or overwrite the whole route table. Remove reverses activation/address effects, then deletes the child. After its cleanup receipt, IPAM/MAC may release that holder. The reservation’s retention is evaluated separately. A failed VLAN step with partial effects still blocks early release.

IP prefixes, gateway reachability, allowed route destinations/tables, default-route preservation and MTU are validated. AddressLease is not permission to modify primary-network DNS or routes. Changing IPv6 address-generation settings or doing DAD/ARP probing requires a declared profile; probes do not replace authoritative allocation or fencing.

The ipvlan limitation is a kernel behavior, not a packaging limitation: sharing the link-plugin service cannot give ipvlan a per-child L2 identity. I4

Allocation roots may declare Connect-only address/MAC reservation inputs in a newer locked contract. They must not depend on derived links or require these values during NodePrepare. The compiler rejects a cycle in which IPAM waits for that root’s interface while the root waits for IPAM. Metadata-aware reservation producers run after all roots are prepared, so a root’s assigned VF and all required CDI access must be known without a Pod-derived IP or MAC.

14.7 KubeVirt migration and the one-active-endpoint rule

Stable IP/MAC reservation support is not proof of live-migration support. KubeVirt separates host-to-Pod connectivity from the guest binding, and network bindings declare their migration support. This project needs a tested binding/migration adapter for the selected KubeVirt version and hardware path. Do not infer that a VF, macvtap, or any arbitrary topology is migratable. I5 I6 I7

Use a verified VM/VMI identity plus the logical guest interface key, not the virt-launcher Pod UID. The adapter resolves the VM/VMI/migration objects, authorized source and target Pods, intended network and guest interface. A Pod annotation claiming vm-name is not migration authorization. Guest addresses belong to the guest/binding; a Pod-interface VLAN example is not a complete VM binding.

The ordinary SinglePod rule remains per DRA claim. Source and target have separate claims and different physical allocations/holders on their nodes, referring to one stable network identity. This does not share one node-local VF claim between nodes or allow two Pods to use one claim. The KubeVirt adapter must provision and authorize that separate target claim through supported integration; it is not automatically wired by these documents.

Migration is an opt-in CoordinatedHandoff identity profile with a trusted migration controller and a backend activation record:

The backend activation epoch fences backend ownership, not packets already transmitted by a kernel interface. Source quiescence requires actual runtime/guest/fabric enforcement. If the source is unreachable and fencing cannot be established, fail closed and keep the target staged; do not activate duplicate addresses based on TTL expiry.

No fifth public hook is added. Staging/activation are explicitly supported ConnectNetwork phases with new monotonic intent generations, separate operation IDs and the same attachment scope. Stage returns STAGED; Activate references its saved base-connect operation, checks the external handoff grant and epoch, and returns READY after activation. Stage journals and resources are reused, not recreated. RemoveNetwork cleans all generations in that scope, including a partial activation, and permanently fences delayed activation. Recovery and inspection do not silently mutate staging into active use.

The restricted VF/bond/link SDK examples are not staging-capable until they implement and advertise that capability. A staged KubeVirt target may need a launcher process before cutover; the runtime adapter may admit that trusted migration-preparation process only under a certified paused/isolated-guest contract. Ordinary application/init containers still require READY. Do not deadlock migration by treating a target launcher as an ordinary active workload, and do not broadly waive the readiness gate.

The adapter owns guest MAC/IP delivery, VM network binding, migration coordination, rollback, and required fabric/neighbor updates. Gratuitous ARP or unsolicited neighbor advertisement can be part of activation but do not replace fencing. Return migration support as unavailable when these pieces are absent.

14.8 Policy example: retained VM identity

# Proposed NetworkTopology.spec fragment, NOT a complete KubeVirt VM manifest.
# Requires an implemented and certified KubeVirt identity/binding adapter.
workloadContext:
  pod:
    labels: [kubevirt.io/domain]
    annotations: [networking.example.com/address-profile]
  resolveOwners: [VirtualMachineInstance, VirtualMachine]
identityBindings:
  - name: guest-data
    networkRef: {name: vm-data}
    interfaceKey: guest-data0
    subject: {strategy: VirtualMachine}
    retention: {mode: SubjectLifetime}
    activation: CoordinatedHandoff

kubevirt.io/domain is merely allowed observation data here. The authoritative VM identity and migration role come from the verified adapter, not from that label. Both IPAM and MAC steps set config.identityBinding: guest-data; a guest-binding consumer receives their leases and a prepared interface. It must not use addressApplication: PodInterface for guest-owned addresses.

14.9 Selection and scheduling boundaries

Reservation backends can use projected Pod metadata and verified owner information, but only through declared administrator-approved policies. Require deterministic replay of those decisions. IPAM/MAC exhaustion occurs after DRA scheduling in this design; it does not trigger automatic reallocation of roots or a scheduler retry on a different node. Global pools, topology-aware scheduling constraints, or explicit pre-reservation are needed when address availability must influence placement.

Tuning remains distinct from allocation: its owned settings must not overwrite lease-controlled MAC/address identity. The primary network, SDN controller and any guest binding retain their explicitly assigned responsibilities. No plugin can release a foreign lease by observing that an interface disappeared.

15. Status, Diagnostics, and Security

15.1 Status maps to actual allocation keys

ResourceClaim.status.devices entries belong to allocated driver/pool/device/shareID keys. Report root device readiness and network observations under those actual keys. Do not add invented allocation entries for derived bonds or VLANs. U3

Publish the canonical NetworkAttachmentStatus defined below for attachment and exported-endpoint consumers. A bounded summary or reference may also appear in driver-owned data; it is not a second authoritative graph. A graph with two roots and one bond has one derived-interface observation instead of conflicting copies under both roots.

Use scoped status updates that preserve other drivers’ entries in mixed claims. Status failures do not rerun already successful hooks. Kubernetes API observation is not the runtime readiness barrier.

IPAM/MAC reservation providers have no upstream device-allocation result of their own. Report sanitized reservation/holder and binding state in attachment status or bounded driver data under a real root; never fabricate a DRA device result for an address lease. Report VM guest identity separately from launcher-Pod interface addressing. Do not expose backend credentials, protected metadata, ownership proofs or bearer grants.

15.2 Operational diagnostics

Record claim UID, root allocation/share ID, topology revision, sandbox ID, step, hook, operation ID, desired generation, plugin digest, duration, retry/error class, and cleanup debt. Emit metrics for prepare/connect/remove latency, failed and uncertain operations, retained namespace pins, unavailable plan revisions, IPAM allocations, and quarantined physical identities.

Avoid unbounded metric labels such as claim UID or operation ID; those belong in logs/traces. Redact authorization tokens, credentials, raw plugin state, and sensitive backend data. Network-specific diagnostics should identify the actual failed step rather than only “could not configure network.”

15.3 Trust and privilege

Platform administrators own topologies, plugin registry entries, exposure policies, generated-class mutation permissions, and node readiness labels. Applications may select approved claims/classes and permitted parameters, not select host paths or execution endpoints.

Plugins receive only the root bindings and input resources they need, plus narrow grants. Validate namespace identity, hardware identity, contract digest, and granted effects before mutation. A plugin must not use an interface-name string supplied in config to escape input ownership.

Authenticate local gRPC peers and enforce size limits, deadlines, rate limits, and state-directory permissions. Do not expose the NetworkPlugin execution service as an unauthenticated cluster-wide service. Remote IPAM/SDN backends use their own authenticated clients and bounded policies.

A trusted privileged plugin can still compromise the host; deployment provenance, artifact signing/verification, admission, OS confinement, and audited code remain necessary. Long term, a narrow privileged device/namespace broker can reduce individual plugin privileges, but it does not replace the resource contract.

15.4 Canonical attachment and export observations

networking.dra.io/v1alpha1, kind NetworkAttachmentStatus, is a proposed namespaced, read-only-for-workloads API. Create one object per complete sandbox attachment identity from Section 3.2, in the Pod and claim namespace. Its immutable spec records the Pod name/UID, claim name/UID, allocation fingerprint, topology revision name/UID/digest, node name/UID/boot ID, runtime sandbox ID, and attachment attempt incarnation. A retry updates the same object; a new sandbox or allocation creates a different one. Name it attachment- plus the first 52 hexadecimal characters of the SHA-256 of the canonical identity. Always compare the full identity before adoption; a name collision is an error.

The following is the normative proposed field contract. Unknown fields are rejected, references require UIDs, and all lists have schema-enforced bounds. It does not add fields to upstream Pod or ResourceClaim APIs.

The proposed phase enum is Pending, Preparing, Connecting, Staged, Ready, Removing, Removed, Failed, or Quarantined. Staged applies only to a certified staged-activation profile. Conditions use Kubernetes True|False|Unknown; the separate per-export integrations[] map-list is keyed by capability and records support, optional contract, state, reason, and observation time. Its state uses the integration states defined in Section 15.7, not the condition-status enum. Readers require matching identity/generation, unexpired observation and the intended export’s required integrations; attachment phase alone is insufficient.

The initial observation schema caps roots at 64, interfaces at 128, exports at 64, steps at 256, conditions at 16 per scope, addresses at 32 and routes at 64 per interface, and DNS nameservers/search/options at 8/32/32 entries. Identifiers and diagnostic strings require explicit CRD length limits; diagnostic messages are capped at 1,024 characters. These are project API bounds, not Kubernetes network limits. Compilation must fit the locked limits or reject the topology; reporting must fail visibly with ObservationLimitExceeded rather than truncate an authoritative export or keep renewing an old Ready record. Complete structural CRDs remain implementation work.

An observationID identifies only a sanitized record within this attachment. It is not a resource handle accepted by any mutation API. Two export aliases for the same link reference one interface observation. Two VF roots feeding a bond and two VLAN exports produce two root records, one bond observation, and one observation per VLAN; each VLAN export names its own final interface and both ancestor roots. Never select an export by the first root, array order, or a reused interface name. Access-only exports retain their DeviceAccess/v1 type and omit kernel-interface fields.

Each interface has separate addresses[], routes[], and dns observations. An address includes family, CIDR, subject, installation state (Reserved, Installed, Observed, or Unknown), reporting step/authority, observation time, and connection generation. Reserved does not mean installed; a guest reservation is not a launcher-Pod address. A route includes destination, optional next hop, table, metric, installation state and owner. DNS records include nameservers/search/options, authority and application state (Unapplied, AppliedByIntegration, or Unknown). Lease-supplied DNS starts Unapplied. The observer reports only approved fields on its authorized endpoints; it does not dump the primary network’s resolver or routing state.

The initial kernel-L2 profile has empty project-managed address/route observations and DNS Unapplied; it claims no address management. This does not assert that the kernel or workload cannot create addresses independently. A future address profile may report its owned installed addresses and routes. Neither an observed address nor a DNS suggestion grants permission to alter primary Pod.status.podIPs, /etc/resolv.conf, default routes, or primary-network status.

15.5 Status writers, freshness, and recovery

One attachment-status controller owns object creation and /status writes. Node coordinators submit sanitized observations through an authenticated internal reporting channel; they do not receive unrestricted cross-namespace status-write credentials. The controller verifies the reporting node UID, current coordinator epoch, full attachment identity, Pod-to-node and explicit Pod-to-claim association, and monotonic report sequence against retained authority records. It rejects forged identities and late reports from fenced coordinators. Cleanup reports after Pod deletion require the previously verified attachment record, not a new authorization inferred from a missing Pod. This reporting channel remains an implementation deliverable, separate from the four plugin RPCs.

The controller’s service account receives explicit CRD create/get/list/watch and /status update/patch permissions, plus the narrowly needed reference reads and retention deletes. Workloads receive only namespace-scoped observation reads when authorized. Labels help lookup, not authorization: ordinary RBAC cannot express “only objects whose Pod UID matches this caller” by label. Per-Pod isolation therefore needs an authorized projection/API or explicit object-name grants with their watch restrictions. The optional annotation adapter has a separate Pod metadata patch role; the status controller has no pods/status write permission. Kubernetes RBAC

For the first profile, coordinators report on state changes and at most 20 seconds apart while active; observations expire after 60 seconds. These bounds are revision policy, not address-lease expiry. The controller sets Fresh=Unknown with ObservationExpired after expiry. Consumers independently enforce validUntil, since a stopped controller cannot update conditions. Loss of reporting leaves the committed dataplane and runtime journal intact; it cannot free hardware or an address. A missing or expired object means unknown connectivity, never confirmed cleanup. Removing and Removed cannot be projected as ready endpoints.

After restart, the controller relists API state and reconciles current authenticated journal observations before advancing readiness. Consumers use list/watch with resource versions and relist after 410 Gone; a quiet watch alone does not prove freshness. Kubernetes API watch recovery Store no networking cleanup finalizer on this observation object: deletion or Pod garbage collection cannot trigger or certify release. A Pod owner reference is permitted for observation retention; unresolved cleanup remains in the durable journal and operator diagnostics even if the object is gone. API retries only republish observations and never rerun successful hooks.

15.6 Compatibility with existing network-status consumers

The optional compatibility adapter projects selected, current kernel-interface exports into k8s.v1.cni.cncf.io/network-status. That JSON format includes network name, interface, IPs, MAC, MTU, DNS and gateway fields, but lacks this attachment identity, graph provenance, and freshness contract. It is a lossy view, never execution authority or proof of readiness. Network attachment status types

The adapter is disabled by default and requires an administrator-defined export-to-network-name mapping plus compatibility tests for each consumer. Consumers that resolve name to a NetworkAttachmentDefinition require a deliberate mapping to that real NAD; do not fabricate NAD identity or execute that NAD a second time. Emit default: false, installed Pod-interface addresses only, and DNS only when a supported integration actually applied it. Omit unsupported fields and device information that cannot be attributed to one real allocation; never assign a bond’s identity to one arbitrary VF.

The annotation is one JSON string, so field managers do not independently own its array entries. Enable writes only with one agreed annotation writer for the Pod: either a cooperative merger owns the complete annotation or this adapter owns it exclusively. A merger retains other producers’ entries and uses resource-version preconditions; this project contributes only its registered export keys. A detected uncoordinated Multus/other writer, duplicate mapping, oversized result, or malformed annotation sets LegacyProjectionReady=False with a precise reason and stops projection. Never replace another writer’s value or resolve conflict by a last-writer loop. Legacy output is best effort: many existing readers cannot detect stale data. Applications needing correct identity/freshness use the canonical API; legacy-reader support is limited to explicitly tested behavior.

15.7 Integration capabilities and scoped readiness

Every compiled revision locks integrationCapabilities, keyed by capability name, with support: Supported|Unsupported. A supported entry additionally names its versioned contract, responsible implementation, required dataplane/runtime prerequisites, and conformance evidence. This describes the whole profile, including controllers and physical paths; a plugin name or a ResourceSlice attribute alone cannot establish service support. The baseline below is the intended initial contract, not a claim that its implementation already exists.

NetworkTopology.spec.requiredIntegrations is a proposed list of {export, capability, contract} requirements. Compilation rejects an unsupported capability, unknown contract or nonexistent export; node readiness and preparation recheck the locked prerequisites before effects. A profile with mandatory policy must block admission/start without its enforcer. An external policy object alone cannot add a requirement to this API: administrators must bind policy requirements through the approved topology/admission contract. Existing unrelated primary-network policy resources retain their own semantics.

Ready=True means the locked attachment contract completed. For the initial L2 profile it does not mean that an address exists, another host is reachable, or Kubernetes Services, DNS, policy or encryption cover the link. Per-export integration conditions report Unsupported, Pending, Ready, Degraded, or Unknown separately. Required integration failures prevent initial attachment readiness; optional integration failures remain visible. A later loss of required enforcement must execute that integration’s declared isolation/quarantine response and block new starts; flipping a status condition alone cannot protect an already running workload. Initial L2 has no such enforcement capability and must not be offered where it is required.

Kubernetes NetworkPolicy requires a networking implementation that enforces it; creating policy objects is insufficient. A secondary profile must identify its exact attachment/export targets and prove enforcement on those packet paths, including any SR-IOV/VFIO/representor or other bypass paths it claims. MultiNetworkPolicy introduces a separate NAD-oriented API and implementations; its presence does not automatically bind this project’s exports. NetworkPolicy prerequisites, MultiNetworkPolicy

A future service adapter may manage EndpointSlices for an explicitly selected Service without a selector, using a distinct endpointslice.kubernetes.io/managed-by value and Service association. It must validate permitted endpoint addresses, port and address-family mappings, uniqueness within the routed network, and actual forward/return connectivity from clients and service proxies. EndpointSlice publication does not configure those routes or distinguish overlapping secondary address domains. It must combine application readiness with fresh attachment/export and required-integration readiness; attachment readiness alone is insufficient. Define removal/termination handling and either reject or explicitly support publishNotReadyAddresses. Headless discovery also requires clients able to reach the advertised addresses. These are future integration criteria; the initial implementation creates no Services or EndpointSlices. Services without selectors, EndpointSlice conditions and ownership

15.8 Attachment and integration acceptance cases

These are required implementation tests, not results of documentation checks.

16. Additional Workload Profiles

16.1 AI/HPC: independent RDMA rails and GPU co-location

For AI networking, start by modeling independent rails rather than assuming that Ethernet bonding yields a supported RDMA aggregate. A root with rdmaAccess: true declares a capability the new SR-IOV/RDMA-capable NetworkPlugin must implement: root preparation establishes permitted device access, and connect validates the supported namespace/RDMA association. This profile is enabled only for certified hardware/driver combinations.

apiVersion: networking.dra.io/v1alpha1
kind: NetworkTopology
metadata:
  name: ai-multirail
spec:
  runtimeProfile: nri-secondary-v1
  attachmentPolicy: SinglePod
  execution:
    maxParallelSteps: 1
    primaryNetworkPolicy: Preserve
  steps:
  - name: rail0
    kind: Allocation
    pluginRef:
      name: sriov
      contractVersion: v1alpha1
    allocation:
      sharing: Exclusive
      selector:
        cel: device.attributes["dra.networking"].type == "vf" && device.attributes["dra.networking"].pfName
          == "enp3s0f0" && device.attributes["dra.networking"].rdma == true
    interfaceName: rail0
    config:
      mtu: 9000
      vf:
        vlan: 0
        spoofCheck: true
        trust: false
      rdmaAccess: true
  - name: rail1
    kind: Allocation
    pluginRef:
      name: sriov
      contractVersion: v1alpha1
    allocation:
      sharing: Exclusive
      selector:
        cel: device.attributes["dra.networking"].type == "vf" && device.attributes["dra.networking"].pfName
          == "enp3s0f1" && device.attributes["dra.networking"].rdma == true
    interfaceName: rail1
    config:
      mtu: 9000
      vf:
        vlan: 0
        spoofCheck: true
        trust: false
      rdmaAccess: true
  exports:
    rail0:
      from:
        step: rail0
        output: interface
    rail1:
      from:
        step: rail1
        output: interface

Address/IPAM operations can be added per rail when required; the minimal example above only exposes the rails. InfiniBand and RoCE have different addressing and device-lifetime requirements and need separate supported contracts where they differ.

A mixed claim can request a GPU from its existing driver and the two network roots:

apiVersion: resource.k8s.io/v1
kind: ResourceClaimTemplate
metadata:
  name: gpu-and-rails
  namespace: default
spec:
  spec:
    devices:
      requests:
      - name: gpu
        exactly:
          deviceClassName: site-gpu-class
          allocationMode: ExactCount
          count: 1
      - name: rail-a
        exactly:
          deviceClassName: site-ai-multirail-revision-rail0
          allocationMode: ExactCount
          count: 1
      - name: rail-b
        exactly:
          deviceClassName: site-ai-multirail-revision-rail1
          allocationMode: ExactCount
          count: 1
      constraints:
      - requests:
        - gpu
        - rail-a
        - rail-b
        matchAttribute: device.k8s.io/pcieRoot

The site-* DeviceClasses stand for real administrator-provided or generated revision classes and must be replaced. This example does not assert any particular GPU vendor’s class name or attribute schema. Use the PCIe constraint only when all participating devices actually publish the same standardized attribute with interoperable values. U2 U3

The scheduler can use matching attributes to constrain root placement; this does not make network and GPU node preparation one transaction. A failure in one driver does not authorize this driver to undo the other driver’s work.

PCIe co-location is a placement constraint, not proof of GPUDirect RDMA functionality or performance. Validate the GPU/NIC/driver/memory-sharing/platform prerequisites independently. RDMA isolation and device-node permissions are also not established merely by mounting /dev/infiniband. U11

16.2 Telco: VFIO access without a kernel netdevice

Cold inventory retains VFIO-bound and unbound functions without requiring a netdevice, but visibility does not enable their execution. The initial kernel execution profiles cover exclusive VFs and explicitly delegated shared parents. This VFIO profile requires a separately certified executor and effect-specific IOMMU/reset-domain ownership checks from Discovery Sections 4.1.3–4.1.5 before its persona can become allocatable.

apiVersion: networking.dra.io/v1alpha1
kind: NetworkTopology
metadata:
  name: telco-vfio
spec:
  runtimeProfile: nri-secondary-v1
  attachmentPolicy: SinglePod
  steps:
  - name: dpdk-vf
    kind: Allocation
    pluginRef:
      name: vfio
      contractVersion: v1alpha1
    allocation:
      sharing: Exclusive
      selector:
        cel: device.attributes["dra.networking"].type == "vf" && device.attributes["dra.networking"].driver
          == "vfio-pci"
    config:
      bindingPolicy: RequireAlreadyBound
      requiredDriver: vfio-pci
      requireExclusiveIOMMUGroup: true
  exports:
    device:
      from:
        step: dpdk-vf
        output: access

The initial VFIO contract conservatively requires the selected function already to be bound to vfio-pci; the root selector expresses that requirement as well. This profile needs a matching exposure policy advertising vfio, rather than the Ethernet reference policy that advertises sriov and host-device. The discovery inventory must retain PCI function identities without a kernel netdevice. Authorized rebinding can be added later with preparation-side rollback and family-level safety checks. An isolated usable IOMMU group must be verified; one PCI function’s allocation is not sufficient when other functions in its group remain owned elsewhere. U12

NodePrepare returns the prepared CDI device IDs. ConnectNetwork verifies the allocation/attachment and returns DeviceAccess, with no invented interface output. RemoveNetwork removes the attachment association after workload quiescence; NodeUnprepare releases the preparation and access state.

A kernel bond or vlan cannot take DeviceAccess where it expects KernelInterface. This is a type mismatch, not a statement that userspace applications cannot implement bonding. CPU isolation, hugepages, IRQ placement, NUMA memory policy, and DPDK application configuration remain separate workload/platform requirements.

16.3 Shared parent: macvlan or ipvlan

apiVersion: networking.dra.io/v1alpha1
kind: NetworkTopology
metadata:
  name: shared-macvlan
spec:
  runtimeProfile: nri-secondary-v1
  attachmentPolicy: SinglePod
  steps:
  - name: parent
    kind: Allocation
    pluginRef:
      name: macvlan
      contractVersion: v1alpha1
    allocation:
      sharing: Shared
      selector:
        cel: device.attributes["dra.networking"].ifName == "enp3s0f1"
      capacityRequests:
        dra.networking/macvlans: '1'
    interfaceName: secondary0
    config:
      mode: bridge
      mtu: 1500
  exports:
    network:
      from:
        step: parent
        output: interface

This shared root reserves one unit explicitly. The generated claim template must contain capacity.requests matching allocation.capacityRequests, unless the compiler has proven the selected persona’s enforced default/allowed value already guarantees the same reservation. The baseline companion policy uses default one with validValues: ["1"]; it does not permit hidden per-plugin scheduler arithmetic.

Root preparation retains the host parent and checks the authorized macvlan persona. Connection creates one attachment child. Removal deletes that child, not the PF. Another claim’s child survives. The corresponding ipvlan topology selects an ipvlan persona; KEP 5963 groups prevent those incompatible modes from coexisting where the policy declares them incompatible.

Repeated connection after a sandbox replacement cannot create a second child against the same one-unit reservation before the first child has been cleaned. A topology requesting several children needs enough root allocations or a supported quantitative contract.

16.4 Shared bridge and software networks

A bridge allocation root represents an existing administrator-owned bridge and one published port entitlement. It can create a veth pair to the sandbox. Derived VLAN/tuning/address operations consume its returned interface handle where their type/effects contracts allow it.

The bridge’s uplink is not a free host device for other steps. The companion host-bridge profile protects physical dependencies instead of pretending a hidden VF has a scheduler reservation. Synthetic capacity devices for pure software networking are a possible extension, but need an explicit inventory/consumption model; they are not smuggled in as unallocated rootless operations.

16.5 OVN-Kubernetes and other SDN integrations

Use an ovn NetworkPlugin identity for a proposed gRPC adapter—not the legacy ovn-k8s-cni-overlay binary name. Its implementation needs a real control-plane contract for network/NAD identity, logical-port readiness, representor/OVS ownership, address allocation, and cleanup. This is a separate integration profile, not an automatic consequence of the four hooks. The referenced OVN DRA proposal is a useful integration input, not proof of this adapter’s existence. U13

Two supported ownership directions are possible: this central network DRA owns selected hardware and an OVN plugin realizes its network attachment through approved OVN interfaces; or OVN’s separate DRA owns disjoint hardware and handles that network independently. Do not let both advertise the same functions, or let both IPAM systems allocate addresses for one attachment.

Layer-2, layer-3 and localnet profiles must each define primary-network coexistence, network-policy ownership, node-local versus controller operations, and which effects are reversible. Exposing an ovn capability on a device is allowed only after those implementation tests pass.

16.6 Stable Pod, replicated-workload and VM endpoints

Use the Section 14 identity strategies with explicit retention and activation policy. PodUID is available to the ordinary single-Pod profile. StatefulSetOrdinal and WorkloadSlot need verified controller identity; slot continuity across a Deployment rollout is a project controller responsibility. ReplicaSetSingleton rejects concurrent use rather than handing the same address to all replicas.

The KubeVirt profile reuses a VM/VMI reservation across different launcher Pods and independent DRA allocations. IP/MAC values alone do not implement guest binding, live VF migration or forwarding convergence. A tested KubeVirt adapter must provide paused-target staging, source fencing, target activation and rollback before the profile is offered as supported. I6 I7

17. Plugin Deployment, Upgrade, and Compatibility

17.1 Deployment model

NetworkPlugins run as services, not init containers that copy executables to /opt/cni/bin and sleep. An operator or explicitly managed DaemonSets deploys the approved plugin artifacts with persistent state volumes, local sockets, and required namespace access. An implementation can package several small logical plugins in one service to reduce deployment overhead.

Only the node coordinator registers with kubelet as the network DRA plugin and with the runtime as the required NRI participant. Individual NetworkPlugins do not need their own kubelet registration or NRI subscription; they implement the four gRPC operations and inspection contract.

Plugin registration is local and authenticated. The central operator correlates the expected artifact digest, reported contract digest, local readiness and node runtime profile before marking plan revisions available.

17.2 Upgrade rules

An active attachment pins a plugin artifact/contract/state format. A replacement implementation must either read and safely operate that state under a declared compatible upgrade path, or the old revision remains available to finish cleanup. Multiple processes cannot concurrently own the same operation journal/resource scope during a handover.

Do not garbage-collect an old plugin because no new topologies reference it while its prepared claims or teardown operations remain. On-disk migrations require rollback/recovery tests. Contract-breaking upgrades create new revisions and rollout through new claims/Pods, not live mutation of existing topology instances.

The first implementation should use a common plugin SDK for journal persistence, fencing, namespace opening, resource-handle validation, error details, and test harnesses. Copying that infrastructure independently into every plugin would multiply the most difficult correctness bugs.

17.3 Capability degradation

When a required plugin becomes unhealthy, withdraw new-plan readiness and reject affected new prepares/connects. Preserve the active network and journal. If recovery requires restart, reconcile first; do not blindly replay all hooks against an already functioning dataplane.

The alpha source-level availability of a DRA field is not evidence that an OpenShift release enables or supports it. The companion feature matrix is a pinned source baseline, and deployment requires target-cluster verification. Unknown or stripped required fields fail closed; there is no automatic fallback that removes compatibility constraints. U10

18. Required Test and Conformance Matrix

No runtime implementation is claimed by this document. The following tests are acceptance criteria.

Test with real network namespaces and virtual links before moving to hardware, but do not treat virtual-link tests as proof of SR-IOV, RDMA, VFIO, or fabric behavior. Validate actual API serialization, CEL selection, allocation, NRI callbacks and privileged effects on each supported cluster/runtime version.

Fault injection must target the boundary after an irreversible observation or side effect but before acknowledgment. A test that only retries successful calls cannot prove recovery safety.

19. Viability, Tradeoffs, and Alternatives

19.1 Architectural assessment

This is a viable proposed separation of concerns: scheduler-visible root resources and compatibility, immutable typed plans, and small node-local operations with explicit ownership and cleanup. It addresses the original coupling between physical allocation and complicated network construction without forcing every plugin to become a DRA driver.

The value comes from a coherent resource and lifecycle contract, not from gRPC alone. This document does not claim the approach is a new upstream networking standard, that arbitrary third-party plugins will interoperate immediately, or that application-level network semantics become atomic because roots share one claim. A gRPC wrapper around an existing process with unchanged assumptions about results, retries, and deletion would retain the earlier defects.

The largest engineering risks are runtime lifecycle enforcement, recovery of partial side effects, stable device identity across mutations, compatibility between advertised capacity and actual effects, and an ecosystem of versioned plugin contracts. These must be validated before claiming general-purpose production support.

19.2 Why keep the four hooks

Two root-only hooks cleanly separate claim-scoped hardware access from sandbox-scoped network realization. Two graph-wide hooks make each plugin responsible for connecting and undoing one small step. This is sufficient for the baseline provided that read-only inspection, a central runtime readiness gate, and recovery are mandatory supporting facilities.

Do not add container-start mutation to every plugin or turn every recovery case into another public lifecycle hook. Conversely, do not omit recovery just to say there are only four RPCs. Four mutation intents can be reconciled many times.

19.3 Alternatives and tradeoffs

The suggested compromise is one shared node execution framework and SDK, with small logical plugin implementations. Deploy them in one trusted process initially when that reduces operational overhead, and use the same gRPC boundary for independently packaged implementations when needed. Keep the kubelet/NRI adapter replaceable so the topology and plugin contracts do not become inseparable from one runtime.

For AI workloads, prioritize independent NIC/RDMA rails and placement constraints before promising general RDMA bonding. For Telco, prioritize exclusive VF and shared-parent isolation, deterministic startup, and cleanup before broad plugin extensibility. Keep primary networking and external fabric controllers separate until a profile explicitly integrates them.

20. Implementation Roadmap and Review Decisions

20.1 First implementation slice

Implement the policy/topology types and pure compiler, the four-hook protobuf/SDK, durable coordinator and plugin journals, root identity/authorization resolution, and one certified runtime adapter. Add an exclusive kernel-VF root and minimal virtual-link test plugins before running the bond/VLAN/address example on hardware.

The first end-to-end milestone is not “a DAG can create a bond.” It is “one claim and one sandbox can connect, fail at every step, recover, and remove without leaking resources or admitting an unconfigured application container.”

20.2 Staged expansion

Foundation: immutable revisions, strict descriptors, named typed ports, exact root mapping and single-consumer enforcement.

Lifecycle: root preparation, durable sandbox attachment, namespace pinning, required runtime participation, readiness gate, duplicate/missed event recovery, and dependency-safe cleanup.

Network realization: Ethernet VF → bond → VLAN, with independent typed IPAM/MAC inputs and optional tuning; common SDK address application and owned routes preserve the primary network. Implement reservation/holder/activation transactions and protected workload context before promising stable identities.

Hardware profiles: shared parents, VFIO, RDMA rails and mixed GPU claims; each adds its own isolation and recovery tests.

Integrations: certified KubeVirt identity/binding handoff, OVN/SDN and shared-bandwidth enforcement only after explicit resource, backend, actual dataplane fencing and runtime contracts are implemented.

20.3 Decisions proposed for review

Remaining work includes choosing the actual owned API/driver domains, target runtime/builds, IPAM backend, namespace-sharing deployment details, journal engine and durability parameters, plugin isolation profile, and implementing the structural CRDs and authenticated reporting controller specified in Section 15. These are concrete implementation decisions, not reasons to leave the four-hook semantics ambiguous.

21. Aligned Discovery Contract

The discovery companion is aligned with the execution and SDK terminology as of 2026-09-23. Its policies, ResourceSlice examples, DeviceClass selectors and discovery diagrams use the following contracts:

The retired draft fields supportedCNIPlugins and supportedCNIs are not accepted aliases. This alignment defines the proposed API; installable CRD schemas, compiler code and admission remain implementation deliverables. Their validation must enforce the canonical names and reject the retired fields. The external primary-network implementation may still be a CNI; that does not change the NetworkPlugin contract.

IPAM/MAC identity records are not synthetic NIC inventory and do not alter the companion’s device counts. Direct Connect-time reservation inputs to shared-parent roots still create exactly the authorized number of children. The companion’s KEP 5963 compatibility model, local-capacity separation and active-publication retention remain in force. This revision strengthens their connection to execution instead of replacing them with plugin-local locks.

Appendix A. Complete Draft NetworkPlugin Protocol

Canonical source: proto/networkplugin.proto. Update that file first, then run make network-docs-sync; CI rejects drift between the source and this appendix.

This is a proposed project protocol. It is intentionally separate from kubelet DRA and NRI wire messages. The service has four mutation hooks plus a separate read-only inspection service. Field numbers are a reviewable draft, not a compatibility promise before implementation.

All context fields, nonempty identifiers, limits, root/operation applicability and typed JSON schemas need semantic validation; protobuf syntax alone does not enforce them. Missing response/opaque state is specifically supported during cleanup after an uncertain call. Mutation grants are authenticated project-internal authority, not tokens applications can supply.

// Proposed project protocol, not an upstream Kubernetes or NRI API.
syntax = "proto3";
package networking.dra.networkplugin.v1alpha1;

// Exactly four mutating lifecycle hooks. Each call targets one DAG step.
service NetworkPlugin {
  rpc NodePrepare(NodePrepareRequest) returns (NodePrepareResponse);
  rpc NodeUnprepare(NodeUnprepareRequest) returns (CleanupResponse);
  rpc ConnectNetwork(ConnectNetworkRequest) returns (ConnectNetworkResponse);
  rpc RemoveNetwork(RemoveNetworkRequest) returns (CleanupResponse);
}

// Read-only management; these are not additional lifecycle hooks.
service NetworkPluginInspection {
  rpc GetInfo(GetInfoRequest) returns (GetInfoResponse);
  rpc InspectOperation(InspectOperationRequest) returns (InspectOperationResponse);
}

message OperationContext {
  string operation_id = 1;             // Stable across retries of this intent.
  string semantic_digest = 2;          // Canonical semantic payload, SHA-256.
  string scope_id = 3;                 // Preparation or attachment-step scope.
  uint64 desired_generation = 4;       // Monotonic within scope; removal fences creation.
  uint64 coordinator_epoch = 5;        // Persisted node coordinator incarnation.
  string node_uid = 6;
  string node_boot_id = 7;
  string claim_uid = 8;
  string allocation_fingerprint = 9;
  string topology_revision_uid = 10;
  string plan_digest = 11;
  string step_name = 12;
  string plugin_artifact_digest = 13;
  string plugin_contract_digest = 14;
  string trace_id = 15;                // Diagnostic; excluded from semantic digest.
  string workload_context_digest = 16; // Empty for Pod-independent root preparation.
}

message AllocationBinding {
  string driver = 1;
  string pool = 2;
  string device = 3;
  string share_id = 4;                 // Empty only for a non-shared allocation.
  string request_name = 5;
  string physical_device_id = 6;       // Driver-owned stable inventory identity.
  string inventory_contract_digest = 7;
  map<string, string> consumed_capacity = 8; // Qualified names -> quantity strings.
}

message NamespaceHandle {
  enum Availability {
    AVAILABILITY_UNSPECIFIED = 0;
    AVAILABLE = 1;
    MISSING = 2;
  }
  string id = 1;                       // Opaque driver-issued identity, not a user path.
  string shared_mount_path = 2;        // Approved mount visible to both processes.
  uint64 nsfs_device = 3;
  uint64 nsfs_inode = 4;
  string node_boot_id = 5;
  Availability availability = 6;
}

message SandboxIdentity {
  string pod_namespace = 1;
  string pod_name = 2;
  string pod_uid = 3;
  string sandbox_id = 4;
  string runtime_handler = 5;
  string attachment_id = 6;
  NamespaceHandle network_namespace = 7;
}

message ResourceHandle {
  string id = 1;
  string type = 2;                     // Versioned type, e.g. KernelInterface/v1.
  string owner_scope_id = 3;
  string owner_step = 4;
  string attachment_id = 5;
  string node_boot_id = 6;
  repeated string allocation_root_ids = 7;
  bytes description_json = 8;          // Validated against this type's schema.
}

message PortBinding {
  string name = 1;
  repeated ResourceHandle resources = 2; // Cardinality defined by descriptor.
}

message MutationGrant {
  string resource_id = 1;
  repeated string allowed_effects = 2;
  string authorization_token = 3;      // Bound to authenticated peer, scope and intent.
}

message PluginState {
  string format_version = 1;
  bytes data = 2;                      // Opaque, bounded recovery state; never user input.
}

message ContainerAccess {
  repeated string cdi_device_ids = 1;  // Produced/validated at root preparation only.
}

// Verified identities are distinct from untrusted projected metadata hints.
message ObjectIdentity {
  string api_version = 1;
  string kind = 2;
  string namespace = 3;
  string name = 4;
  string uid = 5;
}

message ResolvedNetworkIdentity {
  string binding_name = 1;
  string identity_uid = 2;
  string network_uid = 3;
  string interface_key = 4;
  string strategy = 5;
  ObjectIdentity subject = 6;
  string protected_slot = 7;
  string identity_policy_digest = 8;
  string authorization_ref = 9;         // Bounded reference; authority is peer/grant validated.
}

message WorkloadContext {
  string cluster_id = 1;
  string namespace_uid = 2;
  ObjectIdentity pod = 3;
  string pod_resource_version = 4;
  string service_account_name = 5;
  map<string, string> labels = 6;
  map<string, string> annotations = 7;
  repeated ObjectIdentity verified_owners = 8;
  repeated ResolvedNetworkIdentity identities = 9;
  string snapshot_id = 10;
  string projection_policy_digest = 11;
  string content_digest = 12;          // Effective projection and resolved identity semantics.
}

message AttachmentIntent {
  enum Phase {
    PHASE_UNSPECIFIED = 0;
    ACTIVE = 1;                       // Ordinary connection; no staged/migration assumption.
    STAGE = 2;
    ACTIVATE = 3;
  }
  Phase phase = 1;
  string base_connect_operation_id = 2; // Activate reuses the Stage effects in this scope.
  string handoff_uid = 3;              // Verified external handoff transaction, not a Pod hint.
  uint64 activation_epoch = 4;         // Must match backend and authenticated use grants.
}

message NodePrepareRequest {
  OperationContext context = 1;
  AllocationBinding allocation = 2;
  bytes config_json = 3;
  bytes compiled_root_requirements_json = 4;
  repeated MutationGrant grants = 5;
}

message NodePrepareResponse {
  string preparation_id = 1;
  PluginState prepared_state = 2;
  repeated PortBinding prepared_outputs = 3;
  ContainerAccess container_access = 4;
  bytes observed_contract_json = 5;
}

message NodeUnprepareRequest {
  OperationContext context = 1;
  string target_prepare_operation_id = 2;
  AllocationBinding allocation = 3;
  PluginState prepared_state = 4;       // May be empty after a lost response.
  repeated string cleanup_receipt_ids = 5;
  repeated MutationGrant grants = 6;
}

message ConnectNetworkRequest {
  OperationContext context = 1;
  SandboxIdentity sandbox = 2;
  AllocationBinding allocation = 3;    // Present only for an Allocation step.
  string preparation_id = 4;           // Present only for an Allocation step.
  PluginState prepared_state = 5;       // Present only for an Allocation step.
  repeated PortBinding inputs = 6;
  bytes config_json = 7;
  map<string, string> assigned_interface_names = 8;
  repeated MutationGrant grants = 9;
  WorkloadContext workload = 10;       // Frozen, projected by coordinator; not raw Pod JSON.
  AttachmentIntent intent = 11;        // Normal Active, or capability-gated Stage/Activate.
}

message ConnectNetworkResponse {
  repeated PortBinding outputs = 1;
  PluginState connection_state = 2;
  bytes observations_json = 3;
  enum ConnectionReadiness {
    READINESS_UNSPECIFIED = 0;
    READY = 1;
    STAGED = 2;
  }
  ConnectionReadiness readiness = 4;
}

message RemoveNetworkRequest {
  OperationContext context = 1;
  string target_connect_operation_id = 2;
  SandboxIdentity sandbox = 3;         // Namespace may explicitly be MISSING.
  PluginState connection_state = 4;    // May be empty after partial/lost Connect.
  repeated PortBinding previous_outputs = 5;
  repeated PortBinding original_inputs = 6;
  bytes original_config_json = 7;
  repeated string dependent_cleanup_receipt_ids = 8;
  repeated MutationGrant grants = 9;
  WorkloadContext workload = 10;       // Saved original context; Pod may already be deleted.
}

message CleanupResponse {
  enum Outcome {
    OUTCOME_UNSPECIFIED = 0;
    REMOVED = 1;
    ALREADY_ABSENT = 2;
  }
  Outcome outcome = 1;                 // OK means complete cleanup, never "queued".
  string receipt_id = 2;
  uint64 observed_desired_generation = 3;
}

message GetInfoRequest {}
message GetInfoResponse {
  string plugin_name = 1;
  string protocol_version = 2;
  string artifact_digest = 3;
  string contract_digest = 4;
  bytes descriptor_json = 5;
  repeated string readable_state_versions = 6;
  repeated string capabilities = 7;    // E.g. workload-context/v1, lease-inputs/v2, staged-activation/v1.
}

message InspectOperationRequest {
  string scope_id = 1;
  string operation_id = 2;
}
message InspectOperationResponse {
  enum State {
    STATE_UNSPECIFIED = 0;
    NOT_FOUND = 1;                     // Not proof of no effects after state loss.
    IN_PROGRESS = 2;
    SUCCEEDED = 3;
    CLEANED = 4;
    FAILED_WITH_POSSIBLE_EFFECTS = 5;
    QUARANTINED = 6;
  }
  State state = 1;
  uint64 observed_desired_generation = 2;
  string semantic_digest = 3;
  oneof saved_response {
    NodePrepareResponse prepared = 4;
    ConnectNetworkResponse connected = 5;
    CleanupResponse cleanup = 6;
  }
}

// Optional google.rpc.Status detail payload. Transport code alone never proves
// that side effects did not happen; an absent detail means UNKNOWN to the caller.
message OperationErrorDetail {
  enum Effects {
    EFFECTS_UNSPECIFIED = 0;
    NONE_VERIFIED = 1;
    PRESENT = 2;
    UNKNOWN = 3;
  }
  string reason = 1;
  string operation_id = 2;
  Effects effects = 3;
  bool retryable = 4;
  string recovery_action = 5;
}