DRA Network Device Discovery Design
Policy-driven network device discovery with DRA compatibility groups, local consumable capacity, and optional shared budgets
DRA Network Device Discovery and ResourceSlice Design
Status: Revised selected design; driver implementation has not started.
Revision: 2026-09-16 — replace counter-based mode exclusion with KEP 5963.
Scope: This document defines the proposedDeviceExposurePolicyAPI and its translation into Kubernetes DRA objects. Custom policy fields below are our design, not upstream Kubernetes fields.
Upstream baseline and feature boundaries
The implementation observations in this revision are pinned to Kubernetes commit a255808f6c0deb6e75ce031b5804e722306e46a8, inspected on 2026-09-16. A source-level feature version is not a promise that a particular OpenShift release enables or supports that feature. Validate the actual target distribution before deployment. S1 S2
| Concern | Upstream mechanism | Baseline / dependency |
|---|---|---|
| Device inventory and claims | resource.k8s.io/v1 | Use the current v1 object shape. |
| Counter-set associations | KEP 4815 / DRAPartitionableDevices | Inspected source: beta from 1.36, enabled by default. Do not label this dependency GA in 1.35. |
| Repeated allocations and local limits | KEP 5075 / DRAConsumableCapacity | Inspected source: beta from 1.36, enabled by default. |
| Incompatible network modes | KEP 5963 / DRADeviceCompatibilityGroups | Inspected source: alpha from 1.37, disabled by default; requires partitionable devices. |
supportedCNIs list attribute | KEP 5491 / DRAListTypeAttributes | Inspected source: alpha from 1.36, disabled by default. List values use strings. |
| Per-allocation shared budgets | KEP 5941 / PR #140084 | Optional follow-on work. Not required for the baseline exclusion design or reference manifests. |
Important implementation caveat: KEP 5963 describes allocation-time group snapshots, but the inspected allocator reconstructs compatibility from live ResourceSlices and allocated device identities. This revision targets that actual behavior. Do not assume claim status already preserves compatibility independently of the inventory. S3 S4
1. Overview and Design Principles
This document specifies how a DRA network driver discovers host network devices and publishes them as ResourceSlices for scheduler-aware allocation.
The Cardinal Rule: The Driver Is Generic
sysfs / netlink observations
|
v
raw device inventory + PF/VF relationships
|
v
DeviceExposurePolicy selection and validation
|
v
persona entries + scoped compatibility domains + local capacities
|
v
separate counter-bearing and device-bearing ResourceSlices
The discovery engine walks the host interfaces and records raw facts: interface name, MAC, MTU, PCI address, NUMA information, link speed when available, parent PF, VF index, and bridge membership. It does not infer CNI compatibility from plugin names.
The administrator’s policy declares which devices are exposed, which CNI plugins can use an entry, how much local capacity each allocation consumes, and which usage modes may coexist. The driver validates and compiles these declarations. The node-side execution path must enforce the compiled compatibility contract and verify the actual device identity before changing networking; being generic does not mean skipping safety checks.
A PF/VF relationship is a discovered topology fact. A declaration that PF passthrough must exclude all child allocations is a policy rule, not a universally assumed consequence of every operation on every PF. In particular, this design does not assert that every PF namespace move destroys its VFs.
Other Design Decisions
- Real interfaces, not synthetic allocation slots. A physical or software interface may have multiple persona entries when it supports different allocation semantics. A persona is an alternative representation, not additional hardware.
- One winning policy produces one persona.
allowMultipleAllocationsis the single source of truth for sharing that entry. Multiple CNI names in one policy are alternatives with identical allocation semantics. - Local capacity limits repeated allocations. A shared persona uses
allowMultipleAllocations: truepluscapacity. The examples reserve one unit per claim allocation, withvalidValues: ["1"]. - Compatibility groups express mode exclusion. Distinct incompatible personas consume from a common counter set and declare disjoint
compatibilityGroups. No local-capacity mirroring and no cross-draining are generated. - Independent constraints use independent domains. Interface mode and PF-family access are separate counter-set associations when both are needed. PF macvlan and child VFs can share a family group while PF macvlan and PF ipvlan remain incompatible in the interface domain.
- Bandwidth is a separate, optional budget. Link speed remains an attribute in the baseline. Do not publish a bandwidth guarantee or an invented fair-share policy just because a speed was discovered.
- Deny-by-default, without forgetting active allocations. Unmatched or excluded devices are not newly exposed. Existing active publication is handled through the drain protocol, not deleted immediately.
- CNI selection is explicit. DeviceClasses select the driver and test
supportedCNIsusing exact list membership. Advertising a CNI name is not proof that its invocation and controller integration work. - Stable identities and declarations. Active device names, pool membership, counter-set associations, and groups must survive rescan, policy changes, interface moves, and driver restarts until safely drained.
The upstream API separates compatibility predicates from local and shared capacity accounting. Our policy compiler preserves that separation. S1 S3 S4
2. DeviceExposurePolicy CRD Schema
2.1 Proposed Custom Resource Shape
This is an example custom resource, not an apiextensions.k8s.io/v1 CRD definition. The structural OpenAPI schema, defaults, CEL validation, status schema, and admission integration will be implemented after this design is accepted.
apiVersion: networking.dra.io/v1alpha1
kind: DeviceExposurePolicy
metadata:
name: pf0-macvlan
spec:
# Optional; defaults to all nodes.
nodeSelector:
matchLabels:
network.example.com/role: data
# 0-1000, default 100. Used only among expose policies for the same suffix.
priority: 200
selector:
cel: >-
device.attributes["dra.networking"].type == "pf" &&
device.attributes["dra.networking"].ifName == "enp3s0f0"
action: expose
exposure:
# One policy winner per suffix; no automatic splitting by CNI name.
deviceNameSuffix: "-macvlan"
allowMultipleAllocations: true
# Local capacity only. This does not create shared counter capacity.
capacity:
macvlans:
value: "64"
requestPolicy:
default: "1"
validValues: ["1"]
supportedCNIPlugins:
- name: macvlan
# Compiler validation / topology contract, not a Kubernetes field.
consumePerAllocation:
macvlans: "1"
# Proposed policy API. Each binding compiles to a consumesCounters entry.
compatibility:
- scope: interface
domain: interface-mode
groups: [macvlan]
- scope: sriovFamily
domain: pf-access
groups: [host-retained]
# Selection metadata only; do not override discovered or reserved attributes.
additionalAttributes:
dra.networking/physicalNetworkName: data-networkRemoved fields: exposure.exclusionGroup and supportedCNIPlugins[].exclusive are not part of the revised API. There is no implicit conversion of an old exclusion group into a compatibility group: equal compatibility group names mean compatible, whereas the old exclusion group linked conflicting personas. See the migration guide accompanying this document.
2.2 Field Semantics
| Field | Required / default | Meaning |
|---|---|---|
nodeSelector | Optional; all nodes | Nodes on which the policy participates in resolution. |
priority | Optional; 100 | Highest value wins within the same suffix; range 0-1000. |
selector.cel | Required | Boolean expression against the driver’s normalized discovered attributes. |
action | Optional; expose | expose creates desired publication; exclude prevents new exposure of the physical interface. |
exposure.deviceNameSuffix | Optional; empty string | Distinguishes persona policies for one interface. Published identities are pinned after creation. |
exposure.allowMultipleAllocations | Optional; false | Sharing permission for this DRA device entry, not for every alias of the underlying interface. |
exposure.capacity | Required for shared entries in this baseline | Local DeviceCapacity definitions. The compiler qualifies unqualified keys with dra.networking/. |
exposure.supportedCNIPlugins | Required for expose policies | Non-empty list of supported plugin names. All plugins on one entry must have identical allocation semantics. |
consumePerAllocation | Optional; use local defaults | A driver/topology consistency declaration. Baseline amounts must equal requestPolicy.default; the scheduler does not read this custom field. |
exposure.compatibility | Optional unless sharing an exclusion domain | Up to two domain bindings after compilation. Each binding has scope, domain, and non-empty groups. |
compatibility[].scope | Required | interface: this interface’s stable identity. sriovFamily: the PF root’s stable identity, shared with its VFs. |
compatibility[].domain | Required | Administrator-chosen DNS-label name for one independent compatibility constraint. |
compatibility[].groups | Required | One or two unique DNS-label tokens copied into the upstream compatibilityGroups field. |
exposure.additionalAttributes | Optional | Additional scalar selection attributes; string, boolean, or integer. Reserved/discovered keys cannot be overwritten. |
scope and domain resolve a comparison boundary; groups expresses compatibility within that boundary. Group names have no CNI interpretation in the driver.
2.3 Policy Resolution
For each discovered interface:
1. Select policies whose nodeSelector applies to this node.
2. Evaluate selector.cel against normalized discovered facts.
3. If any matching policy says exclude: desired exposure is empty.
4. If no expose policy matches: desired exposure is empty.
5. Otherwise group matching expose policies by deviceNameSuffix.
6. Pick highest priority per group; break ties by policy name.
7. Produce exactly one candidate persona per winner.
8. Resolve compatibility bindings against the discovered topology.
9. Validate the complete interface/family publication, including retained entries.
10. Reconcile desired publication with active allocations and the drain state.
Exclude priority is irrelevant: an exclusion always takes precedence over exposure. Two policies with different suffixes are not automatically mutually exclusive; their explicit compatibility bindings determine which combinations are allowed.
A selector compilation error rejects the policy. A selector evaluation error makes reconciliation fail closed for the affected inventory evaluation and records a diagnostic; it must not silently skip a potentially applicable exclusion policy. Previously committed active publication remains retained until its lifecycle is resolved.
2.4 Validation and Cross-Policy Checks
These are proposed driver requirements, in addition to upstream API validation:
- An exclusive entry (
allowMultipleAllocations: false) has no local request policy orconsumePerAllocationin the baseline. A shared entry has positive local capacity and an explicit default for each dimension. - Plugin-specific amounts are positive, reference existing capacity keys after qualification, and equal those dimensions’ defaults in the initial implementation. Different plugin costs require different persona policies and suitable compatibility guards; they are not inferred at scheduling time.
- When more than one persona aliases an interface, all aliases must participate in a common interface-scoped domain. Any exclusive persona must be incompatible there with every other alias. The compiler rejects ambiguous aliasing rather than assuming different names imply separate hardware.
- When an exposed PF persona excludes child use, every exposed child and every competing PF persona must participate in the same family-scoped domain. This is configured in policy; topology resolution alone does not invent group membership.
- A
sriovFamilybinding on an interface with no resolvable PF root is invalid. Bridges with indirect PF dependencies are not silently assigned a PF family. - Resolve bindings before enforcing limits: a device may consume from at most two counter sets; a counter-set consumption supports at most two compatibility groups. Reject a third independent domain instead of silently dropping it. S1
- Conflicting domain definitions, duplicate references, invalid names, overriding reserved attributes, or exceeding upstream attribute limits invalidate the affected compilation unit.
- All initial counter-set associations generated for exclusion have non-empty groups. An empty list must never be emitted as a wildcard.
Policy admission can validate shape and local consistency. Node reconciliation must validate topology-dependent and cross-policy rules. A future webhook must not claim it can statically validate every node’s current hardware inventory.
3. How Compatibility Domains Work: The Multi-Persona Pattern
The selected pattern remains separate DRA device entries referring to a common resource, but the enforcement mechanism changes to KEP 5963 compatibility groups.
3.1 Three Uses of the Same Compatibility Predicate
| Relationship | Comparison domain | Compatible group assignment |
|---|---|---|
| Macvlan vs ipvlan on one interface | Interface interface-mode | Macvlan: [macvlan]; ipvlan: [ipvlan]. |
| Shared macvlan vs exclusive host-device on one interface | Same interface domain | Macvlan: [macvlan]; passthrough: [passthrough]. |
| PF passthrough vs normal PF/VF use | PF family pf-access | PF passthrough: [pf-passthrough]; retained PF and VFs: [host-retained]. |
PF0 interface-mode: macvlan [macvlan] != passthrough [passthrough]
PF0 family pf-access: macvlan [host-retained]
VF0..VF7 [host-retained]
passthrough [pf-passthrough]
This allows PF macvlan and VFs together while excluding PF passthrough. Adding a PF ipvlan persona uses [ipvlan] in the interface domain and [host-retained] in the family domain. It remains compatible with VFs but not with PF macvlan.
A VF’s CNI can be exclusive for that VF entry while its family group is host-retained: the group describes access to the parent PF family, not whether that VF is shared.
3.2 Exact Upstream Compatibility Semantics
For each counter set, the scheduler maintains an intersection across the active devices’ group sets. A new candidate is compatible only if including its groups leaves a common group. A candidate associated with two counter sets must pass the predicate on both. Devices associated with different counter sets are not compared through this feature. S1 S3
{macvlan} intersection {macvlan} = {macvlan} -> compatible
{macvlan} intersection {ipvlan} = {} -> incompatiblePairwise overlap is insufficient. {a,b}, {b,c}, and {a,c} intersect pairwise, but the intersection of all three is empty. Empty/unset groups only coexist with other empty/unset groups on the same counter set; they do not mean “compatible with everything”. S3
Do not publish groups: [macvlan, shared] and groups: [ipvlan, shared] when the modes conflict: the common shared token would permit them together.
3.3 Local Capacity and the Non-Binding Membership Counter
The upstream field attaches to consumesCounters[]; a compatibility-only relationship still needs a valid counter-set association. This design generates one positive bookkeeping counter named members per compatibility domain. It is not a lock and not a per-Pod quota.
Let N be the number of distinct published persona/device identities referencing a domain, including retained active entries. Publish members = max(1, N) and a fixed consumption of members: "1" for each member. Do not shrink this bookkeeping total during an active drain transition. Normal allocation permits at most one active fixed charge per member identity, so the counter itself does not decide which modes may coexist. Compatibility groups do. S1 S4
For two macvlan/ipvlan personas, members: "2" intentionally leaves enough numerical capacity for both. Their disjoint groups still prevent coexistence.
| Quantity | Role | Consumption |
|---|---|---|
Local capacity.macvlans | Number of macvlan allocation units | Per allocation; examples use one. |
Domain members | Valid counter-set association with non-binding accounting | Fixed charge once per active device identity in the inspected baseline. |
| Optional shared bandwidth | A real shared resource budget | Per allocation only with the separately supported shared-consumable feature. |
A local quota applies to one persona identity. Publishing two compatible aliases with independent local quotas does not create a single aggregate hardware limit. Use one canonical persona for the same allocation contract, explicitly partition the administrator budget, or introduce a genuine shared budget in the follow-on profile. Do not represent the same 64 physical units as two independent 64-unit guarantees.
The old cross-drain model is removed. Local capacity does not automatically decrement a similarly named shared counter. In the old example, allocating macvlan drained the ipvlan mirror but left the macvlan mirror available, so ipvlan could consume that other mirror. Merely charging the fixed amount once did not create mutual exclusion.
3.4 Allocation and Release Lifecycle
With stable publication, the inspected implementation detects that a shared device remains allocated after one of its claims is released. It reconstructs the compatibility baseline using that active device and its live group declarations. The compatibility restriction is not attached to the first claim’s private counter snapshot. S4 S5
| Event | Macvlan allocations | IPvlan result |
|---|---|---|
| Initial | 0 | May allocate. |
| Allocate A | 1 | Blocked. |
| Allocate B on the same macvlan device | 2 | Blocked. |
| Release A; keep B | 1 | Still blocked. |
| Reconstruct scheduler state or restart the driver | 1 | Must remain blocked. |
| Release B and complete cleanup | 0 | May allocate. |
“Allocation released” means the scheduler no longer considers the ResourceClaim allocation active. Pod termination or a driver-local reference count reaching zero is not, by itself, sufficient evidence. One claim may serve multiple Pods; a capacity of 64 means 64 one-unit allocation results, not necessarily 64 Pods. Prefer per-Pod ResourceClaimTemplates where a per-Pod budget is intended. S7
This is not a claim that PR #140084’s static-counter lifetime issue has been fixed. The baseline does not depend on that PR. A combined build must get its own regression tests.
3.5 Feature-Disabled and Unsupported-Cluster Behavior
The baseline requires compatibility support whenever incompatible personas are exposed. Do not remove the group field and publish the same alternatives as a fallback. The API server can strip disabled feature fields; the scheduler can skip slices containing unsupported groups, and a skipped slice can make a pool incomplete. Neither behavior is a substitute for a driver capability contract. S3 S6
Installation must verify the selected API server and scheduler configuration. An explicit deployment capability setting plus API round-trip validation and a controlled scheduler conformance test is preferable to guessing from a version string or OpenAPI schema alone. A schema can describe fields whose feature gates are disabled.
For an unsupported cluster, a separately selected reduced profile may expose only one preselected mode per actual exclusion domain. It must not expose PF passthrough together with active child uses. Node-side checks remain required. This is a deliberate reduced feature profile, not an automatic downgrade of the same policy set.
4. How the Driver Uses Policies
The driver compiles discovered facts and winning policies into a complete desired pool model, validates it, then reconciles it against active allocations.
4.1 Attribute Translation
Discovered facts use the dra.networking domain, except explicitly standardized attributes such as device.k8s.io/pcieRoot. Values below are observations, not inferred CNI capabilities.
| Discovered fact | Published attribute | Discovery input |
|---|---|---|
| Interface name | dra.networking/ifName | Host interface inventory. |
| MAC address | dra.networking/mac | Netlink. |
| MTU | dra.networking/mtu | Netlink / sysfs. |
| Link speed, when known | dra.networking/linkSpeed | Driver/ethtool observation, documented in Mbps. |
| Operational state | dra.networking/operState | Netlink. |
| PCI BDF | dra.networking/pciAddress | Sysfs device relationship. |
| NUMA node | dra.networking/numaNode | PCI sysfs topology, when meaningful. |
| PCIe root | device.k8s.io/pcieRoot | PCI topology walk. |
| Vendor and product IDs | dra.networking/vendor, dra.networking/product | PCI identifiers. |
| Bound kernel driver | dra.networking/driver | Sysfs driver association. |
| Interface classification | dra.networking/type | Discovery classification: pf/vf/bridge/vlan/bond/nic. |
| RDMA association | dra.networking/rdma | Validated netdevice/RDMA relationship. |
| SR-IOV capability and configured VF count | dra.networking/sriovCapable, dra.networking/numVFs | PF sysfs facts. |
| Parent PF and VF index | dra.networking/pfName, dra.networking/vfIndex | physfn / virtfn relationships. |
| Bridge facts | dra.networking/bridgeName, bridgeType, vlanFiltering | Supported bridge discovery backend. |
| Master bridge | dra.networking/masterBridge | Master relationship, resolved to a bridge where appropriate. |
Unknown is not zero. The discovery contract must distinguish unavailable observations from legitimate values. For the bridge-selection examples, masterBridge is normalized to an empty string when the interface is known to have no bridge master. Selectors must guard attributes that may be absent.
Discovery must not forget a device solely because the driver moved it out of the host network namespace or changed its kernel binding. Preserve an allocation-aware inventory record and reconcile against node execution state.
4.2 Policy-Driven Fields
| Output | Source / transformation |
|---|---|
| Device name | Stable published identity for physical interface plus winning suffix. |
allowMultipleAllocations | Copy from policy. |
Local capacity | Policy quantities and request policies; qualify names consistently. |
dra.networking/supportedCNIs | Plugin names, serialized as {strings: [...]}. |
| Additional attributes | Validated non-reserved policy metadata, converted to typed attribute values. |
| Counter-set reference | Resolved (pool, scope root, domain) binding. |
compatibilityGroups | Copy the binding’s groups after validation. |
Domain members total | Number of referencing published identities, including retained entries. |
Fixed members consumption | One per referencing identity. |
attributes and capacity are direct fields under a v1 device entry. Do not wrap them in basic. Counter definitions and device entries belong in different ResourceSlice objects. S1
4.3 Topology Resolution and Pool Layout
One PF and its VFs form a pool. Standalone interfaces use standalone pools unless a separately supported topology relationship requires a common pool. Domain comparisons never cross pool boundaries.
For scope: interface, resolve the interface’s stable identity. For scope: sriovFamily, use the parent PF root for both the PF and all of its VFs. The driver discovers these relationships but does not invent mode groups.
A domain key is:
(driver, pool identity, resolved scope-root identity, administrator domain name)The counter-set name is a stable DNS-label encoding of this key within its pool. The implementation must define deterministic normalization, truncation and collision handling, for example a readable prefix with a stable hash suffix. Group names remain opaque; do not append each device’s unique identity to a shared group token.
The examples use short readable names such as pf0-interface-mode and pf0-family-pf-access. Production identity must not depend only on the current interface name or an unstable enumeration order. Persist the mapping used for active allocations. PCI BDF reuse after hardware replacement must not be mistaken for continuity of the old allocated hardware.
4.4 Translation Algorithm
INPUT:
discovered inventory, all relevant policies, deployment capabilities,
last committed publication, allocated claims, node execution journal
OUTPUT:
validated desired pools, status/diagnostics, staged publication operations
Phase 1 — Resolve policies and personas
Evaluate node and device selectors.
Apply exclude / deny-by-default rules.
Pick one winner per (physical interface, suffix).
Build candidate persona records without publishing them.
Phase 2 — Resolve compatibility domains
Resolve each interface or sriovFamily scope root.
Place family members in the same pool.
Resolve each (root, domain) into one counter-set identity.
Copy groups exactly; do not infer them from CNI names.
Require complete guards for aliases and any declared family-exclusive mode.
Phase 3 — Merge lifecycle state
Retain active entries missing from desired discovery/policy results.
Pin their identity, associations, groups and relevant allocation contract.
Mark incompatible policy changes or hardware changes as pending drain.
Do not publish replacement identities that can alias still-active hardware.
Phase 4 — Build resource accounting
Copy local capacities; never mirror them for exclusion.
For each domain, publish members >= its distinct retained/current members.
Add fixed members: 1 consumption to every referencing device entry.
Do not add automatic bandwidth or fair-share defaults.
Phase 5 — Validate the entire pool
Check names, field limits, no duplicate device or counter-set identities.
Check one consumption entry per counter set and at most two per device.
Check local quantities, group membership, aliases and feature capabilities.
Fail closed on invalid compilation; report the policy and domain involved.
Phase 6 — Pack and publish
Pack counter sets into counter-bearing slices.
Pack devices into device-bearing slices.
Set the same pool name, generation and total resourceSliceCount on all.
Reconcile through the upstream ResourceSlice controller where practical.
Retain old active declarations through transitions; reconcile partial writes.
Phase 7 — Observe and retire
Reconcile allocated claims and node execution state continuously.
Remove retired declarations only after the drain/cleanup conditions pass.
A pool.generation change allows readers to distinguish pool generations; it is not a multi-object transaction or an instruction to revoke existing claims. Publication must recover from incomplete generations and asynchronous watches. S1 S7
4.5 Optional Follow-On: Shared Bandwidth Budgets
The baseline publishes linkSpeed as a fact and makes no aggregate bandwidth reservation guarantee. The original automatic bandwidth and fair-share logic is removed from the baseline rather than presented as implemented functionality.
If bandwidth reservations are added later, use explicit administrator-supplied budgets and KEP 5941’s request-driven shared consumption. Every traffic consumer intended to participate in that budget must be included: accounting only for VFs while PF macvlan traffic is unaccounted for is not a whole-link reservation model.
The following is an experimental future wire-format fragment, not part of the initial DeviceExposurePolicy schema or baseline manifests. It follows PR #140084 at 51d78febb379ae41f7487a50b9718a4ede0e3f3c; that PR uses valueFrom.capacityName, not capacityKey. S8
# Counter-bearing ResourceSlice.spec fragment, future feature profile.
sharedCounters:
- name: pf0-family-pf-access
counters:
members:
value: "10"
bandwidth:
value: "100000" # Contract: Mbps, explicitly configured by admin.
requestPolicy:
default: "1000"
validRange:
min: "100"
max: "100000"
step: "100"# One member's device fragment, future feature profile.
consumesCounters:
- counterSet: pf0-family-pf-access
compatibilityGroups: [host-retained]
counters:
members:
value: "1"
bandwidth:
valueFrom:
capacityName: dra.networking/bandwidth# ResourceClaim request fragment for that future feature profile.
exactly:
deviceClassName: network-vf-pf0
capacity:
requests:
dra.networking/bandwidth: "2500"Reuse a compatible existing domain counter set where appropriate; do not add a third counter-set association to a device that already uses two. Define one authoritative budget specification and conflict resolution before implementing policy-to-budget translation. Capacity accounting alone also does not configure a hardware rate limit: scheduling reservation and enforcement are separate contracts.
The PR’s fixed-versus-per-share accounting, release behavior, feature transitions and combination with KEP 5963 must be retested on the actual combined code. Do not make baseline mode exclusion depend on a presumed fix in that PR.
4.6 ResourceSlice Packing Rules
In the inspected v1 API: S1
spec.devicesandspec.sharedCountersare mutually exclusive in one object.resourceSliceCountis the total number of ResourceSlices in the pool generation, including both kinds of slices.- A device has at most two
consumesCountersentries, one per referenced counter set; each has at most two compatibility group names. - One counter-bearing slice has at most eight counter sets, each with at most 32 counters.
- Use at most 64 devices per slice when counters, list attributes or other applicable advanced features are used. Validate the target API’s combined attribute/capacity and list-value limits too.
Do not duplicate a counter set in each device slice. Split the inventory into additional slices and update the total count when limits require it.
4.7 Capacity Requests and the Topology Contract
The scheduler does not execute our policies or choose an amount based on a CNI binary name. It reads local requestPolicy and any ResourceClaim capacity requests. Consequently, the baseline fixes each example’s default and allowed amount to one unit and requires all listed CNI alternatives to agree with that amount. S1 S7
A request for four units reserves four units; it does not automatically cause the execution engine to create four interfaces. Supporting variable quantities requires an explicit topology/execution contract and generated or user-authored capacity.requests. This is why the examples no longer advertise a one-to-four range while describing exactly one interface per allocation.
DeviceClass selectors select suitable devices but do not create plugin-specific capacity requests. The no-extra-capacity-fields application experience relies on consistent per-device defaults, not hidden scheduler interpretation of consumePerAllocation.
5. Examples
The nine scenarios from the original document are retained. Policy fields are the proposed API in Section 2; ResourceSlice examples use the inspected upstream v1 shape. Examples labelled fragment are not complete Kubernetes objects. Complete objects and every reference VF appear in Section 6.
Interface names, PCI addresses, capacities and speeds are illustrative inventory values from the reference scenario, not discovery defaults or vendor guarantees. The actual driver must use observed identities and administrator-selected limits.
5.1 Example 1: Simple SR-IOV VFs
Scenario: VFs under enp3s0f0 support sriov. Each VF is a normal, exclusively allocated DRA device. Normal use of the family is explicitly labelled host-retained.
DeviceExposurePolicy
apiVersion: networking.dra.io/v1alpha1
kind: DeviceExposurePolicy
metadata:
name: sriov-vfs-pf0
spec:
priority: 200
selector:
cel: device.attributes["dra.networking"].type == "vf" && device.attributes["dra.networking"].pfName
== "enp3s0f0"
action: expose
exposure:
allowMultipleAllocations: false
supportedCNIPlugins:
- name: sriov
compatibility:
- scope: sriovFamily
domain: pf-access
groups:
- host-retainedResulting ResourceSlice Fragments
This scenario exposes eight VFs and no PF persona, so the family domain has eight member identities. A future PF persona must use a compatible family declaration or be drained into a new configuration before publication.
Counter-bearing ResourceSlice.spec fragment:
sharedCounters:
- name: pf0-family-pf-access
counters:
members:
value: '8'Device-bearing ResourceSlice.spec fragment (same pool):
devices:
- name: enp3s0f0v0
attributes:
dra.networking/type:
string: vf
dra.networking/ifName:
string: enp3s0f0v0
dra.networking/pfName:
string: enp3s0f0
dra.networking/vfIndex:
int: 0
dra.networking/pciAddress:
string: '0000:03:00.2'
dra.networking/numaNode:
int: 0
dra.networking/rdma:
bool: true
dra.networking/vendor:
string: 15b3
dra.networking/product:
string: 101e
dra.networking/driver:
string: mlx5_core
device.k8s.io/pcieRoot:
string: pci0000:00
dra.networking/supportedCNIs:
strings:
- sriov
consumesCounters:
- counterSet: pf0-family-pf-access
compatibilityGroups:
- host-retained
counters:
members:
value: '1'The remaining seven VF entries use the same family group and each consume one member. VFs are independently allocatable because they are distinct devices with the same group. A VF is exclusive because allowMultipleAllocations is false/omitted, not because its CNI has a custom exclusive flag. There is no automatic numVFs + 1 lock counter.
5.2 Example 2: SR-IOV VFs with Multiple CNI Options
Scenario: A VF may be selected for sriov or host-device. The two names are execution alternatives for one exclusive entry, not permission to allocate the VF twice. The scenario assumes the execution path validates each plugin’s configuration requirements.
DeviceExposurePolicy
apiVersion: networking.dra.io/v1alpha1
kind: DeviceExposurePolicy
metadata:
name: pf0-vfs
spec:
priority: 200
selector:
cel: device.attributes["dra.networking"].type == "vf" && device.attributes["dra.networking"].pfName
== "enp3s0f0"
action: expose
exposure:
allowMultipleAllocations: false
supportedCNIPlugins:
- name: sriov
- name: host-device
compatibility:
- scope: sriovFamily
domain: pf-access
groups:
- host-retainedResulting ResourceSlice Fragments
Counter-bearing ResourceSlice.spec fragment:
sharedCounters:
- name: pf0-family-pf-access
counters:
members:
value: '8'Device-bearing ResourceSlice.spec fragment (same pool):
devices:
- name: enp3s0f0v0
attributes:
dra.networking/type:
string: vf
dra.networking/ifName:
string: enp3s0f0v0
dra.networking/pfName:
string: enp3s0f0
dra.networking/vfIndex:
int: 0
dra.networking/pciAddress:
string: '0000:03:00.2'
dra.networking/numaNode:
int: 0
dra.networking/rdma:
bool: true
dra.networking/vendor:
string: 15b3
dra.networking/product:
string: 101e
dra.networking/driver:
string: mlx5_core
device.k8s.io/pcieRoot:
string: pci0000:00
dra.networking/supportedCNIs:
strings:
- sriov
- host-device
consumesCounters:
- counterSet: pf0-family-pf-access
compatibilityGroups:
- host-retained
counters:
members:
value: '1'The supported-CNI list is serialized with strings. A generated DeviceClass can use device.attributes["dra.networking"].supportedCNIs.includes("sriov") or the corresponding host-device selection. .includes() is the DRA list-aware CEL helper; it does not reserve resources or invoke the plugin. S7
5.3 Example 3: PF as Macvlan Parent
Scenario: enp3s0f1 has no exposed VFs or competing persona in this example. It provides 64 one-unit macvlan allocations. This isolated, single-persona case does not need a compatibility domain.
DeviceExposurePolicy
apiVersion: networking.dra.io/v1alpha1
kind: DeviceExposurePolicy
metadata:
name: pf1-macvlan-parent
spec:
priority: 200
selector:
cel: device.attributes["dra.networking"].type == "pf" && device.attributes["dra.networking"].ifName
== "enp3s0f1"
action: expose
exposure:
allowMultipleAllocations: true
capacity:
macvlans:
value: '64'
requestPolicy:
default: '1'
validValues:
- '1'
supportedCNIPlugins:
- name: macvlan
consumePerAllocation:
macvlans: '1'Resulting Device Entry
name: enp3s0f1
allowMultipleAllocations: true
attributes:
dra.networking/type:
string: pf
dra.networking/ifName:
string: enp3s0f1
dra.networking/pciAddress:
string: '0000:03:00.1'
dra.networking/mac:
string: 04:3f:72:b0:d4:61
dra.networking/mtu:
int: 1500
dra.networking/linkSpeed:
int: 25000
dra.networking/numVFs:
int: 0
dra.networking/rdma:
bool: true
dra.networking/numaNode:
int: 0
dra.networking/vendor:
string: 15b3
dra.networking/product:
string: 101d
dra.networking/driver:
string: mlx5_core
device.k8s.io/pcieRoot:
string: pci0000:00
dra.networking/supportedCNIs:
strings:
- macvlan
capacity:
dra.networking/macvlans:
value: '64'
requestPolicy:
default: '1'
validValues:
- '1'Each claim allocation consumes one local unit. A 65th one-unit allocation cannot fit. The capacity is an administrator-defined quota, not an automatically discovered macvlan maximum. Adding another persona later requires a common interface domain and a safe migration of any already-active entry; do not silently attach new groups to existing allocations.
5.4 Example 4: PF with Macvlan AND Ipvlan
Scenario: The administrator declares macvlan and ipvlan incompatible on enp3s0f1, as motivated by the original document’s receive-handler conflict. The compiler does not contain that CNI-specific rule. Both modes have independent local limits of 64 allocations.
DeviceExposurePolicy — Macvlan Persona
apiVersion: networking.dra.io/v1alpha1
kind: DeviceExposurePolicy
metadata:
name: pf1-macvlan
spec:
priority: 200
selector:
cel: device.attributes["dra.networking"].type == "pf" && device.attributes["dra.networking"].ifName
== "enp3s0f1"
action: expose
exposure:
deviceNameSuffix: -macvlan
allowMultipleAllocations: true
capacity:
macvlans:
value: '64'
requestPolicy:
default: '1'
validValues:
- '1'
supportedCNIPlugins:
- name: macvlan
consumePerAllocation:
macvlans: '1'
compatibility:
- scope: interface
domain: interface-mode
groups:
- macvlanDeviceExposurePolicy — Ipvlan Persona
apiVersion: networking.dra.io/v1alpha1
kind: DeviceExposurePolicy
metadata:
name: pf1-ipvlan
spec:
priority: 200
selector:
cel: device.attributes["dra.networking"].type == "pf" && device.attributes["dra.networking"].ifName
== "enp3s0f1"
action: expose
exposure:
deviceNameSuffix: -ipvlan
allowMultipleAllocations: true
capacity:
ipvlans:
value: '64'
requestPolicy:
default: '1'
validValues:
- '1'
supportedCNIPlugins:
- name: ipvlan
consumePerAllocation:
ipvlans: '1'
compatibility:
- scope: interface
domain: interface-mode
groups:
- ipvlanResulting ResourceSlices — Two Objects in One Pool
apiVersion: resource.k8s.io/v1
kind: ResourceSlice
metadata:
name: worker-1-pf1-mode-example-counters
spec:
driver: dra.networking
pool:
name: worker-1-pf1-mode-example
generation: 1
resourceSliceCount: 2
nodeName: worker-1
sharedCounters:
- name: pf1-interface-mode
counters:
members:
value: '2'
---
apiVersion: resource.k8s.io/v1
kind: ResourceSlice
metadata:
name: worker-1-pf1-mode-example-devices
spec:
driver: dra.networking
pool:
name: worker-1-pf1-mode-example
generation: 1
resourceSliceCount: 2
nodeName: worker-1
devices:
- name: enp3s0f1-macvlan
allowMultipleAllocations: true
attributes:
dra.networking/type:
string: pf
dra.networking/ifName:
string: enp3s0f1
dra.networking/pciAddress:
string: '0000:03:00.1'
dra.networking/mac:
string: 04:3f:72:b0:d4:61
dra.networking/mtu:
int: 1500
dra.networking/linkSpeed:
int: 25000
dra.networking/numVFs:
int: 0
dra.networking/rdma:
bool: true
dra.networking/numaNode:
int: 0
dra.networking/vendor:
string: 15b3
dra.networking/product:
string: 101d
dra.networking/driver:
string: mlx5_core
device.k8s.io/pcieRoot:
string: pci0000:00
dra.networking/supportedCNIs:
strings:
- macvlan
capacity:
dra.networking/macvlans:
value: '64'
requestPolicy:
default: '1'
validValues:
- '1'
consumesCounters:
- counterSet: pf1-interface-mode
compatibilityGroups:
- macvlan
counters:
members:
value: '1'
- name: enp3s0f1-ipvlan
allowMultipleAllocations: true
attributes:
dra.networking/type:
string: pf
dra.networking/ifName:
string: enp3s0f1
dra.networking/pciAddress:
string: '0000:03:00.1'
dra.networking/mac:
string: 04:3f:72:b0:d4:61
dra.networking/mtu:
int: 1500
dra.networking/linkSpeed:
int: 25000
dra.networking/numVFs:
int: 0
dra.networking/rdma:
bool: true
dra.networking/numaNode:
int: 0
dra.networking/vendor:
string: 15b3
dra.networking/product:
string: 101d
dra.networking/driver:
string: mlx5_core
device.k8s.io/pcieRoot:
string: pci0000:00
dra.networking/supportedCNIs:
strings:
- ipvlan
capacity:
dra.networking/ipvlans:
value: '64'
requestPolicy:
default: '1'
validValues:
- '1'
consumesCounters:
- counterSet: pf1-interface-mode
compatibilityGroups:
- ipvlan
counters:
members:
value: '1'Walkthrough
| Event | Remaining numerical members | Local macvlan units remaining | Group state / outcome |
|---|---|---|---|
| Initial | 2 | 64 | No active mode. |
| First macvlan claim | 1 | 63 | Active group is macvlan. |
| Second macvlan claim | 1 | 62 | Same device; no new fixed member charge. |
| Try ipvlan | 1 | 62 | Numerical member capacity is sufficient, but groups do not intersect: rejected. |
| Release the first macvlan claim | 1 | 63 | Second claim keeps macvlan active; ipvlan remains rejected. |
| Release the last macvlan claim | 2 | 64 | No active mode; ipvlan can allocate. |
These are expected outcomes from the model, not a report of a cluster test performed for this document. The acceptance suite must verify them across successive allocation cycles and state reconstruction.
5.5 Example 5: PF with Macvlan AND Host-Device Passthrough
Scenario: The same interface supports shared macvlan and a normal exclusive passthrough persona. Keep separate entries to make allocation semantics explicit. It is not necessary to claim that capacity can never model a fixed full-capacity request; the selected design simply does not encode mode compatibility through plugin-specific capacity tricks.
DeviceExposurePolicies
apiVersion: networking.dra.io/v1alpha1
kind: DeviceExposurePolicy
metadata:
name: pf0-macvlan
spec:
priority: 200
selector:
cel: device.attributes["dra.networking"].type == "pf" && device.attributes["dra.networking"].ifName
== "enp3s0f0"
action: expose
exposure:
deviceNameSuffix: -macvlan
allowMultipleAllocations: true
capacity:
macvlans:
value: '64'
requestPolicy:
default: '1'
validValues:
- '1'
supportedCNIPlugins:
- name: macvlan
consumePerAllocation:
macvlans: '1'
compatibility:
- scope: interface
domain: interface-mode
groups:
- macvlan
---
apiVersion: networking.dra.io/v1alpha1
kind: DeviceExposurePolicy
metadata:
name: pf0-passthrough
spec:
priority: 200
selector:
cel: device.attributes["dra.networking"].type == "pf" && device.attributes["dra.networking"].ifName
== "enp3s0f0"
action: expose
exposure:
deviceNameSuffix: -passthrough
allowMultipleAllocations: false
supportedCNIPlugins:
- name: host-device
compatibility:
- scope: interface
domain: interface-mode
groups:
- passthroughResulting ResourceSlice Fragments
Counter-bearing ResourceSlice.spec fragment:
sharedCounters:
- name: pf0-interface-mode
counters:
members:
value: '2'Device-bearing ResourceSlice.spec fragment (same pool):
devices:
- name: enp3s0f0-macvlan
allowMultipleAllocations: true
attributes:
dra.networking/type:
string: pf
dra.networking/ifName:
string: enp3s0f0
dra.networking/pciAddress:
string: '0000:03:00.0'
dra.networking/mac:
string: 04:3f:72:b0:d4:60
dra.networking/mtu:
int: 1500
dra.networking/linkSpeed:
int: 100000
dra.networking/numVFs:
int: 0
dra.networking/rdma:
bool: true
dra.networking/numaNode:
int: 0
dra.networking/vendor:
string: 15b3
dra.networking/product:
string: 101d
dra.networking/driver:
string: mlx5_core
device.k8s.io/pcieRoot:
string: pci0000:00
dra.networking/supportedCNIs:
strings:
- macvlan
capacity:
dra.networking/macvlans:
value: '64'
requestPolicy:
default: '1'
validValues:
- '1'
consumesCounters:
- counterSet: pf0-interface-mode
compatibilityGroups:
- macvlan
counters:
members:
value: '1'
- name: enp3s0f0-passthrough
attributes:
dra.networking/type:
string: pf
dra.networking/ifName:
string: enp3s0f0
dra.networking/pciAddress:
string: '0000:03:00.0'
dra.networking/mac:
string: 04:3f:72:b0:d4:60
dra.networking/mtu:
int: 1500
dra.networking/linkSpeed:
int: 100000
dra.networking/numVFs:
int: 0
dra.networking/rdma:
bool: true
dra.networking/numaNode:
int: 0
dra.networking/vendor:
string: 15b3
dra.networking/product:
string: 101d
dra.networking/driver:
string: mlx5_core
device.k8s.io/pcieRoot:
string: pci0000:00
dra.networking/supportedCNIs:
strings:
- host-device
consumesCounters:
- counterSet: pf0-interface-mode
compatibilityGroups:
- passthrough
counters:
members:
value: '1'
Macvlan and passthrough use disjoint groups in the same domain. Passthrough does not drain a mirrored macvlan quota. The same exclusion applies in either allocation order. A second allocation of the passthrough entry is rejected by ordinary device exclusivity.
Multiple exclusive aliases are not safe just because each is individually exclusive. They must also be disjoint from each other in the shared interface domain. Prefer one exclusive entry with several valid CNI alternatives when the allocation contract is identical.
5.6 Example 6: PF with VFs + Macvlan + Host-Device
Scenario: PF0 has eight VFs. Shared PF macvlan and child VF use are permitted concurrently. PF passthrough is declared incompatible with either. This is a policy assumption for the reference node, not a blanket hardware claim.
DeviceExposurePolicies
apiVersion: networking.dra.io/v1alpha1
kind: DeviceExposurePolicy
metadata:
name: pf0-macvlan
spec:
priority: 200
selector:
cel: device.attributes["dra.networking"].type == "pf" && device.attributes["dra.networking"].ifName
== "enp3s0f0"
action: expose
exposure:
deviceNameSuffix: -macvlan
allowMultipleAllocations: true
capacity:
macvlans:
value: '64'
requestPolicy:
default: '1'
validValues:
- '1'
supportedCNIPlugins:
- name: macvlan
consumePerAllocation:
macvlans: '1'
compatibility:
- scope: interface
domain: interface-mode
groups:
- macvlan
- scope: sriovFamily
domain: pf-access
groups:
- host-retained
---
apiVersion: networking.dra.io/v1alpha1
kind: DeviceExposurePolicy
metadata:
name: pf0-passthrough
spec:
priority: 200
selector:
cel: device.attributes["dra.networking"].type == "pf" && device.attributes["dra.networking"].ifName
== "enp3s0f0"
action: expose
exposure:
deviceNameSuffix: -passthrough
allowMultipleAllocations: false
supportedCNIPlugins:
- name: host-device
compatibility:
- scope: interface
domain: interface-mode
groups:
- passthrough
- scope: sriovFamily
domain: pf-access
groups:
- pf-passthrough
---
apiVersion: networking.dra.io/v1alpha1
kind: DeviceExposurePolicy
metadata:
name: pf0-vfs
spec:
priority: 200
selector:
cel: device.attributes["dra.networking"].type == "vf" && device.attributes["dra.networking"].pfName
== "enp3s0f0"
action: expose
exposure:
allowMultipleAllocations: false
supportedCNIPlugins:
- name: sriov
- name: host-device
compatibility:
- scope: sriovFamily
domain: pf-access
groups:
- host-retainedResulting ResourceSlice Fragments
PF0 has ten published member identities in the family: two PF personas and eight VFs. Only the two PF personas belong to its interface-mode domain. The following fragment shows both personas and VF0; Section 6 includes all eight VFs in complete objects.
Counter-bearing ResourceSlice.spec fragment:
sharedCounters:
- name: pf0-interface-mode
counters:
members:
value: '2'
- name: pf0-family-pf-access
counters:
members:
value: '10'Device-bearing ResourceSlice.spec fragment (same pool):
devices:
- name: enp3s0f0-macvlan
allowMultipleAllocations: true
attributes:
dra.networking/type:
string: pf
dra.networking/ifName:
string: enp3s0f0
dra.networking/pciAddress:
string: '0000:03:00.0'
dra.networking/mac:
string: 04:3f:72:b0:d4:60
dra.networking/mtu:
int: 1500
dra.networking/linkSpeed:
int: 100000
dra.networking/numVFs:
int: 8
dra.networking/rdma:
bool: true
dra.networking/numaNode:
int: 0
dra.networking/vendor:
string: 15b3
dra.networking/product:
string: 101d
dra.networking/driver:
string: mlx5_core
device.k8s.io/pcieRoot:
string: pci0000:00
dra.networking/supportedCNIs:
strings:
- macvlan
capacity:
dra.networking/macvlans:
value: '64'
requestPolicy:
default: '1'
validValues:
- '1'
consumesCounters:
- counterSet: pf0-interface-mode
compatibilityGroups:
- macvlan
counters:
members:
value: '1'
- counterSet: pf0-family-pf-access
compatibilityGroups:
- host-retained
counters:
members:
value: '1'
- name: enp3s0f0-passthrough
attributes:
dra.networking/type:
string: pf
dra.networking/ifName:
string: enp3s0f0
dra.networking/pciAddress:
string: '0000:03:00.0'
dra.networking/mac:
string: 04:3f:72:b0:d4:60
dra.networking/mtu:
int: 1500
dra.networking/linkSpeed:
int: 100000
dra.networking/numVFs:
int: 8
dra.networking/rdma:
bool: true
dra.networking/numaNode:
int: 0
dra.networking/vendor:
string: 15b3
dra.networking/product:
string: 101d
dra.networking/driver:
string: mlx5_core
device.k8s.io/pcieRoot:
string: pci0000:00
dra.networking/supportedCNIs:
strings:
- host-device
consumesCounters:
- counterSet: pf0-interface-mode
compatibilityGroups:
- passthrough
counters:
members:
value: '1'
- counterSet: pf0-family-pf-access
compatibilityGroups:
- pf-passthrough
counters:
members:
value: '1'
- name: enp3s0f0v0
attributes:
dra.networking/type:
string: vf
dra.networking/ifName:
string: enp3s0f0v0
dra.networking/pfName:
string: enp3s0f0
dra.networking/vfIndex:
int: 0
dra.networking/pciAddress:
string: '0000:03:00.2'
dra.networking/numaNode:
int: 0
dra.networking/rdma:
bool: true
dra.networking/vendor:
string: 15b3
dra.networking/product:
string: 101e
dra.networking/driver:
string: mlx5_core
device.k8s.io/pcieRoot:
string: pci0000:00
dra.networking/supportedCNIs:
strings:
- sriov
- host-device
consumesCounters:
- counterSet: pf0-family-pf-access
compatibilityGroups:
- host-retained
counters:
members:
value: '1'Compatibility Matrix
| Candidate combination | Interface-mode result | PF-family result | Allowed? |
|---|---|---|---|
| PF macvlan + another share of PF macvlan | Same persona/group | Host-retained | Yes, within local capacity. |
| PF macvlan + VF0 | VF0 is outside interface domain | Both host-retained | Yes. |
| VF0 + VF1 | Neither uses PF interface domain | Both host-retained | Yes. |
| PF passthrough + PF macvlan | Disjoint | Disjoint | No. |
| PF passthrough + any VF | VF outside interface domain | Disjoint | No. |
| Another allocation of the same VF | Not relevant | Compatible group | No; the VF entry itself is exclusive. |
An optional PF ipvlan persona would use [ipvlan] on pf0-interface-mode and [host-retained] on pf0-family-pf-access. VFs need no extra macvlan/ipvlan token. This two-domain construction avoids treating an empty VF group as a wildcard or forcing every VF to enumerate all PF modes.
All exclusion in this example is group-based. The baseline has no exclusion-slots, no mirrored macvlan capacity and no bandwidth reservation counter. linkSpeed is still published as an attribute.
5.7 Example 7: VF as Bridge Uplink OR SR-IOV
Scenario: A free VF is directly allocatable. A VF pre-enslaved to a host bridge is not directly allocatable; the bridge is the shared device offered to Pods. Bridge ownership is host preconfiguration, not a DRA claim for the hidden VF.
DeviceExposurePolicies
apiVersion: networking.dra.io/v1alpha1
kind: DeviceExposurePolicy
metadata:
name: free-vfs-sriov
spec:
priority: 200
selector:
cel: device.attributes["dra.networking"].type == "vf" && device.attributes["dra.networking"].pfName
== "enp3s0f0" && device.attributes["dra.networking"].masterBridge == ""
action: expose
exposure:
allowMultipleAllocations: false
supportedCNIPlugins:
- name: sriov
- name: host-device
compatibility:
- scope: sriovFamily
domain: pf-access
groups:
- host-retained
---
apiVersion: networking.dra.io/v1alpha1
kind: DeviceExposurePolicy
metadata:
name: exclude-bridge-slave-vfs
spec:
priority: 900
selector:
cel: device.attributes["dra.networking"].type == "vf" && device.attributes["dra.networking"].masterBridge
!= ""
action: exclude
---
apiVersion: networking.dra.io/v1alpha1
kind: DeviceExposurePolicy
metadata:
name: bridge-sriov-data
spec:
priority: 200
selector:
cel: device.attributes["dra.networking"].type == "bridge" && device.attributes["dra.networking"].ifName
== "br-sriov-data"
action: expose
exposure:
allowMultipleAllocations: true
capacity:
ports:
value: '64'
requestPolicy:
default: '1'
validValues:
- '1'
supportedCNIPlugins:
- name: bridge
consumePerAllocation:
ports: '1'
---
apiVersion: networking.dra.io/v1alpha1
kind: DeviceExposurePolicy
metadata:
name: exclude-pf0-host-bridge-uplink
spec:
priority: 900
selector:
cel: device.attributes["dra.networking"].type == "pf" && device.attributes["dra.networking"].ifName
== "enp3s0f0"
action: excludeResulting Bridge Entry
name: br-sriov-data
allowMultipleAllocations: true
attributes:
dra.networking/type:
string: bridge
dra.networking/ifName:
string: br-sriov-data
dra.networking/bridgeName:
string: br-sriov-data
dra.networking/bridgeType:
string: linux
dra.networking/mtu:
int: 9000
dra.networking/supportedCNIs:
strings:
- bridge
capacity:
dra.networking/ports:
value: '64'
requestPolicy:
default: '1'
validValues:
- '1'The final policy deliberately excludes every PF0 persona in this conservative bridge profile. Do not combine it with PF passthrough while the host bridge depends on a child VF. Free sibling VFs can still be exposed under the family policy.
A hidden VF does not contribute a scheduler compatibility group or consume a counter merely because it is busy outside DRA. Removing one from an advertised quota is not a substitute for an active compatibility declaration. This design does not fabricate a DRA allocation for it. Supporting host reservations with simultaneous PF exposure requires a separate, explicit reservation model; it is outside the baseline.
The node execution path must recheck host ownership before a destructive operation. Host-network configuration must be coordinated with DRA: an administrator attaching a bridge to hardware already allocated to a workload is outside the scheduler’s view.
5.8 Example 8: Excluding Cluster CNI and Management Interfaces
Scenario: Cluster networking and management interfaces are protected from new workload allocations.
DeviceExposurePolicies
apiVersion: networking.dra.io/v1alpha1
kind: DeviceExposurePolicy
metadata:
name: exclude-cluster-cni
spec:
priority: 1000
selector:
cel: device.attributes["dra.networking"].ifName in ["br-int", "ovn-k8s-mp0", "breth0", "ovn-k8s-gw0"]
action: exclude
---
apiVersion: networking.dra.io/v1alpha1
kind: DeviceExposurePolicy
metadata:
name: exclude-management
spec:
priority: 1000
selector:
cel: device.attributes["dra.networking"].ifName == "eno1"
action: excludeUnallocated matching interfaces are absent from desired publication. Exclusions win regardless of priority. If a newly added exclusion matches an already-allocated device, transition it to drain handling and preserve the active declaration; do not interpret the policy as permission to destroy the running workload’s inventory or network state.
A hard emergency shutdown is an explicit administrative recovery action, not the ordinary reconciliation behavior.
5.9 Example 9: OVN-Kubernetes Networks via the Generic Driver
Scenario retained as a proposed integration, not an established capability. The policy and publication model can advertise ovn-k8s-cni-overlay as a supported CNI for an exclusive VF or a shared localnet uplink. That does not by itself establish that calling the binary is sufficient to provision an OVN network or accelerated VF.
DeviceExposurePolicy — Candidate Accelerated VF Integration
apiVersion: networking.dra.io/v1alpha1
kind: DeviceExposurePolicy
metadata:
name: ovnk-accelerated-vfs
spec:
priority: 200
selector:
cel: device.attributes["dra.networking"].type == "vf" && device.attributes["dra.networking"].pfName
== "enp4s0f0"
action: expose
exposure:
allowMultipleAllocations: false
supportedCNIPlugins:
- name: ovn-k8s-cni-overlay
compatibility:
- scope: sriovFamily
domain: pf-access
groups:
- host-retainedDeviceExposurePolicy — Candidate Localnet Uplink Integration
apiVersion: networking.dra.io/v1alpha1
kind: DeviceExposurePolicy
metadata:
name: ovnk-localnet-uplink
spec:
priority: 150
selector:
cel: device.attributes["dra.networking"].type == "pf" && device.attributes["dra.networking"].ifName
== "enp6s0f0"
action: expose
exposure:
deviceNameSuffix: -localnet
allowMultipleAllocations: true
capacity:
connections:
value: '128'
requestPolicy:
default: '1'
validValues:
- '1'
supportedCNIPlugins:
- name: ovn-k8s-cni-overlay
consumePerAllocation:
connections: '1'
additionalAttributes:
dra.networking/physicalNetworkName: localnet1Resulting Device Fragments
devices:
- name: enp4s0f0v0
attributes:
dra.networking/type:
string: vf
dra.networking/ifName:
string: enp4s0f0v0
dra.networking/pfName:
string: enp4s0f0
dra.networking/pciAddress:
string: '0000:04:00.2'
dra.networking/numaNode:
int: 0
device.k8s.io/pcieRoot:
string: pci0000:00
dra.networking/supportedCNIs:
strings:
- ovn-k8s-cni-overlay
consumesCounters:
- counterSet: pf4-family-pf-access
compatibilityGroups:
- host-retained
counters:
members:
value: '1'
- name: enp6s0f0-localnet
allowMultipleAllocations: true
attributes:
dra.networking/type:
string: pf
dra.networking/ifName:
string: enp6s0f0
dra.networking/numaNode:
int: 1
dra.networking/physicalNetworkName:
string: localnet1
device.k8s.io/pcieRoot:
string: pci0000:00
dra.networking/supportedCNIs:
strings:
- ovn-k8s-cni-overlay
capacity:
dra.networking/connections:
value: '128'
requestPolicy:
default: '1'
validValues:
- '1'
These entries belong in their respective pools. The VF’s pool must also publish the referenced family counter set with a non-binding membership total for its complete inventory; the standalone localnet example has no competing persona and needs no domain.
Network selection metadata such as physicalNetworkName belongs in published attributes. Runtime CNI JSON belongs to the companion NetworkTopology/execution contract. The integration work must separately verify network-controller provisioning, required annotations and identities, representor ownership, VF handoff, namespace timing, CNI arguments, chaining support, cleanup and restart behavior.
Until those checks pass, remove claims such as “no separate OVN-K DRA driver is needed” or “our driver invokes it just like any other plugin” from the implementation guarantees. The discovery policy remains generic; external control-plane integration is not automatically generic.
6. Complete Reference Node
6.1 Node Inventory
Node worker-1 uses the same illustrative inventory as the original design:
| Device | Interface | Type | PCI / topology | Speed | NUMA | Role |
|---|---|---|---|---|---|---|
| PF0 | enp3s0f0 | PF | 0000:03:00.0 | 100 GbE | 0 | Two personas, eight VFs. |
| PF1 | enp3s0f1 | PF | 0000:03:00.1 | 25 GbE | 0 | Passthrough persona, four VFs. |
| PF0 VFs | enp3s0f0v0 through enp3s0f0v7 | VF | Enumerated addresses below | Shared link | 0 | Exclusive sriov/host-device alternatives. |
| PF1 VFs | enp3s0f1v0 through enp3s0f1v3 | VF | Enumerated addresses below | Shared link | 0 | Exclusive sriov/host-device alternatives. |
| Data bridge | br-data | Linux bridge | No PCI identity in this example | Not specified | Not specified | Shared bridge ports. |
| Management | eno1 | NIC | Not exposed | Not relevant | Not specified | Excluded. |
| Cluster networking | br-int, ovn-k8s-mp0 | Infrastructure | Not exposed | Not relevant | Not specified | Excluded. |
Reference state assumes no unmodeled host-owned dependency on the PFs that are exposed for passthrough. The host-bridge/VF profile in Example 7 has different exclusions and must not be blindly merged into this reference profile.
6.2 Complete DeviceExposurePolicy Set
These eight proposed policy resources are scoped to the reference node. The policy API itself must be installed by the future implementation; these are not replacements for a CRD manifest.
apiVersion: networking.dra.io/v1alpha1
kind: DeviceExposurePolicy
metadata:
name: exclude-cluster-cni
spec:
nodeSelector:
matchLabels:
kubernetes.io/hostname: worker-1
priority: 1000
selector:
cel: device.attributes["dra.networking"].ifName in ["br-int", "ovn-k8s-mp0", "breth0", "ovn-k8s-gw0"]
action: exclude
---
apiVersion: networking.dra.io/v1alpha1
kind: DeviceExposurePolicy
metadata:
name: exclude-management
spec:
nodeSelector:
matchLabels:
kubernetes.io/hostname: worker-1
priority: 1000
selector:
cel: device.attributes["dra.networking"].ifName == "eno1"
action: exclude
---
apiVersion: networking.dra.io/v1alpha1
kind: DeviceExposurePolicy
metadata:
name: pf0-macvlan
spec:
nodeSelector:
matchLabels:
kubernetes.io/hostname: worker-1
priority: 200
selector:
cel: device.attributes["dra.networking"].type == "pf" && device.attributes["dra.networking"].ifName
== "enp3s0f0"
action: expose
exposure:
deviceNameSuffix: -macvlan
allowMultipleAllocations: true
capacity:
macvlans:
value: '64'
requestPolicy:
default: '1'
validValues:
- '1'
supportedCNIPlugins:
- name: macvlan
consumePerAllocation:
macvlans: '1'
compatibility:
- scope: interface
domain: interface-mode
groups:
- macvlan
- scope: sriovFamily
domain: pf-access
groups:
- host-retained
---
apiVersion: networking.dra.io/v1alpha1
kind: DeviceExposurePolicy
metadata:
name: pf0-passthrough
spec:
nodeSelector:
matchLabels:
kubernetes.io/hostname: worker-1
priority: 200
selector:
cel: device.attributes["dra.networking"].type == "pf" && device.attributes["dra.networking"].ifName
== "enp3s0f0"
action: expose
exposure:
deviceNameSuffix: -passthrough
allowMultipleAllocations: false
supportedCNIPlugins:
- name: host-device
compatibility:
- scope: interface
domain: interface-mode
groups:
- passthrough
- scope: sriovFamily
domain: pf-access
groups:
- pf-passthrough
---
apiVersion: networking.dra.io/v1alpha1
kind: DeviceExposurePolicy
metadata:
name: pf0-vfs
spec:
nodeSelector:
matchLabels:
kubernetes.io/hostname: worker-1
priority: 200
selector:
cel: device.attributes["dra.networking"].type == "vf" && device.attributes["dra.networking"].pfName
== "enp3s0f0"
action: expose
exposure:
allowMultipleAllocations: false
supportedCNIPlugins:
- name: sriov
- name: host-device
compatibility:
- scope: sriovFamily
domain: pf-access
groups:
- host-retained
---
apiVersion: networking.dra.io/v1alpha1
kind: DeviceExposurePolicy
metadata:
name: pf1-passthrough
spec:
nodeSelector:
matchLabels:
kubernetes.io/hostname: worker-1
priority: 200
selector:
cel: device.attributes["dra.networking"].type == "pf" && device.attributes["dra.networking"].ifName
== "enp3s0f1"
action: expose
exposure:
allowMultipleAllocations: false
supportedCNIPlugins:
- name: host-device
compatibility:
- scope: sriovFamily
domain: pf-access
groups:
- pf-passthrough
---
apiVersion: networking.dra.io/v1alpha1
kind: DeviceExposurePolicy
metadata:
name: pf1-vfs
spec:
nodeSelector:
matchLabels:
kubernetes.io/hostname: worker-1
priority: 200
selector:
cel: device.attributes["dra.networking"].type == "vf" && device.attributes["dra.networking"].pfName
== "enp3s0f1"
action: expose
exposure:
allowMultipleAllocations: false
supportedCNIPlugins:
- name: sriov
- name: host-device
compatibility:
- scope: sriovFamily
domain: pf-access
groups:
- host-retained
---
apiVersion: networking.dra.io/v1alpha1
kind: DeviceExposurePolicy
metadata:
name: bridge-br-data
spec:
nodeSelector:
matchLabels:
kubernetes.io/hostname: worker-1
priority: 150
selector:
cel: device.attributes["dra.networking"].type == "bridge" && device.attributes["dra.networking"].ifName
== "br-data"
action: expose
exposure:
allowMultipleAllocations: true
capacity:
ports:
value: '64'
requestPolicy:
default: '1'
validValues:
- '1'
supportedCNIPlugins:
- name: bridge
consumePerAllocation:
ports: '1'6.3 Resulting ResourceSlices
There are five ResourceSlice objects, not three. Each PF pool has one counter-bearing slice and one device-bearing slice. The standalone bridge needs only a device-bearing slice. Every VF is included; there are no omitted placeholder devices in this reference output.
PF0 Pool — Counter Slice and Device Slice
apiVersion: resource.k8s.io/v1
kind: ResourceSlice
metadata:
name: worker-1-sriov-pf0-counters
spec:
driver: dra.networking
pool:
name: worker-1-sriov-pf0
generation: 1
resourceSliceCount: 2
nodeName: worker-1
sharedCounters:
- name: pf0-interface-mode
counters:
members:
value: '2'
- name: pf0-family-pf-access
counters:
members:
value: '10'
---
apiVersion: resource.k8s.io/v1
kind: ResourceSlice
metadata:
name: worker-1-sriov-pf0-devices
spec:
driver: dra.networking
pool:
name: worker-1-sriov-pf0
generation: 1
resourceSliceCount: 2
nodeName: worker-1
devices:
- name: enp3s0f0-macvlan
allowMultipleAllocations: true
attributes:
dra.networking/type:
string: pf
dra.networking/ifName:
string: enp3s0f0
dra.networking/pciAddress:
string: '0000:03:00.0'
dra.networking/mac:
string: 04:3f:72:b0:d4:60
dra.networking/mtu:
int: 1500
dra.networking/linkSpeed:
int: 100000
dra.networking/numVFs:
int: 8
dra.networking/rdma:
bool: true
dra.networking/numaNode:
int: 0
dra.networking/vendor:
string: 15b3
dra.networking/product:
string: 101d
dra.networking/driver:
string: mlx5_core
device.k8s.io/pcieRoot:
string: pci0000:00
dra.networking/supportedCNIs:
strings:
- macvlan
capacity:
dra.networking/macvlans:
value: '64'
requestPolicy:
default: '1'
validValues:
- '1'
consumesCounters:
- counterSet: pf0-interface-mode
compatibilityGroups:
- macvlan
counters:
members:
value: '1'
- counterSet: pf0-family-pf-access
compatibilityGroups:
- host-retained
counters:
members:
value: '1'
- name: enp3s0f0-passthrough
attributes:
dra.networking/type:
string: pf
dra.networking/ifName:
string: enp3s0f0
dra.networking/pciAddress:
string: '0000:03:00.0'
dra.networking/mac:
string: 04:3f:72:b0:d4:60
dra.networking/mtu:
int: 1500
dra.networking/linkSpeed:
int: 100000
dra.networking/numVFs:
int: 8
dra.networking/rdma:
bool: true
dra.networking/numaNode:
int: 0
dra.networking/vendor:
string: 15b3
dra.networking/product:
string: 101d
dra.networking/driver:
string: mlx5_core
device.k8s.io/pcieRoot:
string: pci0000:00
dra.networking/supportedCNIs:
strings:
- host-device
consumesCounters:
- counterSet: pf0-interface-mode
compatibilityGroups:
- passthrough
counters:
members:
value: '1'
- counterSet: pf0-family-pf-access
compatibilityGroups:
- pf-passthrough
counters:
members:
value: '1'
- name: enp3s0f0v0
attributes:
dra.networking/type:
string: vf
dra.networking/ifName:
string: enp3s0f0v0
dra.networking/pfName:
string: enp3s0f0
dra.networking/vfIndex:
int: 0
dra.networking/pciAddress:
string: '0000:03:00.2'
dra.networking/numaNode:
int: 0
dra.networking/rdma:
bool: true
dra.networking/vendor:
string: 15b3
dra.networking/product:
string: 101e
dra.networking/driver:
string: mlx5_core
device.k8s.io/pcieRoot:
string: pci0000:00
dra.networking/supportedCNIs:
strings:
- sriov
- host-device
consumesCounters:
- counterSet: pf0-family-pf-access
compatibilityGroups:
- host-retained
counters:
members:
value: '1'
- name: enp3s0f0v1
attributes:
dra.networking/type:
string: vf
dra.networking/ifName:
string: enp3s0f0v1
dra.networking/pfName:
string: enp3s0f0
dra.networking/vfIndex:
int: 1
dra.networking/pciAddress:
string: '0000:03:00.3'
dra.networking/numaNode:
int: 0
dra.networking/rdma:
bool: true
dra.networking/vendor:
string: 15b3
dra.networking/product:
string: 101e
dra.networking/driver:
string: mlx5_core
device.k8s.io/pcieRoot:
string: pci0000:00
dra.networking/supportedCNIs:
strings:
- sriov
- host-device
consumesCounters:
- counterSet: pf0-family-pf-access
compatibilityGroups:
- host-retained
counters:
members:
value: '1'
- name: enp3s0f0v2
attributes:
dra.networking/type:
string: vf
dra.networking/ifName:
string: enp3s0f0v2
dra.networking/pfName:
string: enp3s0f0
dra.networking/vfIndex:
int: 2
dra.networking/pciAddress:
string: '0000:03:00.4'
dra.networking/numaNode:
int: 0
dra.networking/rdma:
bool: true
dra.networking/vendor:
string: 15b3
dra.networking/product:
string: 101e
dra.networking/driver:
string: mlx5_core
device.k8s.io/pcieRoot:
string: pci0000:00
dra.networking/supportedCNIs:
strings:
- sriov
- host-device
consumesCounters:
- counterSet: pf0-family-pf-access
compatibilityGroups:
- host-retained
counters:
members:
value: '1'
- name: enp3s0f0v3
attributes:
dra.networking/type:
string: vf
dra.networking/ifName:
string: enp3s0f0v3
dra.networking/pfName:
string: enp3s0f0
dra.networking/vfIndex:
int: 3
dra.networking/pciAddress:
string: '0000:03:00.5'
dra.networking/numaNode:
int: 0
dra.networking/rdma:
bool: true
dra.networking/vendor:
string: 15b3
dra.networking/product:
string: 101e
dra.networking/driver:
string: mlx5_core
device.k8s.io/pcieRoot:
string: pci0000:00
dra.networking/supportedCNIs:
strings:
- sriov
- host-device
consumesCounters:
- counterSet: pf0-family-pf-access
compatibilityGroups:
- host-retained
counters:
members:
value: '1'
- name: enp3s0f0v4
attributes:
dra.networking/type:
string: vf
dra.networking/ifName:
string: enp3s0f0v4
dra.networking/pfName:
string: enp3s0f0
dra.networking/vfIndex:
int: 4
dra.networking/pciAddress:
string: '0000:03:00.6'
dra.networking/numaNode:
int: 0
dra.networking/rdma:
bool: true
dra.networking/vendor:
string: 15b3
dra.networking/product:
string: 101e
dra.networking/driver:
string: mlx5_core
device.k8s.io/pcieRoot:
string: pci0000:00
dra.networking/supportedCNIs:
strings:
- sriov
- host-device
consumesCounters:
- counterSet: pf0-family-pf-access
compatibilityGroups:
- host-retained
counters:
members:
value: '1'
- name: enp3s0f0v5
attributes:
dra.networking/type:
string: vf
dra.networking/ifName:
string: enp3s0f0v5
dra.networking/pfName:
string: enp3s0f0
dra.networking/vfIndex:
int: 5
dra.networking/pciAddress:
string: '0000:03:00.7'
dra.networking/numaNode:
int: 0
dra.networking/rdma:
bool: true
dra.networking/vendor:
string: 15b3
dra.networking/product:
string: 101e
dra.networking/driver:
string: mlx5_core
device.k8s.io/pcieRoot:
string: pci0000:00
dra.networking/supportedCNIs:
strings:
- sriov
- host-device
consumesCounters:
- counterSet: pf0-family-pf-access
compatibilityGroups:
- host-retained
counters:
members:
value: '1'
- name: enp3s0f0v6
attributes:
dra.networking/type:
string: vf
dra.networking/ifName:
string: enp3s0f0v6
dra.networking/pfName:
string: enp3s0f0
dra.networking/vfIndex:
int: 6
dra.networking/pciAddress:
string: '0000:03:01.0'
dra.networking/numaNode:
int: 0
dra.networking/rdma:
bool: true
dra.networking/vendor:
string: 15b3
dra.networking/product:
string: 101e
dra.networking/driver:
string: mlx5_core
device.k8s.io/pcieRoot:
string: pci0000:00
dra.networking/supportedCNIs:
strings:
- sriov
- host-device
consumesCounters:
- counterSet: pf0-family-pf-access
compatibilityGroups:
- host-retained
counters:
members:
value: '1'
- name: enp3s0f0v7
attributes:
dra.networking/type:
string: vf
dra.networking/ifName:
string: enp3s0f0v7
dra.networking/pfName:
string: enp3s0f0
dra.networking/vfIndex:
int: 7
dra.networking/pciAddress:
string: '0000:03:01.1'
dra.networking/numaNode:
int: 0
dra.networking/rdma:
bool: true
dra.networking/vendor:
string: 15b3
dra.networking/product:
string: 101e
dra.networking/driver:
string: mlx5_core
device.k8s.io/pcieRoot:
string: pci0000:00
dra.networking/supportedCNIs:
strings:
- sriov
- host-device
consumesCounters:
- counterSet: pf0-family-pf-access
compatibilityGroups:
- host-retained
counters:
members:
value: '1'PF1 Pool — Counter Slice and Device Slice
apiVersion: resource.k8s.io/v1
kind: ResourceSlice
metadata:
name: worker-1-sriov-pf1-counters
spec:
driver: dra.networking
pool:
name: worker-1-sriov-pf1
generation: 1
resourceSliceCount: 2
nodeName: worker-1
sharedCounters:
- name: pf1-family-pf-access
counters:
members:
value: '5'
---
apiVersion: resource.k8s.io/v1
kind: ResourceSlice
metadata:
name: worker-1-sriov-pf1-devices
spec:
driver: dra.networking
pool:
name: worker-1-sriov-pf1
generation: 1
resourceSliceCount: 2
nodeName: worker-1
devices:
- name: enp3s0f1
attributes:
dra.networking/type:
string: pf
dra.networking/ifName:
string: enp3s0f1
dra.networking/pciAddress:
string: '0000:03:00.1'
dra.networking/mac:
string: 04:3f:72:b0:d4:61
dra.networking/mtu:
int: 1500
dra.networking/linkSpeed:
int: 25000
dra.networking/numVFs:
int: 4
dra.networking/rdma:
bool: true
dra.networking/numaNode:
int: 0
dra.networking/vendor:
string: 15b3
dra.networking/product:
string: 101d
dra.networking/driver:
string: mlx5_core
device.k8s.io/pcieRoot:
string: pci0000:00
dra.networking/supportedCNIs:
strings:
- host-device
consumesCounters:
- counterSet: pf1-family-pf-access
compatibilityGroups:
- pf-passthrough
counters:
members:
value: '1'
- name: enp3s0f1v0
attributes:
dra.networking/type:
string: vf
dra.networking/ifName:
string: enp3s0f1v0
dra.networking/pfName:
string: enp3s0f1
dra.networking/vfIndex:
int: 0
dra.networking/pciAddress:
string: '0000:03:01.2'
dra.networking/numaNode:
int: 0
dra.networking/rdma:
bool: true
dra.networking/vendor:
string: 15b3
dra.networking/product:
string: 101e
dra.networking/driver:
string: mlx5_core
device.k8s.io/pcieRoot:
string: pci0000:00
dra.networking/supportedCNIs:
strings:
- sriov
- host-device
consumesCounters:
- counterSet: pf1-family-pf-access
compatibilityGroups:
- host-retained
counters:
members:
value: '1'
- name: enp3s0f1v1
attributes:
dra.networking/type:
string: vf
dra.networking/ifName:
string: enp3s0f1v1
dra.networking/pfName:
string: enp3s0f1
dra.networking/vfIndex:
int: 1
dra.networking/pciAddress:
string: '0000:03:01.3'
dra.networking/numaNode:
int: 0
dra.networking/rdma:
bool: true
dra.networking/vendor:
string: 15b3
dra.networking/product:
string: 101e
dra.networking/driver:
string: mlx5_core
device.k8s.io/pcieRoot:
string: pci0000:00
dra.networking/supportedCNIs:
strings:
- sriov
- host-device
consumesCounters:
- counterSet: pf1-family-pf-access
compatibilityGroups:
- host-retained
counters:
members:
value: '1'
- name: enp3s0f1v2
attributes:
dra.networking/type:
string: vf
dra.networking/ifName:
string: enp3s0f1v2
dra.networking/pfName:
string: enp3s0f1
dra.networking/vfIndex:
int: 2
dra.networking/pciAddress:
string: '0000:03:01.4'
dra.networking/numaNode:
int: 0
dra.networking/rdma:
bool: true
dra.networking/vendor:
string: 15b3
dra.networking/product:
string: 101e
dra.networking/driver:
string: mlx5_core
device.k8s.io/pcieRoot:
string: pci0000:00
dra.networking/supportedCNIs:
strings:
- sriov
- host-device
consumesCounters:
- counterSet: pf1-family-pf-access
compatibilityGroups:
- host-retained
counters:
members:
value: '1'
- name: enp3s0f1v3
attributes:
dra.networking/type:
string: vf
dra.networking/ifName:
string: enp3s0f1v3
dra.networking/pfName:
string: enp3s0f1
dra.networking/vfIndex:
int: 3
dra.networking/pciAddress:
string: '0000:03:01.5'
dra.networking/numaNode:
int: 0
dra.networking/rdma:
bool: true
dra.networking/vendor:
string: 15b3
dra.networking/product:
string: 101e
dra.networking/driver:
string: mlx5_core
device.k8s.io/pcieRoot:
string: pci0000:00
dra.networking/supportedCNIs:
strings:
- sriov
- host-device
consumesCounters:
- counterSet: pf1-family-pf-access
compatibilityGroups:
- host-retained
counters:
members:
value: '1'Standalone Bridge Pool
apiVersion: resource.k8s.io/v1
kind: ResourceSlice
metadata:
name: worker-1-bridge-br-data
spec:
driver: dra.networking
pool:
name: worker-1-bridge-br-data
generation: 1
resourceSliceCount: 1
nodeName: worker-1
devices:
- name: br-data
allowMultipleAllocations: true
attributes:
dra.networking/type:
string: bridge
dra.networking/ifName:
string: br-data
dra.networking/bridgeName:
string: br-data
dra.networking/bridgeType:
string: linux
dra.networking/mtu:
int: 9000
dra.networking/vlanFiltering:
bool: true
dra.networking/supportedCNIs:
strings:
- bridge
capacity:
dra.networking/ports:
value: '64'
requestPolicy:
default: '1'
validValues:
- '1'6.4 Summary
| Pool | ResourceSlice objects | Device entries | Compatibility domains |
|---|---|---|---|
worker-1-sriov-pf0 | 2 | 10: two PF personas and eight VFs | pf0-interface-mode has members=2; pf0-family-pf-access has members=10. |
worker-1-sriov-pf1 | 2 | 5: one PF persona and four VFs | pf1-family-pf-access has members=5. |
worker-1-bridge-br-data | 1 | 1 | None. |
| Total | 5 | 16 | Three counter sets across the two PF counter slices. |
Not published: management and cluster-network interfaces selected by exclusion policies. Speeds remain attributes; there is no bandwidth reservation in these baseline manifests.
The numerical member totals accommodate all published identities, including incompatible alternatives. They are deliberately not the source of mutual exclusion.
6.5 App Developer Experience
The platform administers exposure policy and DeviceClasses. Applications reference a class, normally through a ResourceClaimTemplate. The companion NetworkTopology controller may generate those classes, but the discovery implementation does not rely on unverified hidden capacity injection.
The following DeviceClass shows the actual selection logic for the reference PF0 macvlan entry. The VF class is included for the optional bandwidth fragment and future examples. Driver and attribute domains are illustrative; a production project must select its owned driver domain consistently.
apiVersion: resource.k8s.io/v1
kind: DeviceClass
metadata:
name: network-macvlan-pf0
spec:
selectors:
- cel:
expression: device.driver == "dra.networking" && device.allowMultipleAllocations == true && device.attributes["dra.networking"].type
== "pf" && device.attributes["dra.networking"].ifName == "enp3s0f0" && device.attributes["dra.networking"].supportedCNIs.includes("macvlan")
---
apiVersion: resource.k8s.io/v1
kind: DeviceClass
metadata:
name: network-vf-pf0
spec:
selectors:
- cel:
expression: device.driver == "dra.networking" && device.allowMultipleAllocations == false && device.attributes["dra.networking"].type
== "vf" && device.attributes["dra.networking"].pfName == "enp3s0f0" && device.attributes["dra.networking"].supportedCNIs.includes("sriov")
---
apiVersion: resource.k8s.io/v1
kind: ResourceClaimTemplate
metadata:
name: my-app-network
namespace: default
spec:
spec:
devices:
requests:
- name: net
exactly:
deviceClassName: network-macvlan-pf0
allocationMode: ExactCount
count: 1No explicit capacity request is needed for this baseline: the selected device’s local policy reserves one macvlan unit. networkTopologyRef and CNI execution configuration are driver/topology integration concerns, not new standard fields on a ResourceClaim. The selectors above demonstrate the allocation layer only; they do not provide the complete CNI execution configuration from the companion proposal. S7 S9
Claims that combine two VF requests, as in the original bonded-RDMA application example, follow the same class-reference pattern. Their network topology and distinct-device/topology constraints must be generated and tested by the companion controller rather than assumed from request names.
7. Decisions and Remaining Questions
The original consume-once question is no longer a blocker: mode exclusion uses compatibility groups. Remaining implementation choices are recorded separately from resolved API behavior.
| Topic | Decision / remaining work |
|---|---|
| Mode exclusion | Resolved: KEP 5963 with explicit scoped group bindings. No mirrored-capacity cross-drain. |
| PF/VF interaction | Explicit family policy. The reference permits host-retained PF macvlan plus VFs and excludes PF passthrough. Hardware support and cleanup remain platform test requirements. |
| Macvlan/bridge capacity discovery | Administrator supplied. The driver does not invent maximum counts. |
| Default local consumption | Explicit local request policies provide one-unit defaults. Validate round-trip behavior and actual allocator results in tests. |
| Shared-allocation identity | Log the claim’s metadata UID, request, driver/pool/device and result shareID when present. Do not claim that a claim UID is a field inside every AllocationResult. |
| Cross-pool associations | Not supported by this design. A common compatibility domain must be placed in one pool. |
| Dynamic VF count / device disappearance | Needs allocation-aware drain and identity retention. Generation changes alone do not make hardware reconfiguration safe. |
| Bridge creation | Discovery-only in the initial implementation. Host topology changes require coordination. |
| List-attribute fallback | Initial list-based profile requires its feature gate. No substring-based comma-separated fallback. A reduced exact-scalar profile would need a separate schema/selector contract. |
| Policy validation | Local admission validation plus node-level whole-pool checks. Normal exclusive policies without capacity are valid. |
| Shared bandwidth | Follow-on feature requiring an explicit budget policy schema, target PR/version integration and accounting/enforcement tests. |
| OVN-K integration | Proposed example only; requires end-to-end control-plane and execution validation. |
| Exact stable identity encoding | Define persisted keys, replacement detection and hashed DNS-label encoding before coding publication. |
| Deployment target | Pin Kubernetes/client libraries and determine supported OpenShift feature configuration. Do not infer support from generic Kubernetes version numbers. |
8. Lifecycle, Reconciliation and Safety Contract
8.1 Why Active Publication Must Be Preserved
The inspected KEP 5963 implementation reads groups from live device declarations. Removing an allocated device from the inventory removes its contribution to the compatibility baseline. Changing its groups changes the baseline used for future allocations. These are implementation facts, not behavior this driver should rely on to switch a mode in use. S3 S4
The driver must pin the active publication contract: device and pool identities, domain associations, group tokens, sharing semantics, CNI alternatives and relevant execution/capacity configuration. Store enough committed publication and node execution state to recover this contract after restart; a fresh scan of the host namespace is insufficient when devices have been moved.
Policy state is desired state. It is not permission to mutate the meaning of an already allocated device. A new desired contract becomes active only after a safe transition.
8.2 Drain Protocol
A change that removes a persona, changes its groups, switches sharing semantics, changes a scope root, renames an active identity, or replaces physical hardware enters a drain workflow:
- Record the desired change and keep the previous active declaration.
- Prevent new admissions into the affected transition through the deployment’s maintenance/admission mechanism; optionally mark retained devices
NoSchedulewithout evicting running Pods. - Wait for allocated claims, in-flight preparation and node cleanup to reconcile. Recheck before committing a new generation.
- Publish the new complete model and retire the old one only when safe.
A NoSchedule taint can be tolerated by a claim. It is an operational aid, not a security boundary or proof that there are no in-flight allocations. Preventing bypass requires appropriate admission/RBAC policy, and scheduler/watch races still require node-side verification. A poll observing zero claims is not an atomic lock against a concurrent allocator.
For a policy exclusion on an active interface, report PendingDrain rather than immediately deleting the device. For unexpected hardware disappearance, retain the allocation record, fail new preparation clearly, and require recovery; do not expose a replacement under an identity still in use.
8.3 Node-Side Enforcement
The execution path must maintain a persistent, idempotent allocation journal and serialize conflicting work by resolved domain. Acquire multiple domain locks in deterministic order. Validate the actual physical identity and the pinned contract before CNI or device operations, including the actual networking hook if it occurs after NodePrepareResources.
Preparation, retries, cleanup failures, kubelet reconnect and process restarts must not release a domain prematurely. A local cleanup completion does not remove scheduler allocation state, and scheduler deallocation does not justify forgetting hardware cleanup that is still pending.
The journal must distinguish claim identity, allocation result/share identity and Pod consumers. Count claim allocations separately from Pod references when a claim can be reused. Track external host ownership explicitly where supported; otherwise exclude unsafe exposure profiles, as in Example 7.
8.4 Capability and Authorization Boundaries
The installation profile requires DRAPartitionableDevices, DRAConsumableCapacity, DRADeviceCompatibilityGroups and DRAListTypeAttributes for the full examples. DRASharedConsumableCapacity is not part of the baseline. S2
Use a deployment capability contract and verification, not a fictional generic runtime API for reading all feature-gate values. API write/read tests must detect stripped fields. Scheduler behavior needs its own controlled test. Loss of a required capability must not trigger a fallback that removes exclusion constraints from still-published alternatives.
Restrict writes to driver ResourceSlices, exposure policies and relevant claim status. Compatibility is an allocation predicate, not a tenant authorization system or protection against arbitrary privileged host reconfiguration. Administrative-access claims and custom schedulers require an explicit support policy; the baseline guarantees target normal scheduler-managed allocations.
9. Required Acceptance Tests
The following are implementation acceptance criteria. They have not been run against Kubernetes as part of preparing this document.
| Test | Required result |
|---|---|
| Policy priority and exclusions | One winner per suffix; lexical tie break; any exclusion wins; no match publishes nothing new. |
| Selector failure | Invalid CEL is rejected; runtime evaluation errors fail closed with diagnostics. |
| Policy-to-v1 serialization | No basic, no stringSlice; attributes/capacity at device level; quantities and attribute types preserved. |
| Slice separation and limits | Counter and device objects separated; consistent total count and generation; third consumption or third group rejected. |
| Counter-set resolution | PF/VF members resolve to the same family domain; different interfaces/pools stay independent. |
| Alias coverage | Multiple personas without a common guard rejected; exclusive aliases cannot coexist. |
| Macvlan repeated allocations | A and B on one device both allocate, consuming two local units and one fixed member. |
| Cross-mode rejection | IPvlan rejected while any macvlan claim remains, in either allocation order. |
| First-share release | Release A while B remains; reconstruct allocator state; incompatible mode stays rejected. |
| Last-share release | Release B, reconcile cleanup and allocation state; the alternate mode can allocate. |
| Driver/scheduler state reconstruction | Restarts preserve compatible outcomes and cannot unlock an active mode. |
| Local capacity boundary | 64 one-unit allocations fit; 65th does not. Releasing one restores exactly one unit. |
| Shared versus exclusive | Both orderings blocked; second allocation of a normal exclusive entry rejected. |
| PF family combinations | PF macvlan plus all eligible VFs permitted; PF passthrough rejected until all incompatible family members are released. |
| Independent interface and family domains | Adding PF ipvlan does not block VFs but does block PF macvlan; all two-domain checks apply. |
| Whole-set intersection | Pairwise-overlapping but globally disjoint group sets cannot all allocate. |
| Empty group behavior | Empty only matches empty; missing groups never treated as a wildcard. |
| Single allocation search and backtracking | Incompatible requests in one claim/Pod rejected; failed candidates restore group and capacity search state. |
| Policy / identity mutation | Active rename, group change, suffix replacement, sharing change and VF-count shrink require drain; no early retirement. |
| Host namespace move / rebinding | Rescan does not forget a still-allocated device after a supported operation moves or rebinds it. |
| Hidden bridge-owned VF | Exclusion alone creates no reservation; unsafe PF passthrough is unavailable in the selected host-bridge profile. |
| Feature disabled / stripped / skewed | Required fields missing or unenforced prevents unsafe publication; no silent downgrade to unconstrained alternatives. |
| Partial ResourceSlice updates | Incomplete generations and retry/restart converge without exposing conflicting new aliases. |
| Claim reused by Pods | Pod reference changes do not double-charge local capacity or release the claim’s device early. |
| Prepare/unprepare retries and failures | Idempotent journal replay; deterministic domain locking; cleanup failure retains safety state. |
| Optional shared-bandwidth profile | Only on a combined supported build: per-share accounting, release order, defaults, policy changes, feature transitions and compatibility all tested. |
| OVN-K profile | Advertised support is promoted to an implementation guarantee only after provisioning, handoff and cleanup tests pass. |
Use both upstream structured-allocator tests with published fixtures and integration/e2e tests exercising real claim transitions. A mathematical model can check desired combinations, but it does not prove the actual scheduler, API server, feature gates or node driver implement them.
10. Implementation Handoff
The first implementation should deliver the baseline, not all optional integrations at once:
Policy API and compiler. Implement the proposed schema and admission rules, normalized discovery facts, deterministic policy resolution, scoped domain resolution, stable identity mapping and whole-pool validation. Pure compiler tests should use the examples in this document as fixtures.
Publication and allocation-aware state. Integrate the upstream ResourceSlice publishing machinery, correct packing/counts, committed inventory state, allocated-claim observation and drain handling. Add failure/restart tests before allowing automatic destructive transitions.
Node enforcement and topology integration. Implement the journal and domain safety checks, then connect the companion NetworkTopology/CNI execution path. Keep plugin-specific arguments separate from exposure metadata. Confirm the one-allocation/one-interface baseline contract and claim-sharing behavior.
Target-cluster verification. Run API validation, CEL evaluation and allocator/e2e acceptance on the exact Kubernetes or OpenShift target. Alpha source availability is not production support approval.
Follow-ons. Add an explicit shared-budget policy API and KEP 5941 integration only after the baseline works. Treat OVN-K and host-managed reservations as separately tested integrations, not assumed consequences of the generic discovery engine.
Suggested initial interfaces are conceptual boundaries, not preselected Go APIs: Discover, ResolvePolicies, CompilePools, ReconcilePublication, ObserveAllocations, and EnforceDomainContract. Their concrete signatures should be designed against the actual driver repository in the implementation phase.
No implementation or live site update is implied by this document revision.
11. Source Notes
The uploaded design provides the original scope, terminology, nine scenarios and reference inventory. This revision replaces the exclusion model and adds explicitly proposed policy/lifecycle rules. The custom compatibility schema, non-binding members counters and compiler checks are design decisions in this document, not features supplied automatically by Kubernetes.
Upstream facts were checked against the following primary sources. Prefer the pinned implementation over older prose when the two disagree. Companion documents and the OVN integration were not independently revalidated in this revision.
- S1: Resource API v1: object shapes, quantities, groups and limits
- S2: Feature gates and source-level version/default registration
- S3: Compatibility intersection and feature-disabled slice handling
- S4: Allocator integration and live-slice compatibility baseline
- S5: Allocated-device detection, including shared devices
- S6: API server feature-field stripping
- S7: DRA API concepts: classes, claims and list attributes
- S8: KEP 5941 PR-head API: valueFrom.capacityName and shared consumption
- S9: DeviceClass API and CEL selector environment
- S10: KEP 5963 design text (may differ from implementation)
- S11: Shared consumable capacity PR and review discussion
- S12: KEP 5075 consumable-capacity design