Skip to content

Concurrent backend reconciliation clears backend volume associations #1166

Open

Description

@s-starostin

Description

When controller concurrency is enabled, periodic backend reconciliation
replaces existing backend objects with newly initialized backend objects.

The replacement backend has an empty backend-local volume map. As a result,
tridentctl get backend reports VOLUMES=0, while tridentctl get volume
continues to return all managed volumes with valid backend UUIDs.

Environment

  • Trident: 26.06.0
  • Driver: ontap-san
  • Controller concurrency: enabled
  • Persistence: Kubernetes CRDs

The issue was also observed with Trident 26.02.1.

Observed behavior

All backends remain online and normal:

netapp-qos-01-02  online  0
netapp-qos-04-02  online  0
netapp-qos-03-01  online  0
netapp-qos-03-02  online  0
netapp-qos-04-01  online  0
netapp-qos-01-01  online  0

At the same time, the global volume store remains populated and all volumes
have valid backend UUIDs.

Disabling controller concurrency and restarting the controller restores the
backend volume counts.

Suspected cause

ConcurrentTridentOrchestrator.reconcileBackendState replaces the backend
through upsertBackend, but the replacement backend does not retain the
original backend-local volume associations.

Proposed fix

Copy the existing backend-local volume associations from the original backend
to the replacement backend before node access reconciliation and cache
replacement.

I have prepared a focused fix and regression tests and plan to submit a PR.

Activity

  1. nikolaitsarenkov commented on Nov 10, 2026

    @nikolaitsarenkov

    Hi everyone!
    I’ve run into the same problem.

    Are there any updates on this issue?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions