PVC failing to mount - how to fix

James Ressler
James Ressler
  • Updated

Author: James Ressler

Date: December 15, 2023

Audience: Everyone

Environment: Self-hosted, Replicated - KOTS


Sometimes, pods are blocked from starting if their associated PVC fails to mount. To check, describe the affected pod.

kubectl describe pod <POD-NAME>

And look for an event like:

ControllerPublishVolume: err: rpc error: code = DeadlineExceeded desc = volume pvc-64a0b176-3982-4a75-83c9-7cc5857cb44b failed to attach to node


First, check the share manager pod in the 

kubectl get pods -n longhorn-system | grep "share-manager"

Check the status of the share-manager pod. It should report 1/1 Running.

Scale down the statefulset associated with the failing pod and then force the share manager to recreate itself. After the share manager pod returns, scale the statefulset back up. In the example below, the core-0 pod fails to mount its PVC, so we scale down the core statefulset.

kubectl scale statefulset/core --replicas=0 -n default
kubectl delete pod <SHARE-MANAGER-POD-NAME> -n longhorn-system --grace-period=0 --force
kubectl scale statefulset/core --replicas=1 -n default

After the affected pod comes back up, you should see the PVC mount correctly this time.

Related to

Was this article helpful?

0 out of 0 found this helpful

Have more questions? Submit a request



Please sign in to leave a comment.