KOTS 9.0.3 - SAML service has a "Liveness Probe failed" error at startup

Author: Jason De Paola

Date: December 06, 2023

Audience: Everyone

Environmental details: Self Hosted

Summary: 

KOTS 9.0.3 has a known issue with the SAML pod not starting up with the following errors:

Solution:

Step 1:

Create a local copy of the SAML StatefulSet yaml file. The samlSts.yaml file be located in the directory in which you run this command:

kubectl get sts/saml -o yaml > samlSts.yaml

Step 2:

Delete this data in samlSts.yaml and save the file:

Step 3:

 kubectl delete sts/saml

Step 4:

kubectl apply -f samlSts.yaml

After Step 4, the same pod will start automatically and should be OK.

Note, after this fix is applied, going forward, this must be done:
If SAML config changes are made and the application is redeployed, you must follow steps 1 - 4.
If no SAML configuration changes are made, only steps 3 - 4 are needed.

As of version 9.6, the SAML and Oauth configs have been moved from statefulsets to deployments. The fix should be the same, but need to replace 'sts' with 'deployment'

Cause:

Jama Connect® KOTS requires a health check endpoint in our services “/health.” Still, in the Jama Connect® KOTS v9.0.3, the new SAML version is included (1.4.0), causing problems in the KOTS environment because it doesn’t have the “/health” endpoint and Kubernetes is killing the SAML service because when it tries to check the status it gets: “Connection refused” Resolving KOTS Runtime Error: Provisioning Jama on Migrated Server.

Prevention:

This is fixed in SAML 1.5.2

Additional Documentation and Resources for Support Engineers:

Review requests #71941, 72930, and 72937

Feedback: Please leave feedback in the comments below.