Author: Jenna Zwick
Date: December 12th 2023
Audience: Agents and Admins
Environmental details: 9.0, KOTS, Self-Hosted
KOTS runtime error while provisioning a new KOTS application server.
The tenant-manager pod logs indicated someone had migrated the server from a Native production server. It still retains invalid URL references from the old instance and provisioning data created when the origin server was that prevented the provisioning from finishing successfully.
After a basic tenant data cleanup was completed, the server did not provision correctly. (See bash commands below)
kubectl exec core-0 -- sh -c 'rm -rf /home/contour/tenant_properties /home/contour/tenant.properties /home/contour/setup.properties /home/contour/services.properties /home/contour/node.properties /home/contour/database.properties'
A more assertive strategy was adopted, involving the deletion of configuration files and various Kubernetes components:
1. Delete the tenant-manager job and stateful sets:
kubectl delete job/tenant-manager
kubectl delete sts/hazelcast sts/activemq
kubectl delete pod/core-0
2. Execute the following six commands to remove problematic properties and reset the system:
kubectl exec core-0 -- sh -c 'rm -rf /home/contour/database.properties'
kubectl delete pod/core-0
kubectl delete sts/core-ingress sts/core-jobs sts/core-reports sts/core sts/search sts/activemq sts/elasticsearch sts/oauth sts/nginx sts/diff sts/hazelcast sts/saml
kubectl delete pvc/tenantfs pvc/volume-core-ingress-0 pvc/volume-core-jobs-0 pvc/volume-core-reports-0 pvc/volume-core-0 pvc/volume-activemq-0 pvc/volume-diff-0
kubectl delete pvc/volume-elasticsearch-0 pvc/volume-hazelcast-0 pvc/volume-nginx-0 pvc/volume-oauth-0 pvc/volume-saml-0 pvc/volume-search-0
kubectl delete sts/elasticsearch jobs/tenant-manager jobs/tenant-manager-for-cluster
kubectl delete pvc/volume-elasticsearch-0 pvc/volume-elasticsearch-1 pvc/volume-elasticsearch-2
kubectl delete svc/core svc/activemq svc/core-ingress svc/core-jobs svc/core-reports svc/elasticsearch svc/elasticsearch-discovery svc/search svc/diff svc/hazelcast svc/oauth svc/saml svc/nginx
3. Reset all settings in the KOTS Admin Panel, as they will be completely reset to their default values.
4. Ensure the Host Name is correctly set, then deploy Jama Connect again via the Admin Panel.
Jama Connect® should now provision correctly on the new application server.
After provisioning a new server, you should update all URL references inside Jama Connect® and Update your instance URL in Jama Connect®'s root menu.
Also, performing a full re-index, as detailed in the "Reindex All Search Items" section on page 529 of the Jama Connect User’s Guide (9.0.x), is strongly recommended.
Migrating an instance from Jama Connect Native to KOTS and instead of retaining the DNS name, the new server has a new name.
Instead of giving the new server a new name, decommission the old server or shut down the old VM and associate the original DNS name of the application server with the IP address of your new server, or follow this process to make sure your new server is provisioned the way it should be. (Either is acceptable with Jama Connect, depending on your network environment.)
Please contact our support team if you need more help or questions.
Feedback: Please leave feedback in the comments below.