Jama Connect® 8 Self-Hosted Traditional Installation and Upgrade Troubleshooting Tips

• Best Practices
• Does the Jama Connect Application Server Meet the Specifications for Jama 8?
• Is Jama Connect Configured Behind a Proxy?
• Is a Firewall Blocking the Communication Between Jama Connects’s Containers and the Server?
• Is the Jama Server Connecting With the Database?
• Have You Changed the Jama Server IP Address?
• Are You Pointing an Existing Jama 8 Environment to a New Database?
• Are All Replicated and Jama Containers Running?
• Have You Restarted Docker and Replicated Services Properly After a Configuration Change?
• Is the Application No Longer Starting After Syncing a New License?
• Useful Commands

Best Practices

1. Back-Up the Application - There are a couple of different aspects of the application that should be backed up prior to every upgrade. This ensures that you will be able to safely roll back to your starting version if you run into any problems with upgrading Jama Connect. There are two main steps to take a backup of your application:
   1. Create a Manual Snapshot via Replicated - This creates a snapshot of Jama Connect and includes your admin console and settings, all current Docker containers that make up the Jama Connect application, and your license provided by Jama Software.
      
      
   2. Create a Database Backup - The Replicated snapshot referenced above does not include a backup of the database so this step needs to be done separately. Creating a database backup enables you to roll back your database along with your application snapshot. If you roll back the application without also rolling back the database to the same version, there is a risk that the database version and application version will be out of sync, which can lead to difficult-to-diagnose problems. Jama recommends using a proprietary database backup (mysqldump for MySQL or through SQL Server Management Studio for SQL Server).
      
      
2. Restart the Database Before Upgrade - Before upgrading Jama through the Replicated console, it is recommended that you also restart the database. Since upgrading Jama can include database schema changes, it can be necessary to perform a database restart prior to performing the upgrade in order to successfully start up the application.
   
   
3. Rebuild the Search Index - New releases will often include changes to the search index that require rebuilding the search index after upgrades. This should be done with every major Jama upgrade but can be skipped for patch releases unless otherwise noted.  

Does the Jama Application Server Meet the Specifications for Jama 8?

• Check the Supported Software and System Requirements to make sure your Jama application server is compliant. The installation can fail if one (or more) of the system requirements is not met.
  
  
• Pay special attention to the operating system, the total memory of the Jama server, the Docker version that is installed, and whether it is supported by Docker. The errors due to an unsupported operating system or missing dependencies for the installation of Docker usually show up while running the install script.
  
  
• Verify if your Jama server has enough memory. Even if you are running Jama on a test server with potentially few users, you still need to allocate the memory required for the SMALL category.
  
  
• Make sure Linux is installed and configured properly.
  
  
• If you are using RHEL, make sure you have an active subscription.
  
  
• The Docker repository on GitHub is a good place to consider if you face issues during the installation of Docker.
  
  
• Running in loopback mode is discouraged for production. It has known performance problems and a different storage driver should be used. Please visit the article Warning for recommended storage drivers for Docker.

Known Symptoms

• The dashboard gets stuck at “Waiting for Jama Core.”
  
  
• Docker installation fails due to a missing dependency.
  
  
• Install script halts with the error: Either your platform is not easily detectable, is not supported by this installer script, or does not yet have a package for Replicated.
  • If you face the above error, you are using a Linux distribution that is not yet supported by the install script. Please follow the manual instructions instead. Open a Support ticket and include your Linux distribution and we will file a request with our Product team to add support for it.

• The install script shows a warning on Docker.
  • Running in loopback mode is discouraged for production. It has known performance problems and a different storage driver should be used. Please visit the article Warning for recommended storage drivers for Docker.
    • Note: Troubleshooting Docker installation issues is out of Support's scope.

Is Jama Configured Behind a Proxy?

Proxy misconfiguration has been the root cause of most of the issues that our customers have faced during the installation of Jama 8.

• Make sure that you follow the instructions to Install the Admin Console and configure the NO_PROXY environment variable.
  
  
• Verify that both HTTP_PROXY and NO_PROXY environment variables are set properly and are picked up by Docker:
  • Download a Support Bundle and go to scheduler > node > {node_id} > docker > docker_info.json.
  • Look for "HTTPPROXY" and "NOPROXY" attributes and make sure the right values are set for both.

• Verify that replicated file has HTTP_PROXY variable set in REPLICATED_OPTS and it has the correct value:
  • Find the replicated file and open it.
                  Debian and Ubuntu: /etc/default/replicated
                  CentOS and RHEL: /etc//replicated
  • Locate the line that starts with REPLICATED_OPTS
  • Make sure "-e HTTP_PROXY={proxy_url}" is set properly

• If your proxy requires authentication, the URL should include the user: password e.g. http://user:password@proxy_url. Make sure you are using the right credentials.

• Check to see if the proxy variables are indeed working to verify communication outside of server

curl -s http://help.jamasoftware.com | grep title

curl -s ://[some_intranet_url]

Known Symptoms

• Admin Console throws an error while pulling images. e.g.:

API error (500): Get ://{ip_address} EOF

• Snapshots are failing.

• Jama and the Admin Console are running but the Admin Console is stuck at "Starting" on the Dashboard.

• Jama license does not get updated after a license sync even after restarting Jama.

Is a Firewall Blocking the Communication Between Jama’s Containers and the Server?

Firewalls can block communication among containers or block communications from outside of the Jama server if the required ports are not open. We recommend disabling firewalls on the Jama server, if possible. If you need to have firewalls, make sure that routing rules are not interfering with the rules that Docker adds.

• Make sure all the required ports are open on the Jama server.
  
  
• If you are running CentOS or RHEL, make sure that is not running. There is a conflict between and Docker that may cause communication issue among running containers. 
  
  
• Disable the firewall on your server and restart the setup to verify that the problem is, in fact, the firewall.
  
  
• If you need to have enabled on your Jama app server, Jama requires all the ports to be open for internal IP addresses used by Docker. You can use the following rule to allow traffic on all ports for 172.17.0.0/24 subnet which is used by Docker. 

-A INPUT -p -s 172.17.0.0/24 -j ACCEPT

Known Symptoms

• Pre-flight checks show an error: Can connect to address - ERROR: dial {ip_address}:9874: : no route to host

• Admin Console throws an error while deploying search or elastic search containers.

Is the Jama Server Connecting Successfully?

Shows the error Can't populate.

It is very likely that your Jama server cannot communicate properly with the database. There are several common reasons:

• Your Jama server cannot communicate with the database server. You can use the telnet command to check the connection

telnet {database_url or ip_address} {database_port}

e.g.

telnet mysql.jama.com 3306

• If you are using MySQL, verify if Jama database user has permission to connect from server host.
  
  
• You have entered the wrong credentials for the database user. Try logging into your database directly using the same credentials to make sure it works.
  
  
• Your database user does not have enough permission to create and modify the database. See the Help Guide.

You can look into the following log files on your Jama server to get more details on what can be wrong:

/logs/tomcat/contour/contour-c3p0.log
/logs/tomcat/contour/contour.log

Have You Changed the Jama Server IP Address?

• If you have to change the IP address of your Jama 's Application Server, please follow the instructions to Update Jama's IP Address.
  
  
• You may get an error in Pre-Flight checks (Can't connect to address - ERROR: dial {ip_address}:9874: : no route to host) showing the old IP address of the server if you miss a step.
  
  

Are You Pointing an Existing Jama 8 Environment to a New Database?

If you want to point an existing Jama 8 environment to a new database, or you are dropping an existing database and creating a new one, make sure that you are following the instructions:

1. Stop Jama from Admin Console.

2. Drop the existing database and create a new one with the same name.

3. Go to the /data/ folder and remove the following folders:

ActiveMQData , , tenant

4. If loading from an put the file in restore and make sure it has all the permissions as stated on the settings page.

5. Go to the Settings page and make sure all the database fields are filled properly, then select Save.

6. Start Jama.

Are All Replicated and Jama Containers Running?

If none of the above scenarios apply to you, use the following command to check if any of the main Replicated or Jama containers have exited.

docker ps -a

The main Replicated containers are:

• replicated
• replicated-
• replicated-operator

The main Jama containers are:

• search
• elastic-search

If the above containers not running, you can take a step and check the logs for the specific containers using:

docker logs {container_id}

Have You Restarted Docker and Replicated Services Properly After a Configuration Change?

If you have changed a configuration file in Docker or Replicated you need to restart those services properly to make sure they take effect. There are times when restarting Docker and Replicated services can fix an installation issue.

Note: Restarting Docker and Replicated services means stopping Jama.

1. Stop Jama from the Admin Console Dashboard. 

2. Go to the Jama application server and stop all running containers using the below command.

docker stop $( docker ps –a –q)

Skip the steps above if the Admin Console is not accessible. 

3. Restart Docker.

restart docker

4. Restart all Replicated services.

restart replicated replicated- replicated-operator

5. Go to the Admin Console and start Jama.

Is the Application No Longer Starting After Syncing a New License?

Check to ensure that your current user account settings are compliant with the new license (i.e. you have the same number of users configured as your license entitles you to have). 

Known Symptoms

Review the stderr.log log inside the Support Bundle > scheduler > container > to confirm the following entry:

[timestamp information] INFO __main__: sending status to Replicated: problem: problem set application license: Error license update for 'contour'; Cannot load this license. It would put the organization out of compliance.

Next Steps

• If you believe you received an errant license, contact your Account Manager to remedy the issue.
  
  
• If you received the correct but forgot to update your Jama user accounts, contact our Support team and they will provide you with steps to manually edit the existing user accounts.

Useful Commands

You need root access to run Docker commands.

docker info
A useful command to verify the Docker version, the total number of Docker containers and their status, kernel version and OS.

docker ps -a
Lists all the docker containers and their status (whether they are currently running or not).

docker logs {container_id}
Shows the logs for the container specified by container_id. Very useful for initial troubleshooting if a container is not running.

telnet {database_host_server} {database_port}
Useful command to verify if Jama server can communicate with database server.

You can always open a Support ticket and attach a Support Bundle if you still cannot determine the problem.