Indexing aborted due to corrupt item threshold exception error (Native)

Author: Sravya Bandari

Date: December 29, 2023

Audience: Everyone

Environmental details: Self-hosted Traditional/Native

Summary: 

If the maximum allowable number of broken items exceeds, the complete reindexing process on the instance will be aborted.

Solution:

• If you see the error mentioned in the logs:

2022-02-21 23:49:40,233 WARN  idx-47 jama_test  [com.jamasoftware.contour.search.indexer.overseer.TaskOverseer] - !!!!!!!!!!!!!!!!!!! Index task aborted because of exception "CorruptItemThresholdException" !!!!!!!!!!!!!!!!!!! 
com.jamasoftware.contour.search.indexer.exception.CorruptItemThresholdException: The maximum number of broken items has been exceeded. Aborting reindex.

2022-02-21 23:49:40,471 INFO  QuartzScheduler_Worker-1 jama_test  [com.jamasoftware.contour.service.impl.IndexServiceImpl] - End reindex for tenant [jama_test]
2022-02-21 23:49:40,474 INFO  QuartzScheduler_Worker-1 jama_test  [com.jamasoftware.contour.service.MaintenanceModeServiceImpl] - End maintenance mode

• You can fix it by increasing the limit for broken items. Add the below line in the admin console under Settings > Advanced Startup Settings. Make sure to check "Show advanced startup settings."

jamasearch.reindexBrokenItemThreshold = 10000000

• After adding this line, restart Jama and then run the reindex.

Cause:

The issue arises because Elasticsearch cannot index everything in an item, such as images inside PDFs. This limitation is called "corrupt" items, but it doesn't impact Jama's performance, except for reaching a limit on broken items.

Feedback: Please leave feedback in the comments below.