Affects Version/s: Search Services 2.0
Fix Version/s: Search Services 2.0.1
Sprint:Team Ninja-King - S&I 52, Team Ninja-King - S&I 53, Team Ninja-King - S&I 54
The environment was set up for
The reindex speed is ~150k docs/hour vs ~600k docs/hour in 1.4.3.
The problem are:
- CPU on solr is loaded on ~100% constantly. It is expected with mock indexing but distribution of load across solr treads are different in comparison with 1.4.
New threads appeared for 2.0 that there were not for 1.4.
It is ForkJoinPoolThreads. There are 64 threads in the pool:
In comparison with 1.4 there were no such thread among solr threads. Please pay attention that when ForkJoinThreads are running there are no SolrTrackingPool among threads what means that tracking of metadata and content is not being done during that period.
As we can see from Top every Runnable thread from this pool are using ~15-20% CPU on the average. Considering that 18 thread are Runnable then it can be calculated that ~70% of overall CPU is occupied by these threads.
Due to that every index portion happens every 20-30 minutes only so indexing is so slow.
Because of solr is overloaded response time for tracking requests is long tool but it is consequence of the issue above.