Backport (reimplement) FIX tool enhancements in 1.3.x (SEARCH-2368)

[SEARCH-2404] Reimplement SEARCH-2248 and SEARCH-2233 on SS 1.3.x Created: 03-Sep-20  Updated: 15-Oct-20  Resolved: 15-Sep-20

Status: Done
Project: Search and Discovery
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Sub-task Priority: Blocker
Reporter: Tom Page Assignee: Unassigned
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
relates to SEARCH-2233 Expose additional parameters for FIX ... Done
relates to SEARCH-2248 Add a limit to a number of txns per r... Done
Sprint: Team Ninja-King - S&I 45, Team Ninja-King - S&I 46, Team Ninja-King - S&I 47
Release Train: Hayes & Harlington
Delivery Team: Search

 Description   

Reimplement the fixes from SEARCH-2248 and SEARCH-2233 on SS 1.3.x:

  • Add ability to restrict transactions to fix by start and end times.
  • Add ability to limit number of transactions to repair (together with a system-wide default)
  • Add a dry-run parameter that will output the number of transactions and nodes that would be affected (split by "Tx in index and not in DB", "Tx in index multiple times" and "Tx in DB and not in index")


 Comments   
Comment by Tom Page [ 03-Sep-20 ]

On master (2.1.x) then the core name is a mandatory parameter when calling the FIX action: https://github.com/Alfresco/InsightEngine/blob/8c9083208fbdb8d3f0091707e5f12e95e9ef3236/search-services/alfresco-search/src/main/java/org/alfresco/solr/AlfrescoCoreAdminHandler.java#L1704

On 1.3.x then if the core name is not supplied the FIX action will be run against all cores:

                    if (cname != null) {
                        rsp.add(ACTION_LABEL, actionFIX(cname));
                    } else {
                        for (String coreName : getTrackerRegistry().getCoreNames()) {
                            rsp.add(ACTION_LABEL, actionFIX(coreName));
                        }
                    }

Master also contains some code to fix all cores:, but it comes after the validation that a core name has been supplied: https://github.com/Alfresco/InsightEngine/blob/8c9083208fbdb8d3f0091707e5f12e95e9ef3236/search-services/alfresco-search/src/main/java/org/alfresco/solr/AlfrescoCoreAdminHandler.java#L1742

For the time being I'll proceed with the assumption that the core name is not mandatory on 1.3.x, but we may need to revisit this decision later.

Comment by Tom Page [ 07-Sep-20 ]

While checking the new parameters were being accepted by the server without causing errors I discovered that the fromTxCommitTime being in the past causes the process to take longer, even if the toTxCommitTime is very close to it. For example the following FIX requests takes about 12 seconds locally:
http://localhost:8083/solr/admin/cores?action=FIX&core=alfresco&fromTxCommitTime=1590000000000&toTxCommitTime=1590000000002
As does this:
http://localhost:8083/solr/admin/cores?action=FIX&core=alfresco&fromTxCommitTime=1590000000000&toTxCommitTime=1599469360003
Omitting the toTxCommitTime takes about 6s:
http://localhost:8083/solr/admin/cores?action=FIX&core=alfresco&fromTxCommitTime=1590000000000
Omitting the fromTxCommitTime returns instantly:
http://localhost:8083/solr/admin/cores?action=FIX&core=alfresco
As does sending timestamps in the very recent past:
http://localhost:8083/solr/admin/cores?action=FIX&core=alfresco&fromTxCommitTime=1599469360000&toTxCommitTime=1599469360003

Comment by Tom Page [ 07-Sep-20 ]

Merged https://github.com/Alfresco/InsightEngine/pull/923

Comment by Tom Page [ 07-Sep-20 ]

Merged an extra test on master: https://github.com/Alfresco/InsightEngine/pull/924

Comment by Tom Page [ 09-Sep-20 ]

Merged PR for dry run option: https://github.com/Alfresco/InsightEngine/pull/934

Generated at Mon Jun 21 20:18:31 BST 2021 using Jira 7.13.15#713015-sha1:7c5ddd2c3e1709974ae9c48c17df8edd3919fe2c.