Details

    • Bug Priority:
      Category 3
    • ACT Numbers:

      00699477

    • Sprint:
      Docs Sprint 18

      Description

      I am investigating methods for setting up Disaster Recovery for a client (TIAA) and have been thinking about a couple of options.

      1. Periodic back up and restores (from the primary to the DR site)
      2. Echoing the Elastic Search Events to the DR site.

      In both cases, the biggest challenge will be synchronization. When recovering from a Business Continuity Incident, there is a high probability that the Elastic Search will not be in sync with the database. The challenge will be to:

      • Detect the level of misalignment between the database and the Elastic Search
      • Performing the appropriate clean up to force alignment between the two

      In considering this challenge, a couple of questions come to mind.

      • Is there a way to tell, by looking at the elastic search information only, the last event that was processed.
      • If the elastic search has processed events that are not yet in the database (a possibility in a DR scenario) is there any way to roll back the processed events.

      Here are my assessments of each of the methods –
      please advise on any pros and cons of the methods and on what is possible. thanks

      Periodic Backup and restore

      Method

      1. Periodically back up the elastic search index
      2. Send the back up across to the DR site
      3. Restore the index on the DR site

      Advantages

      • This uses common infrastructure utilities requires no special extensions to the Activiti App.
      • Very low potential of the index being ahead of the database

      Disadvantages

      • Backup will grow over time causing the size of the periodic data transfer to grow over time

      Echoing the Elastic Search Events to the DR site.

      Method

      1. Echo the elastic search events to an additional host or set of hosts (on the DR site) that are not queried as part of any search/reporting requests

      Advantages

      • This is an incremental method that will have more predictable bandwitdh requirements

      Disadvantages

      • Requires extension code to be added to the activiti app
      • May require an intermediate message queue to account for the possibility that the DR site is down.
      • Increased possibility that the elastic search index could be ahead of the Database (which would mean that we would need to back out some of the events from the index)

        Attachments

          Activity

            People

            • Assignee:
              djohnson Doug Johnson
              Reporter:
              rmcknight Richard Mcknight
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: