Upgrade exist-db

Since ART 1.6.0 we recommend updating the eXist-db database to version 2.2. This section documents the actions that are needed to upgrade from eXist-db 2.1 to eXist-db 2.2. If you installed eXist-db from eXist-db-setup-2.1-rev0000.jar, you are running the outdated eXist-db 2.1 and should consider updating to eXist-db 2.2.

As an alternative to the export/import for all data and users listed below: If you are running with a couple of users, no SNOMED CT data and very little project data, it might be easier to create a full database export (see below), install eXist-db 2.2 as explained here: ART-DECOR installation on linux, recreate the users as explained in the Maintenance manual/create users, import your projects into /db/apps/decor/data (use dashboard/Collections/upload resources) and run the fix permissions xquery.

The procedure the export/import all data and users is listed here.

Please note that the full terminology package SNOMED CT data is not supplied from our repository, due to licensing issues. We do supply an empty stub package Terminology Data - SNOMED CT version 14.1.1, but this essentially contains no SNOMED CT, but just serves as an example on how to host this content. Please skip the steps regarding SNOMED if you do not have the full terminology package SNOMED CT data installed.

On the old eXist-db 2.1 instance

Perform the actions in this section on the old eXist-db 2.1 version to create a full backup and export user information.

Create a full database export

Create a full database export. See Create database backup.

Exporting usergroup information, user passwords and meta-information

When exporting/importing a full exist-db database dump, user groups are not correctly handled. For that reason we first export user group information. To get all user groups, (hashed) user passwords and meta-information for each user: Get account data xquery

Stop eXist-db

To stop the server perform the following actions:

  • Log in with SSH
  • Stop the old exist-db server
sudo service exist_atp stop

Install the new eXist-db instance

Perform the actions in this section to install eXist-db 2.2.

Start with the steps explained here: ART-DECOR installation on linux and stop at: eXist-db logging, so you don't install ART-DECOR software & data because we first will import the data from the eXist-db 2.1 backup.

Restore from the eXist-db 2.1 full backup

Modify the backup data before importing it

eXist-db 2.2 comes with preinstalled applications: dashboard, eXide and shared-resources. We will remove these applications from the eXist-db 2.1 backup, to not overwrite the 2.2 versions. Also we will remove the data from the package SNOMED CT data because restoring this data does not work properly (so we will deinstall/install it later on). At the end of these steps we have a directory with unzipped database data with some parts deleted.

  • Locate the eXist-db 2.1 full backup on the old server (of from the old eXist-db 2.1 instance):

/usr/local/exist_atp/webapp/WEB-INF/data/export/full-<datetime>.zip

  • Log into the new server with SSH
  • Copy the full backup to a temporary location and modify it:
# copy the full backup to your home directory in a directory like 20151031_exist_data
cp /usr/local/exist_atp/webapp/WEB-INF/data/export/full20151030-0323.zip ~/$(date '+%Y%m%d')_exist_data
# chown the zipped data
sudo chown $USER:$USER full20151030-0935.zip
# unzip the backup
unzip full*
# change directory
cd ./db/apps
# remove dashboard, eXide and shared-resources applications
# first edit the contents file which contains information on the exported data:
vi __contents__.xml
# remove the following lines:
  <subcollection name="dashboard" filename="dashboard"/>
  <subcollection name="eXide" filename="eXide"/>
  <subcollection name="monex" filename="monex"/>
  <subcollection name="shared-resources" filename="shared-resources"/>
# exist vi and save the file:
<ESC>:wq
# remove the data folders for the dashboard, eXide and shared-resources applications
rm -Rf dashboard eXide monex shared-resources
# remove the data from the package SNOMED CT data, skip this step if you do not have this package installed
# change directory
cd terminology-data/snomed-data
# first edit the contents file which contains information on the exported data:
vi __contents__.xml
# Change these lines:
  <subcollection name="meta" filename="meta"/>
  <subcollection name="Delta" filename="Delta"/>
  <subcollection name="en-GB" filename="en-GB"/>
# into:
  <subcollection name="meta" filename="meta"/>
# remove the data collections for SNOMED CT data
rm -Rf Delta en-GB
# note that we leave the meta collection intact.

Importing the modified backup data

In this step we will import the modified backup data from eXist 2.1 into eXist-db 2.2. We need root permissions to restore data into the server, so we use sudo.

(optional) we use the linux application screen to continue the backup process if the SSH-session disconnects.

(optional) we use the linux application script to record any errors. 


# Start a screen session
sudo yum install screen
screen
# change directory
cd /usr/local/exist_atp/
# Start a script session
sudo script $(date '+%Y%m%d%H')_log.txt
# restore data into eXist-db 2.2. Use the admin account and password. Replace the location for the backup with your own useraccount.
bin/backup.sh -u admin -p <pass> -r /home/user/$(date '+%Y%m%d')_exist_data/db/__contents__.xml -ouri=xmldb:exist://127.0.0.1:8877/xmlrpc
# exit the script session
exit
# output shows:
Script done, file is 201510301913_log.txt

Some notes on the backup.sh script:

-u: the user we run the import process as

-r: the location on the filesystem where the backup data is located. We need to provide the location to the db/__contents__.xml file that is located in unzipped backup directory.

-ouri: on which server to perform this action on. We use 127.0.0.1: this linux instance.

  • Check for errors during the import phase:
grep -i 'Error\|warn\|fail' $(date '+%Y%m%d')*_log.txt

No output expected. 

grep -v '^Restored\|^Processing\|^Creating' $(date '+%Y%m%d')*_log.txt|less

Here we expect output like: 

  Starting restore of backup...
  (possible blank lines)
  Finished restore of backup.

# exit the screen session
exit
# output shows:
[screen is terminating]
  • Next we need to rebuild the application repository. The reason is that we imported ART-DECOR packages inside /db/apps, rather then installing them, so they don't show up as installed.

By rebuilding the application repository the imported packages also show up as installed: Repair application repository xquery

Create user groups

  • During the export process the file accounts.xml with user groups, (hashed) user passwords and meta-information was created.

In the new eXist-db instance we upload this file as: /db/apps/accounts.xml.

  • First login at the eXist-db dashboard:
    • Open a webbrowser, surf to: http://<server>:8877
    • Click on ‘dashboard’.
    • Log in as ‘admin’.

Now upload the resource by performing the following steps:

  • Click Collections
  • Doubleclick on apps the enter the apps collection
  • Click upload resources
  • Select the file from the local filesystem
  • After uploading the resource accounts.xml should be listed in /db/apps
  • Next run the xquery to import user group information per user:

Create account data xquery

Delete the resource: /db/apps/accounts.xml:

  • First login at the eXist-db dashboard (see above)
  • Click Collections
  • Doubleclick on apps the enter the apps collection
  • Click accounts.xml
  • Click delete resources
  • After deleting the resource accounts.xml should no longer be listed in /db/apps

Deinstall, then install SNOMED CT data

  • Deinstall SNOMED CT data

Skip this step if you did not have this package installed. Since we deleted the data from this package when we modified backup data, we first do a deinstall for this package and then install a full version with data.

To deinstall the package SNOMED CT data on the server:

  • First login at the eXist-db dashboard (see above)
  • Open Package Manager.
  • Click installed
  • Find the package listed as SNOMED CT Data
  • Hover over it and click uninstall
Error creating thumbnail: Unable to save thumbnail to destination

Install the package 'SNOMED CT data

Please note that during installation of this package, a full (re)index is done on the SNOMED CT data that typically takes about one hour during which the server is unavailable to users. We install the SNOMED CT data package with the following steps:

  • First login at the eXist-db dashboard (see above)
  • Open Package Manager.
  • Click installed
  • Select the plus icon add a package
  • Select the file from the local filesystem. Note that the installation process will start immediatly after selecting the file.
Retrieved from "https://wiki.art-decor.org/index.php?title=Upgrade_exist-db&oldid=6303"