Upgrading SlashDB 1.6.x to 1.7.x

Before upgrading, make sure you have a valid license key for version 1.7.x

Note that SlashDB 1.7.x supports Python versions 3.8 - 3.10

If you have set the SlashDB INI parameter slashdb_env to a custom directory, replace any instances of /etc/slashdb in this document with the custom directory.

General Upgrade Notes

Configuration Files Permissions

Files in the /etc/slashdb folder should have the following permissions:

[root@8fc37589d913 /]# ls -l /etc/slashdb/
total 48
-rw-rw---- 1 root    slashdb 2349 Jul 22 11:23 auth.cfg
-rw-rw---- 1 slashdb slashdb  382 Jul 22 11:23 databases.cfg
-rw-rw---- 1 slashdb slashdb  468 Jul 22 11:23 license.key
-rw-r--r-- 1 root    root    4927 Jul 22 11:23 nginx.conf
-rw-rw---- 1 slashdb slashdb 4661 Jul 22 11:23 querydefs.cfg
-rw-r--r-- 1 slashdb slashdb    0 Jul 22 11:23 regular-initial
drwxr-xr-x 2 root    root    4096 Aug 23 12:48 sites-available
drwxr-xr-x 2 root    root    4096 Jul 22 11:23 sites-enabled
-rw-rw---- 1 root    slashdb 6055 Jul 22 11:23 slashdb.ini
-rw-rw---- 1 slashdb slashdb  474 Jul 22 11:23 users.cfg

RPM / DEB Packages

You have two options to perform the upgrade:

Installing over an existing installation
- can be faster for upgrading existing hosts
- manual edits of certain configuration files may be required
Installing on a new system and copying the config files from the existing installation
- best for creating a new "golden image" or new client instance
- avoids stale components or application files whose locations might have changed
- reduces downtime
- does not affect production service in case any issues are encountered during the installation

Installing Over Existing Installation

1. Obtain and install the latest RPM/DEB on the system Create a snapshot image of the existing instance in case something goes wrong and the instance needs to be restored quickly.

Download the 1.7.x installer package:

RPM https://downloads.slashdb.com/versions/1.7.0/slashdb-1.7.0.x86_64.rpm
DEB https://downloads.slashdb.com/versions/1.7.0/slashdb_1.7.0_amd64.deb

(these links are for SlashDB v1.7.0. Replace 0 in the link with the specific 1.7 subversion you want to download)

Red Hat Linux/RPM upgrade

cd /tmp
wget https://downloads.slashdb.com/versions/1.7.0/slashdb-1.7.0.x86_64.rpm
sudo su
yum upgrade ./slashdb-1.7.0.x86_64.rpm

Ubuntu/Debian/DEB upgrade

cd /tmp
wget https://downloads.slashdb.com/versions/1.7.0/slashdb_1.7.0_amd64.deb
sudo su
apt-get --only-upgrade install ./slashdb_1.7.0_amd64.deb

Install and configure an Oracle client, following these steps:

2. Review and update the slashdb.ini file as needed, as well as any other config files that were customized

Immediately after upgrading with the 1.7.x package, SlashDB will automatically start and use the existing configuration files from the v1.6.x instance.

The new instance will be in initial setup mode, which requires visiting the web UI and going through the welcome wizard:

providing license details
accepting terms & conditions
setting the admin password

Alternatively, this can be done in the backend. After upgrading with the new package:

remove file /etc/slashdb/regular-initial
enter a valid license in the file /etc/slashdb/license.key. The original license that was used in the 1.6.x deployment is backed up to a file with a datetime stamp in it e.g. license-202208241610.key.
the configuration files below are automatically updated (there should be no need to edit these):
- /etc/slashdb/users.cfg
- /etc/slashdb/databases.cfg
- /etc/slashdb/querydefs.cfg
- /etc/slashdb/auth.cfg
the configuration files below may require manual review and editing:
- /etc/slashdb/slashdb.ini
- /etc/slashdb/nginx.conf and
- files in /etc/slashdb/sites-available and /etc/slashdb/sites-enabled
These files may be different in the newer version and won't be updated automatically. For these files, if a corresponding file with the suffix .rpmnew exists, then they must be compared using a tool like sdiff and differences resolved manually.

See section: INI File differences between 1.6.x and 1.7.x
If changes were made to any of the configuration files, restart the SlashDB service: systemctl restart slashdb

Installing On New System and Copying Configuration From Existing Instance

1. Create backups of the SlashDB configuration files from the old system On the existing SlashDB 1.6.x instance:

Create backups of the SlashDB configuration and sites directories:

cp /etc/slashdb ~/backups/v1.6.x/etc/slashdb
cp /var/lib/slashdb ~/backup/v1.6.x/var/lib/slashdb/sites

Transfer these backups to the new system

2. Obtain and install the latest RPM/DEB on the new system On the new system:

perform normal installation of the v1.7.x package as per the documentation
install and configure an Oracle client, following these steps:
- Oracle for RedHat, Oracle Linux or CentOS
- Oracle for Debian or Ubuntu

3. Overwrite the new system's SlashDB configuration files with the backups from the old system Transfer and replace some of the configuration files:

make a local backup of the configuration files of the new installation:

cp -r /etc/slashdb ~/backups/clean-v1.7.x

This will be needed to resolve differences between v1.6.x and v1.7.x in some of the configuration files.
remove the SlashDB configuration directory /etc/slashdb on the new machine:

rm -r /etc/slashdb
Copy the 1.6.x instance backups to the /etc/slashdb and /var/lib/slashdb directories

cp ~/backups/v1.6.x/etc/slashdb /etc/slashdb cp ~/backup/v1.6.x/var/lib/slashdb/sites /var/lib/slashdb

Important! Don’t copy or overwrite folders /var/lib/slashdb/www and /var/lib/slashdb/templates. These contain web UI files and must match the installed version.
update /etc/slashdb/license.key with a valid 1.7.x license
the configuration files below are automatically updated (there should be no need to edit these):
- /etc/slashdb/users.cfg
- /etc/slashdb/databases.cfg
- /etc/slashdb/querydefs.cfg
- /etc/slashdb/auth.cfg

4. Review and update the slashdb.ini file as needed, as well as any other config files that were customized The configuration files below may require manual review and editing:

/etc/slashdb/slashdb.ini
/etc/slashdb/nginx.conf and
files in /etc/slashdb/sites-available and /et/slashdb/sites-enabled

These files may be different in the newer version and won't be updated automatically. They must be compared with files in ~/backups/clean-v1.7.x using a tool like sdiff and differences resolved manually.

See section: INI File Differences Between 1.6.x and 1.7.x

5. Reassign the IP address/hostname to the new system and shut down the old one

resassign IP address/hostname to the new system
restart the SlashDB service: systemctl restart slashdb

Docker

To migrate a SlashDB 1.6.x Docker container to SlashDB 1.7.x, the simplest approach is to move the Docker volumes containing configuration data and logs from the 1.6.x container to the new 1.7.x container.

The SlashDB Docker containers include volumes for /etc/slashdb and /var/log/slashdb.

1. Obtain the latest SlashDB Docker image and config file bundle

First, download the SlashDB 1.7.x Docker image:

docker pull slashdb/slashdb

Next, download the default SlashDB configuration files and create a directory for logs

wget -c https://downloads.slashdb.com/versions/1.7.0/default-slashdb-configs_1.7.0.zip
unzip ./default-slashdb-configs_1.7.0.zip
mkdir slashdb-log

In your current directory you should have the zip file, and two directories named slashdb and slashdb-log.

There are a few changes in the SlashDB 1.7.x INI that you may want to keep. If so, make a reference copy of the slashdb/slashdb.ini file before proceeding.

2. Copy existing configuration files from the old Docker container to the new one

Your existing 1.6.x container should have been configured on installation to store its configuration and logs in two directories named slashdb and slashdb-log. Locate these directories, and copy their contents to the newly created slashdb and slashdb-log directories, respectively.

3. Review and update the slashdb.ini file as needed, as well as any other config files that were customized

If you made a reference copy of the slashdb/slashdb.ini file for the 1.7.x container, copy the changes you want from the reference file to slashdb/slashdb.ini file. You may want to use a tool like sdiff to resolve any differences. See INI File differences between 1.6.x and 1.7.x

You can now update the slashdb/license.key file for the 1.7.x container with a valid 1.7.x license using a text editor. If you want to upload a new license via the GUI, navigate to the 1.7.x SlashDB GUI in a browser to upload a license file after the container is running.

4. Launch the new container

Start the SlashDB 1.7.x container:

docker run -d -p 8080:80 -v $(pwd)/slashdb:/etc/slashdb  -v $(pwd)slashdb-log:/var/log/slashdb slashdb/slashdb

Your previous SlashDB configuration should be present on the 1.7.x container.

See the Docker Volumes documentation for more information about Docker volumes.

VMWare/VirtualBox/HyperV

1. Obtain and launch the latest virtual appliance

Download the latest virtual appliance: https://downloads.slashdb.com/versions/1.7.29/slashdb-1.7.29.ova

Install the appliance using the virtual machine management software for your virtual environment.

2. Copy the SlashDB configuration files from the old appliance to the new one

Launch the SlashDB 1.6.x appliance if it is not running, and create backups of /etc/slashdb/ and /var/log/slashdb from a shell:

systemctl stop slashdb
tar cvf etc_slashdb.tar /etc/slashdb
tar cvf var_log_slashdb.tar /var/log/slashdb

Launch the SlashDB 1.7.x appliance if it is not running. You may need to update the network settings first to configure the appliance for your hardware/network environment.

There are a few changes in the SlashDB 1.7.x INI that you may want to keep. If so, make a reference copy of the /etc/slashdb/slashdb.ini file on the 1.7.x appliance before proceeding.

Copy the tar backup files from the 1.6.x appliance to the 1.7.x appliance using SCP, FTP, a shared network resource, USB, etc.

Using SCP to copy the tar files from the 1.6.x appliance (default password for root on the SlashDB 1.7.x appliance is slashdb):

scp etc_slashdb.tar root@<slashdb_17x_appliance_hostname_ip>:
scp var_log_slashdb.tar root@<slashdb_17x_appliance_hostname_ip>:

Open a shell on the 1.7.x appliance. The tar files will be in the /root folder. You'll need to install the tar utility to extract them:

yum install tar

Extract the backups to their target directories:

cd /etc && tar xvf /root/etc_slashdb.tar --strip 1
cd /var/log && tar xvf /root/var_log_slashdb.tar --strip 1

3. Review and update the slashdb.ini file as needed, as well as any other config files that were customized

If you made a reference copy of the /etc/slashdb/slashdb.ini file on the 1.7.x appliance, copy the changes you want from the reference file to the /etc/slashdb/slashdb.ini file. You may want to use a tool like sdiff to resolve any differences. See INI File differences between 1.6.x and 1.7.x for more details.

Restart the SlashDB process:

systemctl restart slashdb-app

4. Reassign the IP address/hostname to the new appliance and shut down the old one

You can now update the /etc/slashdb/license.key file on the 1.7.x appliance with a valid 1.7.x license using a text editor, or navigate to the 1.7.x SlashDB GUI in a browser to upload a license file.

Amazon AWS/Microsoft Azure

Upgrades for Amazon AWS/Microsoft Azure deployments follow a process similar to upgrading virtual appliances. Once the new instance is deployed, you can reassign the IP address/hostname of the old deployment to the new one.

1. Launch the new instance from AWS/Azure Marketplace following normal installation instructions

2. Copy and reconcile the SlashDB configuration files from the existing instance to the new one

From the SlashDB 1.6.x container, create backups of the configuration:

systemctl stop slashdb
tar cvf etc_slashdb.tar /etc/slashdb
tar cvf var_log_slashdb.tar /var/log/slashdb

There are a few changes in the SlashDB 1.7.x INI that you may want to keep. If so, make a reference copy of the /etc/slashdb/slashdb.ini file after launching the AWS/Azure SlashDB 1.7.x container.

Copy the tar backup files from the 1.6.x container to the 1.7.x container using SCP, FTP, or a shared network resource.

Using SCP to copy the tar files from the 1.6.x container:

scp etc_slashdb.tar root@<slashdb_17x_container_hostname_ip>:
scp var_log_slashdb.tar root@<slashdb_17x_container_hostname_ip>:

Open a shell on the 1.7.x container. The tar files will be in the /root folder.

Extract the backups to their target directories:

cd /etc && tar xvf /root/etc_slashdb.tar --strip 1
cd /var/log && tar xvf /root/var_log_slashdb.tar --strip 1

3. Reassign the IP address/hostname to the new instance and shut down the old one

If you made a reference copy of the /etc/slashdb/slashdb.ini file on the 1.7.x container, copy the changes you want from the reference file to the /etc/slashdb/slashdb.ini file. You may want to use a tool like sdiff to resolve any differences. See INI File differences between 1.6.x and 1.7.x for more details.

Finally, restart the SlashDB process:

systemctl restart slashdb-app

INI File Differences Between 1.6.x and 1.7.x

In 1.7.0 INI file:

SlashDB Config

Added

+ # name of SlashDB memcache lock key
+ ;memcache_lock_key = slashdb_registry_lock
+ # max time in seconds to wait before trying to access memcache when lock exists
+ ;memcache_lock_sleep_interval = 0.1

+ # default CommandTimeout in seconds when using adPersistXML option
+ xml_sdb.recordset_command_timeout = 30

+ # JWT_SDB Config
+ # default timeout in seconds for JWT HTTP operations
+ jwt_sdb.http_timeout = 1

+ # SAML_SDB Config
+ # default timeout in seconds for SAML HTTP operations
+ saml_sdb.http_timeout = 5

+ # Amazon Config
+;amazon_host = 169.254.169.254
+ # default timeout in seconds for Amazon HTTP operations
+ ;amazon_http_timeout = 1

Logging Config

Changed

[formatter_slashdb]

- format = %(asctime)-.19s %(created)d%(msecs)03d [sdbapp] [pid: %(process)d %(processName)s %(threadName)s] %(levelname)-5.5s %(name)s %(module)s:%(funcName)s:%(lineno)d [%(user_id)s] %(message)s
+ format = %(asctime)-.19s | %(created)d%(msecs)03d | sdbapp | %(levelname)-5.5s | %(processName)s | %(threadName)s | %(name)s | %(module)s:%(funcName)s:%(lineno)d | %(user_id)s | %(message)s | pid: %(process)d

[formatter_generic]

- format = %(asctime)-.19s %(created)d%(msecs)03d [sdbapp] [pid: %(process)d %(processName)s %(threadName)s] %(levelname)-5.5s %(name)s %(module)s:%(funcName)s:%(lineno)d [] %(message)s
+ format = %(asctime)-.19s | %(created)d%(msecs)03d | sdbapp | %(levelname)-5.5s | %(processName)s | %(threadName)s | %(name)s | %(module)s:%(funcName)s:%(lineno)d |  | %(message)s | pid: %(process)d

[uwsgi]

- log-date = %%Y-%%m-%%d %%H:%%M:%%S
- logformat-strftime = 1
- log-format = %(ftime) %(tmsecs) [uwsgi ] [pid: %(pid) wid: %(wid)] %(addr) %(cl) [pktsize: %(pktsize) bytes] %(method) %(uri) => generated %(rsize) bytes in %(msecs) msecs (%(proto) %(status)) %(headers) headers in %(hsize) bytes (%(switches) switches on core %(core))
+ # logging
+ logto = /var/log/slashdb/uwsgi.log
+ logfile-chown = slashdb:slashdb
+ # 20 MB
+ log-maxsize = %(20 * 1024 * 1024)

+ log-encoder = format ${strftime:%%Y-%%m-%%d %%H:%%M:%%S} | ${millis} | uwsgi  | INFO  |  |  |  |  |  | ${msgnl}

+ # requests-logging
+ log-format = "%(method) %(uri) %(proto)" | pid: %(pid) wid: %(wid) | %(addr) | %(status) | %(secs) s |  | resp %(size) bytes | ipktsize: %(pktsize) bytes | reqbody: %(cl) bytes | %(headers) headers | in %(hsize) bytes | %(switches) switches | on core %(core)

INI File Differences Between 1.7.0 and 1.7.29

In 1.7.29 INI file:

Added

uWSGI options for reloading workers.


+
+;harakiri = n                   ; Reload worker after n seconds waiting for response
+;max-requests = n               ; Reload worker after n requests serviced
+;max-worker-lifetime = n        ; Reload worker after n seconds
+;reload-on-rss = n              ; Reload worker once resident memory goes over n megabytes
+;evil-reload-on-rss = n         ; Reload worker forcefully once resident memory goes over n megabytes
+;worker-reload-mercy = n        ; How long to wait before forcefully killing worker

Removed


-# URL substitution configuration
-;url.substitution.__ = /
-;url.substitution._dot = .

-html_sdb.stream_by_default = False

-xsd_sdb.stream_by_default = False