Databricks

Databricks - Data Intelligence Platform

The SlashDB Databricks connector adds support for Databricks data lakes, further extending SlashDB's generated REST API.

Important

This feature is only available in Enterprise Edition

Installation

SlashDB version >= 1.5.0 must be installed before the connector can be installed. If you don't have SlashDB yet, see the Installation documentation.

From System Package

If SlashDB was installed from the DEB or RPM package, or if you have SlashDB running from a virtual machine or cloud images, the easiest way to add Databricks support is by downloading and installing the corresponding connector package of the same type.

Debian or Ubuntu

wget https://downloads.slashdb.com/versions/2.0.1/slashdb-databricks_1.3.0_amd64.deb
sudo apt-get update
sudo apt-get install -y ./slashdb-databricks_1.3.0_amd64.deb

Red Hat

wget https://downloads.slashdb.com/versions/2.0.1/slashdb-databricks-1.3.0.x86_64.rpm
sudo dnf install -y ./slashdb-databricks-1.3.0.x86_64.rpm

SlashDB will then automatically restart for the changes to take effect.

From Python Wheel

If SlashDB was installed as a Python package, it is recommended to install the Databricks connector as a Python package in the same environment.

Available wheel packages:

• https://downloads.slashdb.com/versions/2.0.1/slashdb_databricks-1.3.0-cp39-cp39-manylinux2014_x86_64.whl

• https://downloads.slashdb.com/versions/2.0.1/slashdb_databricks-1.3.0-cp310-cp310-manylinux2014_x86_64.whl

• https://downloads.slashdb.com/versions/2.0.1/slashdb_databricks-1.3.0-cp311-cp311-manylinux2014_x86_64.whl

• https://downloads.slashdb.com/versions/2.0.1/slashdb_databricks-1.3.0-cp312-cp312-manylinux2014_x86_64.whl

Download and install the package that matches your version of Python, e.g.:

wget https://downloads.slashdb.com/versions/2.0.1/slashdb_databricks-1.3.0-cp39-cp39-manylinux2014_x86_64.whl
/opt/slashdb/bin/pip install ./slashdb_databricks-1.3.0-cp39-cp39-manylinux2014_x86_64.whl

After installation, SlashDB must be restarted manually.

Configuration

License

Uploading a Valid License

Log in as admin and navigate to the Manage > License page.

If your existing license does not allow connections to Databricks databases, the Databricks icon on the page will be grey and have the text "Unlicensed". In that case, you will need to upload a valid license.

If you need a license, please contact us at licensing@slashdb.com.

If you already have a license, please see the License documentation for information about uploading it.

When the new license is uploaded, the Databricks icon will be colorized and the "Unlicensed" text will no longer appear. You can proceed to create a connection to your Databricks data lake.

Acquiring Databricks Credentials

Token

Generate and copy the token as described in the Databricks documentation.

Hostname, HTTP Path

Copy the server hostname and HTTP path from the cluster configuration:

For details, please see the Databricks documentation

Database Name

In the Azure Workspace, in the sidebar go to Data > Databases to list them. Or keep blank to use the default.

More details can be found in the Databricks documentation

Adding a Database Instance

1. In the menu, click on Database Connections, then click the New button in the top right corner

   

2. Select the Databricks type from the database selection screen

   

3. Fill in the configuration details for your database, then click the Next button

   

4. Set the database ID and other options for your database connection, then click the Create button

   

Editing a Database Connection

To edit an existing database connection, go to the Database Connections list and click the Edit icon in the Actions column, or click on the database ID.

The edit form is divided into three sections: Connection, Configuration, Privileges

Connection

• Description - an optional description for the connection
• Type - the database vendor type
• Database Charset - the character set used for string types
• Database Schema - the name of the Databricks database schema to use
• Database Host - the hostname of the Databricks server
• Database Port - the port number that the database server listens on
• Database Catalog - the name of the Databricks catalog
• HTTP Path - the Databricks HTTP Path

Configuration

• Connect Automatically - automatically connect to the database whenever SlashDB starts

• Cache Schema - cache the database schema after the first connection so that subsequent connections complete faster

  Important

  Whenever your database structure changes, you will need to disable this option, disconnect/reconnect, and then enable it again. If your database structure changes frequently, you may want to disable it entirely.

• Auto Discover - automatically find all tables and views in your database whenever SlashDB connects

Database Credentials

Pick one of:

• Local - the Database Login and Database Password to use for database reflection
• Credential Reference - the Database Credentials to use for database reflection

Privileges

• Connect / Disconnect DB - users with privileges to control the database connection
• View DB Config - users with privileges to view the database configuration
• Modify DB Config - users with privileges to edit or delete the database configuration

Updating

To update, install a newer version of the package.

Important

After updating the core SlashDB RPM/DEB package, you must reinstall plugins