SlashDB Server Settings

The INI file /etc/slashdb/slashdb.ini is the core configuration file for setting up SlashDB.

The file contains settings of:

• SlashDB features like branding, default rendering,
• WSGI application and middlwares,
• logging,
• uWSGI server

Changes to this file require restating the SlashDB service with command:

sudo service slashdb restart

Below you can find explained options and examples.

Application section

The section [app:sdbapp] contains WSGI application configuration

use

Option use defines entrypoint for WSGI application. Do not modify.

use = egg:slashdb

mako.directories

Folder with Mako templates for GUI. Modify this option to use different set of templates. Value can be either file system path or location in reference to Python package.

Default: slashdb.formats.html_sdb:templates

Example:

mako.directories = /path/to/folder/with/templates

static_path

Normally SlashDB is delivered with static files being served by NGINX from folder /var/lib/slashdb/static and requests for static files don't even reach WSGI server.

Modify this option to use different set of static files when running pserve.

Value can be either file system path or location in reference to Python package.

Default: slashdb.formats.html_sdb:static

Example:

static_path = /var/lib/slashdb/static

branding.name

This options allows to rename the API. The default name SlashDB appears in various places like root tag for XML. Modify this option to change the product name.

Default: SlashDB

Example:

branding.name = AwesomeAPI

databases_path

Database configurations are stored in YAML file in /etc/slashdb folder. If needed the path can be modified to point other file if configuration is stored on different volume.

Default: /etc/slashdb/databases.cfg

Example:

databases_path = /mnt/remote-storage/slashdb/databases.cfg

users_path

User configurations are stored in YAML file in /etc/slashdb folder. If needed the path can be modified to point other file if configuration is stored on different volume.

Default: /etc/slashdb/users.cfg

Example:

users_path = /mnt/remote-storage/slashdb/users.cfg

querydefs_path

Query configurations are stored in YAML file in /etc/slashdb folder. If needed the path can be modified to point other file if configuration is stored on different volume.

Default: /etc/slashdb/querydefs.cfg

Example:

querydefs_path = /mnt/remote-storage/slashdb/querydefs.cfg

initial_state_path

This option tells SlashDB what file to use when running SlashDB for the first time. The file is removed by SlashDB after setting up admin password. Don't modify unless configuration files are stored in different folder than /etc/slashdb on first run.

Default: /etc/slashdb/regular-initial

Example:

initial_state_path = /mnt/remote-storage/slashdb/regular-initial

license

This option tells SlashDB where to find license file. If needed the path can be modified to point other file if configuration is stored on different volume.

Default: /etc/slashdb/license.key

Example:

license = /mnt/remote-storage/slashdb/license.key

caches_folder

This option tells SlashDB where it should store any schemas that are cached. If the given path does not exist or is not usable for any reason, then there will be debug logs on startup warning that caching will not function without a valid path.

Default: /var/cache/slashdb/models/

Example:

caches_folder = /var/cache/slashdb/models/

require_details_in_terms_and_conditions

When set to True it will be necessary to provide first name, last name and e-mail during first run.

Default: True

Example:

require_details_in_terms_and_conditions = False

html_limit

This option limits the number of records viewed in GUI (HTML). At the same time pagination is enabled and allows to see the rest of data. This is to prevent freezing browser when it to renders a web page with thousands of records.

Default: 29

Example:

html_limit = 139

ignore_default_fields

If set to True it will override default behavior of below columns:

• In MS SQL, columns of TIMESTAMP type will be updated by server not by the passed value.

Default: False

Example:

ignore_default_fields = False

refid_prefix

This option defines the name of attribute used in JSON to keep resource url.
For example the Artist resource has __href to itself and to related object list Album.

[
  {
    "Album": {
      "__href": "/db/Chinook/Artist/ArtistId/1/Album.json"
    },
    "ArtistId": 1,
    "Name": "AC/DC",
    "__href": "/db/Chinook/Artist/ArtistId/1.json"
  }
]

Default: __href

Example:

refid_prefix = __resource_url

url.substitution

This options allows to automatically substitute some character sequence in url with other. A typical case is when url segment contains a slash character / e.g. /db/Chinook/Artist/Name/AC/DC/. To prevent from slash being interpreted as url segment separator and separate AC and DC we can define other sequence to represent slash; for example two underscores __. Then the url would look like /db/Chinook/Artist/Name/AC__DC/ and will be interpreted correclty as AC/DC.

Default:

url.substitution.__ = /
url.substitution._dot = .

Example:
Interpret ^^ as / for segment values.

url.subsitution.^^ = /

Sessions

User sessions are stored in file system.

session.data_dir = %(here)s/sessions/data
session.lock_dir = %(here)s/sessions/lock

API key

The header and query string argument names can be defined to match environment and other systems requirements e.g. 3Scale.

app_id_name (Not used in single parameter credentials)
apikey_name

Example

Changed apikey_name to "secret".

# Header or get parameter authentication.
# Two parameters credentials use appid and apikey (called app_id mode in 3Scale).
# Single parameter credentials userapikey (called user_key mode in 3Scale)
appid_name = appid
apikey_name = secret

Request with modified apikey_name in query string

curl -i 'localhost:8888/db.json?appid=mobile-app-1&secret=v8fcyt1l8jbpp58157eo16456ms9ap61'

HTTP/1.1 200 OK

Request with modified apikey_name in header

curl -i 'localhost:8888/db.json' /
-H 'appid:mobile-app-1' /
-H 'secret:v8fcyt1l8jbpp58157eo16456ms9ap61'

HTTP/1.1 200 OK

SlashDB settings

Branding name

Used in various places where name of the API is needed. For example when URL serach string count is used the header will all records count will contain branding name i.e. SlashDB-All-Record-Count.

Exception in initialization process also has the banding name Development license can be generated and installed only when SlashDB is not already initialized.

Branding name also shows up in settings https://demo.slashdb.com/settings.json under key product.

branding.name = SlashDB

HTTP responses encoding

output_encoding = utf-8

Paths to configuration files

databases_path = /etc/slashdb/databases.cfg
users_path = /etc/slashdb/users.cfg
querydefs_path = /etc/slashdb/querydefs.cfg
initial_state_path = /etc/slashdb/regular-initial
license = /etc/slashdb/license.key

Require terms and conditions in initial setup

Makes accepting terms and conditions required in welcome slides for development license.

require_details_in_terms_and_conditions = True

Register instances

URL used to save user details from welcome slides registration

registration_url = https://demo.slashdb.com/db/SDB/license.json?apikey=7nNH8C7Ge1IPSJvt3l

Default resource limit for GUI

Default limit of resources displayed in GUI. This value is added to URI search string limit=29.

html_limit = 29

Default timestamp value for MSSQL

ignore_default_fields = False  

If True then for MSSQL database server, on insert the timestamp type columns will be updated by server not by the passed value.
False by default.

Models package

models_package = slashdb.models  

Defines where automatically reflected models are kept in package structure. Only for advanced setup.

Reference attribute in JSON

refid_prefix = __href  

For JSON format it defines an object key that keeps URL to the resource.

URL substitution configuration
Default configuration of URL substitution. Sometime there's a need to use URL reserved characters as part of Using syntax url.substitution.__ = / you can change how characters from URL are replaced. Sometimes there's a need to pass in the URL characters that are reserved for URLs. For example slash separates segments. For passing url.append_substitution = False

Sesssion cookie

• How often to reissue a new cookie in sec. Suggested 1/10 of cookie_timeout

  ini cookie_reissue_time = 360

• How long a ticket in cookie valid in sec.

  ini cookie_timeout = 3600

• How long a cookie is valid in sec.

  ini cookie_max_age = 3600

Processes synchronization

SlashDB can work either in multi-thread or multi-processes mode. When using multi-processes mode Memcached server has to be configured to synchronize instances of SlashDB. See section UWSGI for information how to switch between these modes.

memcached_server = 127.0.0.1:11211

API Key

User can be authenticated by passing API Key in URL search parameters or in header. The name of the parameter and header can be configured.

appid_name = appid
apikey_name = apikey

SQLAlchemy Settings

Configuration specific for sqlachemy_sdb component.

Log SQL statements when reflecting databases.

True if you want to log all sql queries sent while reflecting the database.

sqlalchemy_sdb.engine_echo = False

Define MySQL driver

MySQL driver can be switched.

# MySQL-Python (default)
sqlalchemy_sdb.mysql_driver = mysqldb

# PyMySQL
sqlalchemy_sdb.mysql_driver = pymysql

# MySQL-Connector
sqlalchemy_sdb.mysql_driver = mysqlconnector

Reflecting database method

If True using reflection mechanism provided by SQLAlchemy Automap. If False then SlashDB 0.8.x method is used.

sqlalchemy_sdb.use_sqlalchemy_automap = True

Default: True

Database output chunks size for streaming

When streaming responses ?stream=True also data from a database if fetched gradually. With yield_per option you can define how many records are fetched at once.

sqlalchemy_sdb.yield_per = 10

Default: 50

Formats config

Include href to output

xml_sdb.href = True
json_sdb.href = True

Default: True

Logging Section

The section is located between lines # Logging Config and # End of Logging Config and contains configuration of loggers, handlers and formatters. The format of that is standard file config for Python 3 logging.

For more details please see:

• Logging configuration in SlashDB with examples
• Python 3 Logging HOWTO

Full INI file example

Be default the files is located in /etc/slashdb/slashdb.ini

[app:sdbapp]

# Pyramid config
use = egg:slashdb
mako.directories = slashdb.formats.html_sdb:templates
static_path = slashdb.formats.html_sdb:static

# SlashDB Config
branding.name = SlashDB
databases_path = /etc/slashdb/databases.cfg
users_path = /etc/slashdb/users.cfg
querydefs_path = /etc/slashdb/querydefs.cfg
initial_state_path = /etc/slashdb/regular-initial
license = /etc/slashdb/license.key
require_details_in_terms_and_conditions = True
auth_config_path = /etc/slashdb/auth.cfg
caches_folder = /var/cache/slashdb/models/

html_limit = 29
ignore_default_fields = False
refid_prefix = __href

# URL substitution configuration
;url.substitution.__ = /
;url.substitution._dot = .


# IMPORTANT: Please generate a new secret key for this SlashDB instance.
# Instances running in a cluster should have the same secret key.
# Keep the secret key used in production secret!
session.secret = 84eXRmVUdsMCRzSDycbLEQmej8wKRR

# How long a session is valid in sec.
session.timeout = 3600

# How long a cookie is valid in sec.
session.cookie_expires = 86400

# Where session data should be stored
session.data_dir = /var/lib/slashdb/sessions/data

# Where session locked data should be stored
session.lock_dir = /var/lib/slashdb/sessions/lock

# Cookie httpOnly and secure flags
cookie_http_only = True
cookie_secure = auto

# set password strength requirement from 0 to 4
min_password_strength = 3
allow_empty_password = False

# Memcached server
;memcached_server = 127.0.0.1:11211
;memcache_root_key = slashdb

# Header or get parameter authentication.
# Two parameters credentials use appid and apikey (called app_id mode in 3Scale).
# Single parameter credentials useapikey (called user_key mode in 3Scale)
appid_name = appid
apikey_name = apikey

# Should SlashDB stream data by default
stream_by_default = False

# sqlalchemy_sdb config
sqlalchemy_sdb.yield_per = 100
# sqlalchemy mysql diver: mysql+[driver_name]://
sqlalchemy_sdb.mysql_driver = mysqldb
sqlalchemy_sdb.reflect_nopk_views = True
# per database SQLAlchemy engine config
;sqlalchemy_sdb.db.Chinook.pool_size = 1
;sqlalchemy_sdb.db.Chinook.max_overflow = 0
;sqlalchemy_sdb.db.Chinook.pool_recycle = 900
;sqlalchemy_sdb.db.Chinook.pool_timeout = 10

# Default limit to prevent request with unlimited rows. Configurable per database.
# When not set, requests without limit in URL will fetch all rows.
# When set, requests without limit in URL are redirected to URL with the defined limit.
;db.Chinook.limit = 10000

# CSV_SDB Config
csv_sdb.stream_by_default = False
csv_sdb.headers_by_default = True
;csv_sdb.csv_null_str = nil

# HTML_SDB Config
html_sdb.stream_by_default = False
html_sdb.float_decimal_digits = 7

# XML_SDB Config
xml_sdb.root_tag = SlashDB
xml_sdb.xmlns = http://www.vtenterprise.com/slashdb
xml_sdb.nil_visible = True
xml_sdb.stream_by_default = False
xml_sdb.href = True

# XSD_SDB Config
xsd_sdb.stream_by_default = False
xsd_sdb.root_tag = SlashDB
xsd_sdb.xmlns = http://www.vtenterprise.com/slashdb

# JSON_SDB Config
json_sdb.indent = 4
json_sdb.minimize = False
json_sdb.stream_by_default = False
json_sdb.href = True

# TXT_SDB Config
txt_sdb.nil_visible = False
txt_sdb.stream_by_default = False

[server:main]
use = egg:waitress#main
host = 0.0.0.0
port = 6543
threads = 2

[pipeline:main]
pipeline = cors sdbapp

[filter:cors]
use = egg:wsgicors#middleware
# CORS available policies
# "free" policy
free_origin = copy
free_headers = *
free_expose_headers = Set-Cookie, Warning
free_methods = HEAD, OPTIONS, GET, POST, PUT, DELETE
free_maxage = 180
free_credentials = true

# CORS enabled policies
policy = free


# Logging Config

[loggers]
keys = root, slashdb, exc_logger, sqlalchemy
# sqlachemy_engine, sqlachemy_pool

[handlers]
keys = console, console_slashdb, exc_handler

[formatters]
keys = generic, exc_formatter, slashdb

[logger_root]
level = INFO
handlers = console

[logger_slashdb]
level = INFO
handlers = console_slashdb
qualname = slashdb
propagate = 0

[logger_exc_logger]
level = ERROR
handlers = exc_handler
qualname = exc_logger
propagate = 1

[logger_sqlalchemy]
level = WARNING
handlers =
qualname = sqlalchemy
propagate = 1

[logger_sqlalchemy_engine]
level = INFO
handlers =
qualname = sqlalchemy.engine
propagate = 1

[logger_sqlalchemy_pool]
level = INFO
handlers =
qualname = sqlalchemy.pool
propagate = 1

[handler_console]
class = StreamHandler
args = (sys.stderr,)
level = NOTSET
formatter = generic

[handler_console_slashdb]
class = StreamHandler
args = (sys.stderr,)
level = NOTSET
formatter = slashdb

[handler_exc_handler]
class = handlers.RotatingFileHandler
args = ('/var/log/slashdb/exceptions.log','a', 2*1024*1024, 5, 'utf-8')
level = ERROR
formatter = exc_formatter

[formatter_exc_formatter]
format = %(asctime)s %(message)s

[formatter_slashdb]
format = %(asctime)-.19s %(created)d%(msecs)03d [sdbapp] [pid: %(process)d %(processName)s %(threadName)s] %(levelname)-5.5s %(name)s %(module)s:%(funcName)s:%(lineno)d [%(user_id)s] %(message)s

[formatter_generic]
format = %(asctime)-.19s %(created)d%(msecs)03d [sdbapp] [pid: %(process)d %(processName)s %(threadName)s] %(levelname)-5.5s %(name)s %(module)s:%(funcName)s:%(lineno)d [] %(message)s

# End of Logging Config


# uWSGI config for service script starts uWSGI as a daemon
[uwsgi]
socket = 127.0.0.1:8001
virtualenv = /opt/slashdb
logto = /var/log/slashdb/slashdb.log
logfile-chown = slashdb:slashdb
log-maxsize = 20971520
master = true
enable-threads = true
single-interpreter = true
lazy-apps = true
need-app = true
processes = 1
threads = 2
paste = config:%p
paste-logger = %p
vacuum = true
auto-procname = true
procname-prefix-spaced = slashdb
master-fifo = /run/slashdb/master-fifo
listen = 0
log-date = %%Y-%%m-%%d %%H:%%M:%%S
logformat-strftime = 1
log-format = %(ftime) %(tmsecs) [uwsgi ] [pid: %(pid) wid: %(wid)] %(addr) %(cl) [pktsize: %(pktsize) bytes] %(method) %(uri) => generated %(rsize) bytes in %(msecs) msecs (%(proto) %(status)) %(headers) headers in %(hsize) bytes (%(switches) switches on core %(core))