Database Sources

Database sources use dlt’s sql_database connector, which supports any SQLAlchemy-compatible database. Data is synced to parquet and queryable alongside your other sources.

General usage

dinobase add <type> --connection-string <url>
dinobase sync

Credentials can also be set via the DATABASE_URL environment variable.

Supported databases

PostgreSQL

dinobase add postgres --connection-string postgresql://user:pass@host:5432/dbname

Metadata: Column comments and foreign key relationships extracted from pg_catalog.

MySQL

dinobase add mysql --connection-string mysql://user:pass@host:3306/dbname

MariaDB

MySQL-compatible.

dinobase add mariadb --connection-string mysql://user:pass@host:3306/dbname

Microsoft SQL Server

dinobase add mssql --connection-string mssql+pyodbc://user:pass@host/dbname

Requires: pip install pyodbc

Oracle

dinobase add oracle --connection-string oracle://user:pass@host:1521/service

Requires: pip install cx_Oracle

SQLite

dinobase add sqlite --path /path/to/database.db

Snowflake

dinobase add snowflake --connection-string snowflake://user:pass@account/db/schema

Requires: pip install snowflake-sqlalchemy

Google BigQuery

dinobase add bigquery --connection-string bigquery://project/dataset

Requires: pip install sqlalchemy-bigquery

Amazon Redshift

dinobase add redshift --connection-string redshift+psycopg2://user:pass@host:5439/db

Requires: pip install sqlalchemy-redshift

ClickHouse

dinobase add clickhouse --connection-string clickhouse://user:pass@host:8123/db

Requires: pip install clickhouse-sqlalchemy

CockroachDB

PostgreSQL-compatible.

dinobase add cockroachdb --connection-string postgresql://user:pass@host:26257/db

Databricks

dinobase add databricks --connection-string databricks://token:TOKEN@host:443/db

Requires: pip install databricks-sql-connector

Trino

dinobase add trino --connection-string trino://user@host:8080/catalog/schema

Requires: pip install trino

PrestoDB

dinobase add presto --connection-string presto://user@host:8080/catalog/schema

Requires: pip install pyhive

DuckDB (external file)

dinobase add duckdb_source --connection-string duckdb:///path/to/file.duckdb

Connection string format

Connection strings follow SQLAlchemy format:

dialect+driver://username:password@host:port/database

Examples:

postgresql://admin:secret@db.example.com:5432/myapp
mysql://root:pass@localhost:3306/analytics
snowflake://user:pass@xy12345.us-east-1/MYDB/PUBLIC

Syncing databases

Database sources sync all user tables by default. After syncing, tables are accessible as source_name.table_name:

dinobase add postgres --connection-string ... --name production
dinobase sync
dinobase query "SELECT * FROM production.users LIMIT 5" --pretty