Database Connections — User Guide
Audience: Dagen users who add and manage connections to databases, warehouses, lakes, streaming systems, and object storage for their workspace.
Overview
Open Database Connections at /db-connections. Here you create, edit, delete, test, and extract metadata for saved connections. That metadata feeds schema inventory, Data Modeling, agents, Data Ingestion (where compatible), and other features.
A full type-by-type overview—including Iceberg catalogs and ingestion overlap—is in Supported data sources, warehouses, and metadata.
Your workspace may restrict which connection types appear (admin policy).
Page layout
| Area | Purpose |
|---|---|
| Header | Title Database Connections, short description, and Add New Connection (disabled while you are editing another connection in the dialog). |
| Page alert | Success or error messages for save/delete and similar operations. |
| Existing Connections | Grid of connection cards (responsive layout). |
| Available Databases (Across Connections) | List of database names discovered across connections, each labeled with its connection name. |
| Add / Edit dialog | Modal form for new connections or edits; title switches between Add New DB Connection and Edit DB Connection. |
Empty state: If you have no connections yet, you’ll see a prompt to add your first one.
Loading / errors: A spinner appears while connections load; fetch errors show a warning at the top.
What you can connect (summary)
| Category | Examples |
|---|---|
| Relational / MPP | PostgreSQL, MySQL, Oracle, Amazon Redshift, Teradata, Hive |
| Cloud warehouses | Snowflake, Google BigQuery |
| Streaming / SaaS / storage | Kafka, Salesforce, Amazon S3, Google Cloud Storage, Azure Blob Storage |
| Lake | Apache Iceberg — catalog modes include REST, AWS Glue, Hive Metastore, Nessie; storage on S3, GCS, or Azure |
| Other | Apache Ozone |
Connection tests are type-aware (including Iceberg, Kafka, and common object-storage patterns).
Supported database types — required / typical fields
The connection type dropdown includes the native types below (not only PostgreSQL through Hive). Exact labels in the UI may vary slightly by release. Streaming and object-storage types usually show extra fields (TLS, SASL, region, bucket, prefix, etc.)—follow the live form.
| Type | Category | Required / typical fields |
|---|---|---|
| PostgreSQL | Relational | Host, Port, Database name, Username, Password |
| MySQL | Relational | Host, Port, Database name, Username, Password |
| Oracle | Relational | Host, Port, SID or Service name (toggle), Username, Password |
| Amazon Redshift | Relational / MPP | Cluster endpoint (host), Port, Database name, Username, Password |
| Teradata | Relational | Host, Port, Database name, Username, Password |
| Apache Hive | Relational / MPP | Host, Port, Database / metastore-related fields, Username, Password |
| Snowflake | Cloud warehouse | Account identifier, Warehouse, Database, Schema, Username, Password |
| Google BigQuery | Cloud warehouse | Project ID, Dataset, Service account key (JSON) |
| Apache Kafka | Streaming | Bootstrap servers; security (SASL, SSL/TLS); client authentication as shown in the form |
| Salesforce | SaaS | Instance / OAuth and API credentials (per form) |
| Amazon S3 | Object storage | Region; credentials (e.g. access key/secret or IAM-based); bucket and path options (per form) |
| Google Cloud Storage | Object storage | Project; credentials; bucket/path (per form) |
| Azure Blob Storage | Object storage | Storage account; container; SAS or connection string / key (per form) |
| Apache Iceberg | Lake | Catalog type (e.g. REST, AWS Glue, Hive Metastore, Nessie); catalog connection details; warehouse path; storage backend (S3, GCS, Azure); credentials for catalog and storage |
| Apache Ozone | Other | Ozone endpoints / volume; authentication (per form) |
If a type is missing from your dropdown, the workspace may use allowed-types restrictions—ask an admin. For ingestion-only systems, see also Supported data sources and Data Ingestion.
Adding or editing a connection
- Click Add New Connection (or Edit on a card).
- Enter a connection name and choose the type.
- Complete the form fields for that type (see table above).
- Submit to save.
Edit: Opens the same form with existing values. While a connection is open for edit, Add New Connection may be unavailable until you finish or cancel.
Connection cards
Each card shows:
- Icon and type — visual hint for the system you connected.
- Name — your label for this connection.
- Summary fields — type-specific (e.g. host/port/database for Postgres; account/warehouse for Snowflake; project/dataset for BigQuery).
- Added — when the connection was created.
Actions on each card
| Action | What it does |
|---|---|
| Extract Metadata | Runs a metadata scan for this connection. Shows progress on the button; success or error may appear on the card. On success, Available Databases refreshes. |
| Edit | Opens the form with current settings. |
| Delete | Removes the connection (confirm if prompted). |
From elsewhere in the workspace you can also use a saved connection for ingestion, pipelines, data model flows, and exploration—those entry points vary by feature; starting from Data Ingestion or Data Modeling often lets you pick a connection you defined here.
Secret managers and vaults
For credential fields you can often use Secret Manager mode and reference an external vault (for example HashiCorp Vault) instead of storing secrets directly in the form.
The Secret / Vault Providers section (when present) lists configured vault integrations. Use Test Connection on a provider to verify it is reachable. Setup may depend on backend configuration—follow in-app guidance.
Testing connections
- Use Test Connection (or the type-specific test, e.g. for BigQuery) before relying on a connection for production jobs.
- Failures are usually credentials, network path (firewall, VPN, allow lists), or wrong endpoint/catalog for Iceberg and cloud warehouses.
Metadata extraction
- Extract Metadata updates Dagen’s view of schemas and objects for that connection and powers downstream features.
- Agents and tools generally discover schemas and tables with lighter calls before deeper extraction, so exploration stays efficient.
- Structured extraction is strongest for common warehouse and relational paths (e.g. Snowflake, Redshift, PostgreSQL, Oracle); behaviour aligns with your connection type.
More context: Supported data sources — metadata.
Available databases
The Available Databases (Across Connections) list shows database names discovered from your connections, with each row tied to the connection it came from. If nothing appears, run Extract Metadata or confirm the connection test passes.
Troubleshooting
| Symptom | Likely cause | What to try |
|---|---|---|
| Connection type missing in the form | Workspace allowed types policy | Ask a workspace admin |
| Add New Connection disabled | Edit dialog is open | Finish or cancel the current edit |
| Test fails | Network, credentials, or firewall | Verify host, ports, secrets; allow Dagen to reach the endpoint |
| Metadata empty or partial | Permissions or empty catalog | Grant metadata/read access; confirm schemas exist |
| Extract Metadata error on card | Timeout or engine error | Retry; narrow scope; check logs if your role allows |
| Vault option inactive | No vault provider configured | Configure providers per admin docs / in-app alerts |
Related
- Supported data sources — Full native list and ingestion notes
- Data Ingestion — Syncs using connectors and workspace connections
- Database Explorer — Query and browse after connections work
- Runtime Environments — Where some workloads execute