Source Repositories — User Guide

Audience: Engineers connecting Git to Dagen for dbt, Dataform, Spark, agent code, and Git Reviews.


Overview

Connect Repository (/connect-repo) links GitHub (OAuth) or manual Git remotes to your workspace. Connected repos power:

  • Building Pipelines — dbt/Dataform/Spark projects
  • Automatic pipeline detection when project markers exist
  • Git Reviews — repo picker for PR automation
  • VS Code / local editing flows

GitHub OAuth path (recommended)

If GitHub is already linked

  • A GitHub Repositories card lists repos your token can see.
  • Select a repo to connect it to the workspace.

If GitHub is not linked

  • Card Connect GitHub Account with an authorize button.
  • Completing OAuth grants repo listing (and often the scopes needed for webhooks later).
  • Success may show a snackbar confirmation.

After connect

  • Repository connected snackbar.
  • If the repo contains a recognized project (e.g. dbt_project.yml), Dagen may auto-create a Data Pipeline and optionally navigate you to /pipelines after a short delay.
  • If no pipeline is auto-created, you still have the repo for manual pipeline setup or Git Reviews.

Manual repository connection

Use the Manual Repository Connection card when:

  • The remote is not on GitHub, or
  • You prefer URL + branch + credentials (PAT, deploy key) over OAuth UI.

Typical fields: repository URL, branch (often main), credentials for private repos.


PAT (Personal Access Token) workflow

For many GitHub operations you need a PAT with appropriate scope:

  1. Configure Default PAT in the header (when available).
  2. Paste token with at least repo for private repositories (public_repo only for public).
  3. Save—future manual adds may auto-fill the token.

Create a PAT on GitHub: Settings → Developer settings → Personal access tokens → generate with scopes → copy once.

Security: PATs are sensitive—treat like passwords; rotate on team changes.


Managing connected repositories

  • Manage Existing tab (or header control) lists connected repos.
  • Update branch, credentials, or disconnect as needed.
  • Removing a repo may affect pipelines and Git Reviews configs that reference it.

VS Code integration

The VS Code Integration card supplies a token or deep link pattern (product-specific) so you can:

  • Open the same repo in desktop VS Code
  • Keep edits in sync with Dagen’s inline or web editors (Building Pipelines Git menu)

User flow summary

  1. Open Connect Repository.
  2. Link GitHub if needed → browse repos or use manual URL.
  3. Connect → snackbar → optional auto-pipeline + redirect to Pipelines.
  4. Use Pipelines, Spark, Agent Builder, or Git Reviews with the connected code.

Troubleshooting

Symptom Cause Fix
Connect button missing Already linked / wrong auth state Read on-page alert
PAT warning Local storage reminder Expected; ensure machine is trusted
Pipeline not auto-created No dbt_project.yml / supported layout Create pipeline manually from Pipelines
Webhook / PR review fails later Repo renamed or token revoked Re-link GitHub; update Git Reviews config

Related