Pipelingo Docs

Everything you need to connect your stack, monitor pipelines, and route alerts.

Getting started

  1. Create an account at pipelingo.com/login. Email and password — no credit card.
  2. Create or join an organization. If you sign up via an invite link, you'll be added to the inviter's org. Otherwise you start solo and can create one from Settings → Team.
  3. Connect a data source. dbt is recommended; see below.
  4. Configure alerts at Settings → Alerts.

You can use Pipelingo without dbt — Snowflake-only is fine — but dbt unlocks lineage, model-level health, and test results.

Connect dbt

Three integration paths, pick whichever matches your setup:

Option 1 — Upload artifacts (free, any dbt)

Run dbt run or dbt build locally and upload the resulting manifest.json (and optionally run_results.json) at Settings.

# After dbt finishes, you'll have:
target/manifest.json     # required — gives us models + lineage
target/run_results.json  # optional — gives us pipeline runs + test results

Re-upload any time the project changes.

Option 2 — dbt Cloud API (paid tier)

Paste your dbt Cloud API token and Account ID in Settings. We pull recent runs and the latest successful manifest automatically. Requires Starter+ tier (Cloud's API isn't available on the free Developer plan).

Option 3 — CI integration (recommended for prod)

One-time setup that auto-uploads on every CI run. See CI integration below.

Connect Snowflake (optional)

For teams that want raw warehouse query tracking on top of dbt — or are using Snowflake without dbt at all. Go to Settings → Snowflake and enter:

  • Account — e.g. xy12345.us-east-1
  • Username
  • Database and warehouse — defaults are SNOWFLAKE and COMPUTE_WH
  • Auth method — password or key-pair (see below)

Credentials are encrypted with Fernet (AES-128) before they hit our database. The plaintext value never persists. We read your QUERY_HISTORY view, filter to actual transformation work (CREATE / CREATE_TABLE_AS / INSERT / SELECT / COPY), and pretty-name pipelines from the target table.

Key-pair authentication (Snowflake MFA-compliant)

Snowflake is enforcing MFA for human users in 2025–26 (see Strong Authentication Hub in Snowsight). Service accounts can't do MFA, so the compliant alternative is RSA key-pair authentication. Pipelingo supports either method — pick "Key-pair" in the Snowflake form to use it.

One-time setup:

# 1. Generate a 2048-bit RSA key (no passphrase, simplest)
openssl genrsa 2048 | openssl pkcs8 -topk8 -inform PEM -out rsa_key.p8 -nocrypt
openssl rsa -in rsa_key.p8 -pubout -out rsa_key.pub

# 2. In Snowflake, attach the PUBLIC key to a service user:
ALTER USER pipelingo_svc SET RSA_PUBLIC_KEY='<paste contents of rsa_key.pub minus header/footer>';

# 3. Verify
DESC USER pipelingo_svc;  -- look for RSA_PUBLIC_KEY_FP

# 4. Paste the contents of rsa_key.p8 (the PRIVATE key) into Pipelingo's
#    "Private key (PEM)" field. Leave passphrase blank since we used -nocrypt.

Encrypt the key in transit by adding a passphrase: re-run step 1 without -nocrypt and supply the passphrase in the Pipelingo form. Pipelingo encrypts the PEM at rest regardless.

For dedicated service users, Snowflake recommends granting them the SERVICE type so they bypass MFA enrollment entirely:

CREATE USER pipelingo_svc TYPE = SERVICE
  RSA_PUBLIC_KEY = '<your public key>'
  DEFAULT_ROLE = pipelingo_role
  DEFAULT_WAREHOUSE = compute_wh;

The dashboard

The main view shows one card per pipeline, color-coded by source (orange = dbt, sky = Snowflake, purple = Control-M) with a 3px left-border in the latest run's state color so failures jump out.

  • Health score — % of successful runs in the last 7 days.
  • vs last week toggle — week-over-week health delta.
  • Status pills on top — click to filter the grid.
  • Bulk Analyze — when failures exist, fan out AI analysis to all of them in one click.

Click a card to open the pipeline detail view.

AI analysis

On any failed pipeline, click Run Technical Analysis for an engineer view: root cause, suggested fix, prevention steps. Or toggle to Stakeholder for a plain-English summary you can paste in Slack — "Pipeline X failed because Y, expected fix in ~Z hours."

What Claude sees

For dbt failures we bundle the per-result fields from run_results.json (failure count, target relation, execution time, and most importantly the compiled SQL) plus, when available, the matching slice of logs/dbt.log. The log slice surfaces adapter-level errors that aren't in the structured artifact — Snowflake error codes with session IDs, schema-not-found errors, credential issues, etc.

Successful runs never get the log attached — keeps the prompt focused on what's broken and the cached analysis small.

Powered by Claude. Each analysis is cached, so re-opening the same failure costs nothing. There's a daily per-user quota (default 100 fresh analyses) to keep costs bounded.

Lineage & blast radius

The Lineage page renders your dbt manifest as a layered graph (sources → staging → marts → reports). Click any node to highlight its blast radius — every downstream model that goes stale if this one fails.

This is the answer to "if fct_revenue breaks, what reports are affected?" without you having to remember the lineage.

Alerts

Configure where pipeline failures should notify you at Settings → Alerts.

Channels

  • Email — any address, delivered via Resend.
  • SMS — any phone number, delivered via Twilio.
  • Slack — paste a Slack incoming webhook URL; messages land in the channel you set up the webhook for.

Subscriptions

Subscribe a channel to all pipelines or to specific ones, with trigger options:

  • On failure (default) — fires when a run flips to failed.
  • On success — useful for "the nightly batch ran cleanly" pings.
  • All state changes — chatty, mostly used for debugging.

Inline AI summary in failure alerts (opt-in)

By default the alert message is brief — "your pipeline failed, click here." Engineers open Pipelingo and click Run Technical Analysis to get the AI-generated fix.

If you'd rather the proposed fix arrive in the email/SMS/Slack message, toggle on "Include AI summary in failure alerts" at Settings → Alerts. When enabled, every new failure transition triggers a Claude stakeholder summary that's embedded directly in the alert body. Cached summaries are reused for free; fresh generations count against the daily AI quota (default 100/day per user). If the quota is exhausted, the alert still fires — just without the inline summary.

Quiet hours & daily digest

Set quiet hours and a digest delivery time so non-critical alerts batch into one daily summary instead of pinging at 2 a.m. Failures still fire immediately regardless of quiet hours — that's usually the whole point. Successes and other non-critical state changes during quiet hours are recorded with status skipped in alert history and rolled up into the digest.

The digest is delivered to every email channel configured in the org (or the auth account's email for solo users) at the time you set, in your selected timezone.

Self-hosting? Wire your scheduler to POST /api/internal/digest/run hourly with header Authorization: Bearer $INTERNAL_CRON_SECRET. The endpoint is idempotent — calling it more than once an hour won't double-send.

Alerts only fire on actual state transitions — a pipeline that's already failed and stays failed won't spam you.

Teams & roles

Pipelingo is multi-tenant. Connections, pipelines, alerts, and analyses are all scoped to your organization with row-level security in Postgres — one org cannot see another's data.

Roles:

  • Owner — full access; can transfer or delete the org.
  • Admin — can manage connections, members, and alerts.
  • Member — can view dashboards, run AI analyses, and configure their own notification preferences.
  • Viewer — read-only.

Invites are sent by email and expire after 7 days. Manage at Settings → Team.

Multiple dbt projects

Enterprises usually have more than one dbt project — finance, marketing, data-science, etc. Pipelingo lets a single org connect each one separately so models with the same name (e.g. stg_orders in two projects) appear as distinct pipelines on the dashboard, get distinct lineage graphs, and don't collide on run history.

Create a project

At Settings → Projects, click + Add project. Give it a slug (lowercase, used in URLs and CI uploads) and a display name. The first dbt connection you make is auto-routed to a default project so existing setups keep working unchanged.

Connect each project

The project picker at the top of the Projects section sets which project the dbt connection forms below it apply to. Select marketing, click Connect dbt, upload that project's manifest.json; switch to finance, repeat.

From CI

Add a project_slug form field to your curl upload (or omit it to land in default):

curl -fsS -X POST \
  -H "Authorization: Bearer $PIPELINGO_TOKEN" \
  -F "manifest=@target/manifest.json" \
  -F "run_results=@target/run_results.json" \
  -F "project_slug=marketing" \
  https://api.pipelingo.com/api/dbt/ingest

One CI token works for every project in the same org — the form field is what routes the upload.

Filtering on the dashboard

The dashboard's toolbar gets a project picker when more than one project exists. Pick one to scope the grid; pick "All projects" to see everything together.

Permissions today vs later

Project membership is currently delegated to org membership — every member of the org can see every project. Per-project access control (so a Marketing analyst can't see Finance pipelines) is on the roadmap; the schema is already designed to layer it on without breaking changes.

CI integration

The cleanest setup: every time your dbt project runs in CI, the artifacts upload automatically. No manual uploads, no exposing dbt Cloud creds, no scheduled poll needed.

1. Generate a CI token

Go to Settings → CI Automation and click Generate CI token. Copy it immediately — we don't show it again. Tokens are valid for 365 days.

The token encodes your user_id + org_id with the server's ENCRYPTION_KEY; rotating that key invalidates every issued token.

2. Add it as a GitHub secret

In your dbt repo: Settings → Secrets and variables → Actions → New repository secret. Name it PIPELINGO_TOKEN and paste the value.

3. Add the workflow file

Create .github/workflows/pipelingo-sync.yml with the contents below. Adjust the project name and your SNOWFLAKE_* secrets for whatever profile your dbt project uses.

name: Pipelingo Sync

on:
  push:
    branches: [main]
  workflow_dispatch:
  schedule:
    - cron: '0 8 * * *'  # daily at 08:00 UTC

jobs:
  build-and-sync:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: '3.12'

      - name: Install dbt
        run: pip install dbt-core dbt-snowflake

      - name: Configure dbt profile
        run: |
          mkdir -p ~/.dbt
          cat > ~/.dbt/profiles.yml <<EOF
          your_project:
            target: prod
            outputs:
              prod:
                type: snowflake
                account: ${{ secrets.SNOWFLAKE_ACCOUNT }}
                user: ${{ secrets.SNOWFLAKE_USER }}
                password: ${{ secrets.SNOWFLAKE_PASSWORD }}
                database: ${{ secrets.SNOWFLAKE_DATABASE }}
                warehouse: ${{ secrets.SNOWFLAKE_WAREHOUSE }}
                schema: analytics
                threads: 4
          EOF

      - name: dbt build
        run: |
          dbt deps || true
          dbt run
          dbt test
        continue-on-error: true  # still upload even if tests fail

      - name: Upload artifacts to Pipelingo
        if: always()
        run: |
          ARGS=(-F "manifest=@target/manifest.json")
          [ -f target/run_results.json ] && ARGS+=(-F "run_results=@target/run_results.json")
          [ -f logs/dbt.log ] && ARGS+=(-F "dbt_log=@logs/dbt.log")
          curl -fsS -X POST \
            -H "Authorization: Bearer ${{ secrets.PIPELINGO_TOKEN }}" \
            "${ARGS[@]}" \
            https://api.pipelingo.com/api/dbt/ingest

What gets uploaded

  • target/manifest.json → updates your model graph and lineage.
  • target/run_results.json → creates one pipeline_run per dbt result; appends to history (each invocation is a new row).
  • logs/dbt.log (optional but recommended) → the raw dbt log. For any failure we attach the matching slice to the AI analysis so Claude sees adapter-level error text alongside the structured artifact.

Troubleshooting

  • 401 Unauthorized — token wrong or expired. Regenerate at Settings.
  • 413 / payload too large — manifest exceeds 10 MB. Try dbt compile instead of dbt run for the upload step, or trim deselected nodes.
  • 400 "not valid JSON" — dbt didn't finish writing the artifact. Make sure dbt deps ran and that the upload step uses if: always().
  • No runs appearing — you uploaded the manifest but not run_results.json. Add -F "run_results=@target/run_results.json" to the curl.

A working reference repo lives at CovenantDataSolutions/pipelingo-demo-dbt.

FAQ

Where does my data live?

Pipelingo stores pipeline metadata (run names, states, timestamps, error messages) in a Supabase Postgres database. Snowflake/dbt credentials are encrypted with Fernet before they hit the DB. We never store warehouse data — only metadata about your runs.

Can I delete my data?

Yes. Go to Settings → Account → Delete account. If you're the only owner of an org, the org and all its pipelines, connections, and alert history are deleted with you.

Does AI analysis require my own API key?

No — Pipelingo provides Claude API access. There's a daily per-user quota (default 100) to keep costs predictable; cached analyses don't count.

What happens when a pipeline I'm not subscribed to fails?

It still appears on the dashboard and the bulk-analyze panel. Subscriptions only affect notifications.

How do quiet hours work?

During quiet hours, non-critical alerts (success notifications, "already failed" repeats) are suppressed. Genuine new failures still notify immediately because that's usually the point.

Can I self-host?

Pipelingo is a hosted product — there's no self-host option. If you have a specific compliance, residency, or single-tenant need, get in touch and we can talk through a dedicated deployment.