GCP ELT Pipeline
IntermediateDashboardFivetran to BigQuery, dbt transforms, Dagster orchestrates, Metabase visualizes on GCP.
Published 27 September 2026
About GCP ELT Pipeline
The GCP ELT Pipeline uses Google Cloud's ecosystem for a managed, scalable analytics stack. Fivetran handles data ingestion with 300+ managed connectors that keep data continuously synchronized into BigQuery. dbt transforms raw BigQuery data into clean models with SQL, tests, and documentation. Dagster orchestrates the pipeline as typed assets with built-in lineage visualization. Metabase provides the BI dashboard layer on top of the dbt-modeled BigQuery tables.
BigQuery's serverless architecture eliminates cluster management: it scales automatically and charges per query, making it cost-effective for variable analytical workloads. Fivetran's fully managed connectors mean zero ingestion maintenance once set up. Dagster's asset-based model tracks data lineage from source to dashboard, making debugging and impact analysis straightforward.
This is a premium, low-maintenance data stack for organizations that prioritize reliability and scalability over total cost control.
Key Features
- ✓Fivetran fully managed connectors with automatic schema migration to BigQuery
- ✓BigQuery serverless SQL data warehouse with no cluster provisioning
- ✓dbt SQL transformations with testing and column-level documentation
- ✓Dagster asset-based orchestration with visual lineage and data catalog
- ✓Metabase BI dashboards connected to dbt-modeled BigQuery views
- ✓GCP IAM and VPC for secure data access across the pipeline
When to Use GCP ELT Pipeline
- →GCP-centric organizations building a production analytics pipeline with minimal maintenance
- →Teams that want fully managed ingestion to eliminate connector maintenance overhead
- →Data teams that prioritize lineage visibility and pipeline reliability over cost
- →Organizations with variable query patterns where BigQuery's per-query pricing is efficient
- →Analytics programs that need audit trails and data catalog capabilities via Dagster
Pros
- Fivetran fully managed connectors require almost zero ingestion maintenance
- BigQuery serverless eliminates cluster sizing and management decisions
- Dagster asset lineage provides clear visibility into data dependencies
- Metabase OSS keeps visualization costs low on top of a premium pipeline
Cons
- Fivetran is one of the most expensive ingestion tools per connector-month
- BigQuery per-query costs can become significant for exploratory heavy users
- Four distinct tools add integration surface and learning overhead
Orchestrator Options for GCP ELT Pipeline
GCP ELT Pipeline with Dagster
The default: asset-based orchestration where each BigQuery table and dbt model is a tracked asset with lineage, backed by first-party dagster-gcp and dagster-dbt integrations.
GCP ELT Pipeline with Apache Airflow
Swap in Airflow for the most battle-tested option on GCP: Google's own managed offering (Cloud Composer) runs it natively, and its BigQuery and dbt operators are mature.
GCP ELT Pipeline with Prefect
Swap in Prefect for lighter-weight orchestration: the prefect-gcp collection provides BigQuery tasks, and dynamic flows suit pipelines whose steps vary by branch or config.
GCP ELT Pipeline Add-ons
Each addition below extends this stack with a capability the base stack works fine without. None are required: include the ones your product actually needs when building this stack, and skip the rest.
CI/CD Add-ons
Add CI/CD when you want a dedicated pipeline for running tests, linting, or multi-stage builds before a deploy goes out. Many hosting platforms already redeploy automatically on every push on their own — a CI/CD tool adds the most value on top of that by gating the deploy on a passing test suite, and matters even more when the hosting choice does not auto-deploy at all, such as a self-hosted server.
GCP ELT Pipeline with GitHub Actions
Runs dbt test and validates Dagster job definitions on every push, catching a broken model or asset before it reaches the BigQuery tables Metabase reads from.
GCP ELT Pipeline with GitLab CI/CD
The same dbt-and-Dagster validation step, for teams running this pipeline's code from a GitLab repo instead.
These are highlighted picks. To see all the tools, check the CI/CD Pipelines category.
Containerization Add-ons
Add containerization when you want the app packaged the same way across local development, staging, and production, or need to deploy somewhere that isn't a managed serverless platform.
GCP ELT Pipeline with Docker
Package the dbt project and any custom Dagster job code identically across a laptop, CI, and wherever the scheduled run happens.
These are highlighted picks. To see all the tools, check the Containerization category.
Frequently Asked Questions about GCP ELT Pipeline
Fivetran or Airbyte for ingestion?
Fivetran is fully managed with essentially no setup, billed by monthly active rows synced. Airbyte is free to self-host and has more connectors overall, at the cost of running and maintaining it yourself.
Why Dagster instead of Airflow for orchestration?
Dagster's asset-based model tracks the data each step produces, not just the tasks that ran, which makes debugging a broken pipeline and understanding data lineage more direct than Airflow's task-centric DAGs.
How do I keep BigQuery query costs predictable?
On on-demand pricing you pay for bytes scanned, so the goal is to scan less. Partition large tables by date and cluster them on the columns people filter by, and turn on the require-partition-filter setting so a query without a date range fails instead of reading the whole table. Build big dbt models as incremental so each run processes only new data. On the Metabase side, dashboards query BigQuery every time they load, so point them at small aggregated dbt models rather than raw tables and enable Metabase's query caching. A per-user or per-project cap on bytes billed stops one runaway query from becoming the month's biggest line item.
Where do Dagster and Metabase run in a serverless setup?
Not in BigQuery or Fivetran, so they need a home. Fivetran and BigQuery are fully managed, but open-source Dagster and Metabase are services you host. On GCP the common choices are Cloud Run or a small Compute Engine VM for Metabase, and a VM or a GKE cluster for Dagster, each with a Cloud SQL PostgreSQL database for its own metadata (Metabase's built-in H2 database isn't meant for production). The alternative is to buy those two layers managed as well: Dagster+ and Metabase Cloud are hosted versions, at a subscription that replaces the hosting work.
How is this different from the Airbyte + dbt + Snowflake + Tableau stack?
Both are warehouse-plus-dbt pipelines; the choices around dbt differ. This stack stays on Google Cloud with BigQuery, uses Fivetran for fully managed ingestion, orchestrates with Dagster, and keeps BI cheap with open-source Metabase. The Snowflake stack is cloud-neutral, uses Airbyte (free to self-host), and pays for Tableau seats to get deeper enterprise dashboards. Pick this one when the company already runs on GCP and wants ingestion that needs no attention; pick the Snowflake stack when the warehouse must span clouds or when a large business audience needs Tableau's dashboard depth.
Stacks Related to GCP ELT Pipeline
MLOps Pipeline
Project
End-to-end ML pipelines from training to production monitoring.
Databricks Lakehouse Pipeline
Project
Databricks unified lakehouse for large-scale data engineering, ML, and SQL analytics.
Streaming Analytics Pipeline
Project
Real-time streaming analytics with Kafka, dbt, ClickHouse, and Grafana dashboards.
Modern ELT Stack
Project
Airbyte extracts into Snowflake, dbt transforms, Airflow orchestrates: the modern ELT standard.
Scores
Tools in the GCP ELT Pipeline Stack
GCP ELT Pipeline Pricing
dbt Core, Dagster, and Metabase's open-source edition are all free to self-host. Fivetran and BigQuery are the real cost drivers: Fivetran bills by monthly active rows past its free tier, and BigQuery bills by query bytes scanned and storage, so total cost tracks data volume and how often the pipeline runs rather than a flat fee.
The free tier covers light usage; paid tiers bill by monthly active rows synced, which scales with data volume.
Billed by query bytes scanned and storage; GCP's Always Free tier covers a modest amount of both each month.
All three are free to self-host with no usage limits.