Least-Privilege Service Accounts for a BigQuery Pipeline
Re-cut the ETL pipeline so each stage runs as its own service account with only the permissions that stage needs — read this bucket, write that dataset, nothing more — and prove a stolen key from one stage can't touch the rest.
#iam#data#security
LabHoarderly, an analytics startup that wires every job to one "editor-on-everything" service account because it was faster, until a leaked key turned that single account into the keys to the entire data warehouse.all labs
02 - Actions
score -- - -- votes
-- completed
Sign in with a real account to save ratings, completions, and share snapshots.
Stateloading
03 - Scenario
Hoarderly, an analytics startup that wires every job to one "editor-on-everything" service account because it was faster, until a leaked key turned that single account into the keys to the entire data warehouse.
Re-cut the ETL pipeline so each stage runs as its own service account with only the permissions that stage needs — read this bucket, write that dataset, nothing more — and prove a stolen key from one stage can't touch the rest.
Constraints
No primitive roles (roles/editor, roles/owner) anywhere in the pipeline
Each pipeline stage gets a distinct service account scoped to its job
Dataset and bucket access granted at the resource level, not the project
Everything in Terraform, with a written blast-radius analysis per account
ScenarioGCP - advanced
05 - Steps
STEP_01
One service account per stage
Create a distinct service account for each pipeline stage (ingest, transform, publish). Name them for their job so an audit log entry tells you exactly which stage acted.
Grant the ingest account object-read on only its source bucket, scoped to that bucket rather than the project. The transform and publish accounts get nothing here.
Hint: Resource-level bindings beat project-level every time — a project-wide storage.objectViewer can read every bucket you'll ever create, forever.
Give the transform account dataEditor on the staging dataset and the publish account dataEditor on the published dataset — each on its own dataset, not the project. Reads and writes never overlap.
The publish stage needs to run query jobs but must not modify table schemas. No predefined role draws that line, so define a custom role with exactly that permission set and bind it.
For each service account, write down what a stolen key could reach and what it could not. Confirm by impersonating each account and attempting an action it should be denied — the deny is the deliverable.