These are hands-on Do-It-Now activities for the parts of the data engineering stack that sit outside BigQuery and Dataflow: orchestration, governance, Spark, open table formats, and data protection.

Each activity is self-contained: it illustrates one concept, brings its own setup, and can be done on its own. That matters more here than elsewhere — this topic deliberately spans unrelated services, so there is no order to follow. Pick whichever activity matches what you're learning.

Cost

Every activity shows its cost under the title — 🟢 No cost, 💰 Low cost, or 💸 Higher cost.

What you'll need

• A lab-provided project, with the APIs for the service an activity uses enabled
• A Cloud Storage bucket for staging files and job output
• Cloud Shell, which has gcloud and the client libraries already installed

Individual activities state anything beyond this in their own setup section.

Sample code

• Cloud Composer — DAGs for BigQuery export patterns, plus a local DAG validation script: composer/
• Dataplex — data profiling with sample source files, and lineage tooling: dataplex/
• Dataproc — cluster scaling and autoscaling policy demos: dataproc/
• Lakehouse — open table formats, including Apache Iceberg: lakehouse/
• Sensitive data — inspection and de-identification with Cloud DLP: dlp-demo/