These are hands-on Do-It-Now activities for Dataflow, Google Cloud's managed runner for Apache Beam pipelines.

Each activity is self-contained: it illustrates one concept, brings its own setup, and can be done on its own. Work them in any order, or do just the ones you need.

Cost

Every activity shows its cost under the title — 🟢 No cost, 💰 Low cost, or 💸 Higher cost.

What you'll need

• A lab-provided project, with the Dataflow, Pub/Sub, BigQuery, and Cloud Storage APIs enabled
• A Cloud Storage bucket for staging and temporary files
• Cloud Shell, or a local Python environment with the Beam SDK installed

Sample code

• Basic Beam pipelines that run locally on the DirectRunner: dataflow/simple_demos
• A full streaming pipeline — Pub/Sub to Dataflow to BigQuery, with a setup script and an event generator: dataflow/dflow-bq-stream-python