title
Please take a moment to fill out this form. We will get back to you as soon as possible.
All fields marked with an asterisk (*) are mandatory.
Data Engineering on Google Cloud Platform
Course Description
Overview
This Data Engineering on Google Cloud Platform course is designed to provide participants a hands-on introduction to designing and building data processing systems on Google Cloud Platform. Through a combination of presentations, demos, and hand-on labs, participants will learn how to design data processing systems, build end-to-end data pipelines, analyze data and carry out machine learning. The course covers structured, unstructured, and streaming data.Objectives
- Design and build data processing systems on Google Cloud Platform
- Process batch and streaming data by implementing autoscaling data pipelines on Cloud Dataflow
- Derive business insights from extremely large datasets using Google BigQuery
- Train, evaluate and predict using machine learning models using Tensorflow and Cloud ML
- Leverage unstructured data using Spark and ML APIs on Cloud Dataproc
- Enable instant insights from streaming data
Audience
- Extracting, Loading, Transforming, cleaning, and validating data
- Designing pipelines and architectures for data processing
- Creating and maintaining machine learning and statistical models
- Querying datasets, visualizing query results and creating reports
Prerequisites
- Completed Google Cloud Fundamentals: Big Data & Machine Learning course OR have equivalent experience
- Basic proficiency with common query language such as SQL
- Experience with data modeling, extract, transform, load activities
- Developing applications using a common programming language such Python
- Familiarity with Machine Learning and/or statistics
Topics
- Explore the role of a data engineer.
- Analyze data engineering challenges.
- Intro to BigQuery.
- Data Lakes and Data Warehouses.
- Demo: Federated Queries with BigQuery.
- Transactional Databases vs Data Warehouses.
- Website Demo: Finding PII in your dataset with DLP API.
- Partner effectively with other data teams.
- Manage data access and governance.
- Build production-ready pipelines.
- Review GCP customer case study.
- Lab: Analyzing Data with BigQuery.
- Introduction to Data Lakes.
- Data Storage and ETL options on GCP.
- Building a Data Lake using Cloud Storage.
- Optional Demo: Optimizing cost with Google Cloud Storage • classes and Cloud Functions.
- Securing Cloud Storage.
- Storing All Sorts of Data Types.
- Video Demo: Running federated queries on Parquet and ORC files in BigQuery.
- Cloud SQL as a relational Data Lake.
- Lab: Loading Taxi Data into Cloud SQL.
- The modern data warehouse.
- Intro to BigQuery.
- Demo: Query TB+ of data in seconds.
- Getting Started.
- Loading Data.
- Video Demo: Querying Cloud SQL from BigQuery.
- Lab: Loading Data into BigQuery.
- Exploring Schemas.
- Demo: Exploring BigQuery Public Datasets with SQL using • INFORMATION_SCHEMA.
- Schema Design.
- Nested and Repeated Fields.
- Demo: Nested and repeated fields in BigQuery.
- Lab: Working with JSON and Array data in BigQuery.
- Optimizing with Partitioning and Clustering.
- Demo: Partitioned and Clustered Tables in BigQuery.
- Preview: Transforming Batch and Streaming Data.
- EL, ELT, ETL.
- Quality considerations.
- How to carry out operations in BigQuery.
- Demo: ELT to improve data quality in BigQuery.
- Shortcomings.
- ETL to solve data quality issues.
- The Hadoop ecosystem.
- Running Hadoop on Cloud Dataproc.
- GCS instead of HDFS.
- Optimizing Dataproc.
- Lab: Running Apache Spark jobs on Cloud Dataproc.
- Cloud Dataflow.
- Why customers value Dataflow.
- Dataflow Pipelines.
- Lab: A Simple Dataflow Pipeline (Python/Java).
- Lab: MapReduce in Dataflow (Python/Java).
- Lab: Side Inputs (Python/Java).
- Dataflow Templates.
- Dataflow SQL.
- Building Batch Data Pipelines visually with Cloud Data Fusion.
- Components.
- UI Overview.
- Building a Pipeline.
- Exploring Data using Wrangler.
- Lab: Building and executing a pipeline graph in Cloud Data Fusion.
- Orchestrating work between GCP services with Cloud Composer.
- Apache Airflow Environment.
- DAGs and Operators.
- Workflow Scheduling.
- Optional Long Demo: Event-triggered Loading of data with Cloud Composer, Cloud Functions, Cloud Storage, and BigQuery.
- Monitoring and Logging.
- Lab: An Introduction to Cloud Composer.
- Processing Streaming Data.
- Cloud Pub/Sub.
- Lab: Publish Streaming Data into Pub/Sub.
- Cloud Dataflow Streaming Features.
- Lab: Streaming Data Pipelines.
- BigQuery Streaming Features.
- Lab: Streaming Analytics and Dashboards.
- Cloud Bigtable.
- Lab: Streaming Data Pipelines into Bigtable.
- Analytic Window Functions.
- Using With Clauses.
- GIS Functions.
- Demo: Mapping Fastest Growing Zip Codes with BigQuery GeoViz.
- Performance Considerations.
- Lab: Optimizing your BigQuery Queries for Performance.
- Optional Lab: Creating Date-Partitioned Tables in BigQuery.
- What is AI?.
- From Ad-hoc Data Analysis to Data Driven Decisions.
- Options for ML models on GCP.
- Unstructured Data is Hard.
- ML APIs for Enriching Data.
- Lab: Using the Natural Language API to Classify Unstructured Text.
- Whats a Notebook.
- BigQuery Magic and Ties to Pandas.
- Lab: BigQuery in Jupyter Labs on AI Platform.
- Ways to do ML on GCP.
- Kubeflow.
- AI Hub.
- Lab: Running AI models on Kubeflow.
- BigQuery ML for Quick Model Building.
- Demo: Train a model with BigQuery ML to predict NYC taxi fares.
- Supported Models.
- Lab Option 1: Predict Bike Trip Duration with a Regression Model in BQML.
- Lab Option 2: Movie Recommendations in BigQuery ML.
- Why Auto ML?
- Auto ML Vision.
- Auto ML NLP.
- Auto ML Tables.
Related Courses
-
Machine Learning with Sagemaker (AWS)
DCSK-125- Duration: 3 Days
- Delivery Format: Classroom Training, Online Training
- Price: ???
-
From Data to Insights with Google Cloud Platform
GCP-320- Duration: 3 Days
- Delivery Format: Classroom Training, Online Training
- Price: 2,241.00 USD
Self-Paced Training Info
Learn at your own pace with anytime, anywhere training
- Same in-demand topics as instructor-led public and private classes.
- Standalone learning or supplemental reinforcement.
- e-Learning content varies by course and technology.
- View the Self-Paced version of this outline and what is included in the SPVC course.
- Learn more about e-Learning
Course Added To Shopping Cart
bla
bla
bla
bla
bla
bla
Self-Paced Training Terms & Conditions
Exam Terms & Conditions
Sorry, there are no classes that meet your criteria.
Please contact us to schedule a class.
![self-paced](fynwyd/images/course-outline-self-paced-35x35.png)
STOP! Before You Leave
Save 0% on this course!
Take advantage of our online-only offer & save 0% on any course !
Promo Code skip0 will be applied to your registration
Purchase Information
title
Please take a moment to fill out this form. We will get back to you as soon as possible.
All fields marked with an asterisk (*) are mandatory.