This five-day workshop covers data science and machine learning workflows at scale using Apache Spark 2 and other key components of the Hadoop ecosystem. The workshop emphasizes the use of data science and machine learning methods to address real-world business challenges.
Using scenarios and datasets from a fictional technology company, students discover insights to support critical business decisions and develop data products to transform the business. The material is presented through a sequence of brief lectures, interactive demonstrations, extensive hands-on exercises, and discussions.
The workshop is designed for data scientists who currently use Python to work with smaller datasets on a single machine and who need to scale up their analyses and machine learning models to large datasets on distributed clusters. Data engineers and developers with some knowledge of data science and machine learning may also find this workshop useful.
Workshop participants should have a basic understanding of Python and some experience exploring and analyzing data and developing statistical or machine learning models. Knowledge of Hadoop or Spark is not required.
Overview of data science and machine learning at scale
Overview of the Hadoop ecosystem
Working with HDFS data and Hive tables using Hue
Introduction to Cloudera Data Science Workbench
Overview of Apache Spark
Reading and writing data
Inspecting data quality
Cleansing and transforming data
Summarizing and grouping data
Combining, splitting, and reshaping data
Configuring, monitoring, and troubleshooting Spark applications
Overview of machine learning in Spark MLlib
Extracting, transforming, and selecting features
Building and evaluating regression models
Building and evaluating classification models
Building and evaluating clustering models
Cross-validating models and tuning hyperparameters
Building machine learning pipelines
Deploying machine learning models