Introduction

This course Provides instruction on the processes and practice of data science, including machine learning and natural language processing.

Included are: tools and programming languages (Python, IPython, Mahout, Pig, NumPy, pandas, SciPy, Scikit learn), the Natural Language Toolkit (NLTK), and Spark MLlib.

Audience

Architects, software developers, analysts and data scientists who need to apply data science and machine learning on Hadoop.

Prerequisites

Students must have experience with at least one programming or scripting language, knowledge in statistics and/or mathematics,and a basic understanding of big data and Hadoop principles.

Students new to Hadoop are encouraged to attend the HDPOverview: Apache Hadoop Essentials course.

Outcomes

• recognise use cases for data science on Hadoop
• describe the Hadoop and YARN architecture
• describe supervised and unsupervised learning differences
• use Mahout to run a machine learning algorithm on Hadoop
• describe the data science life cycle
• use Pig to transform and prepare data on Hadoop
• write a Python script
• describe options for running Python code on a Hadoop cluster
• write a Pig User-Defined Function in Python
• use Pig streaming on Hadoop with a Python script
• use machine learning algorithms
• describe use cases for Natural Language Processing (NLP)
• use the Natural Language Toolkit (NLTK)
• describe the components of a Spark application
• write a Spark application in Python
• run machine learning algorithms using Spark MLlib
• take data science into production.