Introduction to Data Science

•Need for Data Scientists

•Foundation of Data Science

•What is Business Intelligence

•What is Data Analysis

•What is Data Mining

•What is Machine Learning

•Analytics vs Data Science

•Value Chain

•Types of Analytics

•Lifecycle Probability

•Analytics Project Lifecycle

Data

•Basis of Data Categorization

•Types of Data

•Data Collection Types

•Forms of Data & Sources

•Data Quality & Changes

•Data Quality Issues

•Data Quality Story

•What is Data Architecture

•Components of Data Architecture

•OLTP vs OLAP

•How is Data Stored?

Big Data

•What is Big Data?

•5 Vs of Big Data

•Big Data Architecture

•Big Data Technologies

•Big Data Challenge

•Big Data Requirements

•Big Data Distributed Computing & Complexity

•Hadoop

•Map Reduce Framework

•Hadoop Ecosystem

Data Science Deep Dive

•What Data Science is

•Why Data Scientists are in demand

•What is a Data Product

•The growing need for Data Science

•Large Scale Analysis Cost vs Storage

•Data Science Skills

•Data Science Use Cases

•Data Science Project Life Cycle & Stages

•Map Reduce Framework

•Hadoop Ecosystem

•Data Acuqisition

•Where to source data

•Techniques

•Evaluating input data

•Data formats

•Data Quantity

•Data Quality

•Resolution Techniques

•Data Transformation

•File format Conversions

•Annonymization

Intro to R Programming

•Introduction to R

•Business Analytics

•Analytics concepts

•The importance of R in analytics

•R Language community and eco-system

•Usage of R in industry

•Installing R and other packages

•Perform basic R operations using command line

•Usage of IDE R Studio and various GUI

R Programming Concepts

•The datatypes in R and its uses

•Built-in functions in R

•Subsetting methods

•Summarize data using functions

•Use of functions like head(), tail(), for inspecting data

•Use-cases for problem solving using R

Data Manipulation in R

•Various phases of Data Cleaning

•Functions used in Inspection

•Data Cleaning Techniques

•Uses of functions involved

•Use-cases for Data Cleaning using R

Data Import Techniques in R

•Import data from spreadsheets and text files into R

•Importing data from statistical formats

•Packages installation for database import

•Connecting to RDBMS from R using ODBC and basic SQL queries in R

•Web Scraping

•Other concepts on Data Import Techniques

Exploratory Data Analysis (EDA) using R

•What is EDA?

•Why do we need EDA?

•Goals of EDA

•Types of EDA

•Implementing of EDA

•Boxplots, cor() in R

•EDA functions

•Multiple packages in R for data analysis

•Some fancy plots

•Use-cases for EDA using R

Data Visualization in R

•Story telling with Data

•Principle tenets

•Elements of Data Visualization

•Infographics vs Data Visualization

•Data Visualization & Graphical functions in R

•Plotting Graphs

•Customizing Graphical Parameters to improvise the plots

•Various GUIs

•Spatial Analysis

•Other Visualization concepts

HADOOP

Big Data and Hadoop Introduction

•What is Big Data and Hadoop?

•Challenges of Big Data

•Traditional approach Vs Hadoop

•Hadoop Architecture

•Distributed Model

•Block structure File System

•Technologies supporting Big Data

•Replication

•Fault Tolerance

•Why Hadoop?

•Hadoop Eco-System

•Use cases of Hadoop

•Fundamental Design Principles of Hadoop

•Comparison of Hadoop Vs RDBMS

Understand Hadoop Cluster Architecture

•Hadoop Cluster & Architecture

•5 Daemons

•Hands-On Exercise

•Typical Workflow

•Hands-On Exercise

•Writing Files to HDFS

•Hands-On Exercise

•Reading Files from HDFS

•Hands-On Exercise

•Rack Awareness

•Before Map Reduce

Map Reduce Concepts

•Map Reduce Concepts

•What is Map Reduce?

•Why Map Reduce?

•Map Reduce in real world.

•Map Reduce Flow

•What is Mapper?

•What is Reducer?

•What is Shuffling?

•Word Count Problem

•Hands-On Exercise

•Distributed Word Count Flow & Solution

•Log Processing and Map Reduce

•Hands-On Exercise

Advanced Map Reduce Concepts

•What is Combiner?

•Hands-On Exercise

•What is Partitioner?

•Hands-On Exercise

•What is Counter?

•Hands-On Exercise

•InputFormats/Output Formats

•Hands-On Exercise

•Map Join using MR

•Hands-On Exercise

•Reduce Join using MR

•Hands-On Exercise

•MR Distributed Cache

•Hands-On Exercise

•Using sequence files & images with MR

•Hands-On Exercise

•Planning for Cluster & Hadoop 2.0 Yarn

•Configuration of Hadoop

•Choosing Right Hadoop Hardware?

•Choosing Right Hadoop Software?

•Hadoop Log Files?

Hadoop 2.0 & YARN

•Hadoop 1.0 Challenges

•NN Scalability

•NN SPOF & HA

•Job Tracker Challenges

•Hadoop 2.0 New Features

•Hadoop 2.0 Cluster Architecture & Federation

•Hadoop 2.0 HA

•Yarn & Hadoop Ecosystem

•Yarn MR Application Flow

PIG

•Introduction to Pig

•What Is Pig?

•Pig’s Features & Pig Use Cases

•Interacting with Pig

•Basic Data Analysis with Pig

•Hands-On Exercise

•Pig Latin Syntax

•Loading Data

•Hands-On Exercise

•Simple Data Types

•Field Definitions

•Data Output

•Viewing the Schema

•Hands-On Exercise

•Filtering and Sorting Data

•Hands-On Exercise

•Commonly-Used Functions

•Hands-On Exercise: Pig for ETL Processing

•Processing Complex Data with Pig

•Hands-On Exercise

•Storage Formats

•Complex/Nested Data Types

•Hands-On Exercise

•Grouping

•Hands-On Exercise

•Built-in Functions for Complex Data

•Hands-On Exercise

•Iterating Grouped Data

•Hands-On Exercises

•Multi-Dataset Operations with Pig

•Hands-On Exercise

•Techniques for Combining Data Sets

Practice Set – 1

•Joining Data Sets in Pig

•Hands-On Exercise

•Splitting Data Sets

•Hands-On Exercise

HIVE

•Hive Fundamentals & Architecture

•Loading and Querying Data in Hive

•Hands-On Exercise

•Hive Architecture and Installation

•Comparison with Traditional Database

•HiveQL: Data Types, Operators and Functions,

•Hands-On Exercise

•Hive Tables ,Managed Tables and External Tables

•Hands-On Exercise

•Partitions and Buckets

•Hands-On Exercise

•Storage Formats, Importing Data, Altering Tables, Dropping Tables

•Hands-On Exercise

•Querying Data, Sorting and Aggregating, Map Reduce Scripts,

•Hands-On Exercise

Practice Set – 2

•Joins & Sub queries, Views

•Hands-On Exercise

•Integration, Data manipulation with Hive

•Hands-On Exercise

•User Defined Functions,

•Hands-On Exercise

•Appending Data into existing Hive Table

•Hands-On Exercise

•Static partitioning vs dynamic partitioning

•Hands-On Exercise

HBASE

•CAP Theorem

•HBase Architecture and concepts

•Introduction to HBase

•Client API’s and their features

•HBase tables The ZooKeeper Service

•Data Model, Operations

Practice Set – 3

•Programming and Hands on Exercises

SQOOP

•Introduction to Sqoop

•MySQL Client & server

•Connecting to relational data base using Sqoop

•Importing data using Sqoop from Mysql

•Exporting data using Sqoop to MySql

•Incremental append

•Importing data using Sqoop from Mysql to hive

•Exporting data using Sqoop to MySql from hive

•Importing data using Sqoop from Mysql to hbase

•Using queries and sqoop