created & maintained by @clarecorthell, founding partner of Luminant Data Science Consulting

The Open-Source Data Science Masters

The open-source curriculum for learning Data Science. Foundational in both theory and technologies, the OSDSM breaks down the core competencies necessary to making use of data.

The Internet is Your Oyster

With Coursera, ebooks, Stack Overflow, and GitHub -- all free and open -- how can you afford not to take advantage of an open source education?

The Motivation

We need more Data Scientists.

...by 2018 the United States will experience a shortage of 190,000 skilled data scientists, and 1.5 million managers and analysts capable of reaping actionable insights from the big data deluge.

-- McKinsey Report Highlights the Impending Data Scientist Shortage 23 July 2013

There are little to no Data Scientists with 5 years experience, because the job simply did not exist.

-- David Hardtke "How To Hire A Data Scientist" 13 Nov 2012

An Academic Shortfall

Classic academic conduits aren't providing Data Scientists -- this talent gap will be closed differently.

Academic credentials are important but not necessary for high-quality data science. The core aptitudes – curiosity, intellectual agility, statistical fluency, research stamina, scientific rigor, skeptical nature – that distinguish the best data scientists are widely distributed throughout the population. We’re likely to see more uncredentialed, inexperienced individuals try their hands at data science, bootstrapping their skills on the open-source ecosystem and using the diversity of modeling tools available. Just as data-science platforms and tools are proliferating through the magic of open source, big data’s data-scientist pool will as well. And there’s yet another trend that will alleviate any talent gap: the democratization of data science. While I agree wholeheartedly with Raden’s statement that “the crème-de-la-crème of data scientists will fill roles in academia, technology vendors, Wall Street, research and government,” I think he’s understating the extent to which autodidacts – the self-taught, uncredentialed, data-passionate people – will come to play a significant role in many organizations’ data science initiatives.

-- James Kobielus, Closing the Talent Gap 17 Jan 2013

The Open Source Data Science Curriculum

Start here.

Intro to Data Science / UW Videos

Topics: Python NLP on Twitter API, Distributed Computing Paradigm, MapReduce/Hadoop & Pig Script, SQL/NoSQL, Relational Algebra, Experiment design, Statistics, Graphs, Amazon EC2, Visualization.

Data Science / Harvard Videos & Course

Topics: Data wrangling, data management, exploratory data analysis to generate hypotheses and intuition, prediction based on statistical methods such as regression and classification, communication of results through visualization, stories, and summaries.

Data Science with Open Source Tools Book $27

Topics: Visualizing Data, Estimation, Models from Scaling Arguments, Arguments from Probability Models, What you Really Need to Know about Classical Statistics, Data Mining, Clustering, PCA, Map/Reduce, Predictive Analytics

Example Code in: R, Python, Sage, C, Gnu Scientific Library

A Note About Direction

This is an introduction geared toward those with at least a minimum understanding of programming, and (perhaps obviously) an interest in the components of Data Science (like statistics and distributed computing). Out of personal preference and need for focus, I geared the original curriculum toward Python tools and resources. R resources can be found here.

Math

★ What are some good resources for learning about numerical analysis? / Quora

Computing

Get your environment up and running with the Data Science Toolbox

OSDSM Specialization: Web Scraping & Crawling

Data Analysis

One of the "unteachable" skills of data science is an intuition for analysis. What constitutes valuable, achievable, and well-designed analysis is extremely dependent on context and ends at hand.

in Python Data Analysis in Python Tutorial Python for Data Analysis Book $24 An Example Data Science Process ipynb



Data Communication and Design

OSDSM Specialization: Data Journalism

Python (Learning)

Python (Libraries)

Installing Basic Packages Python, virtualenv, NumPy, SciPy, matplotlib and IPython & Using Python Scientifically

Command Line Install Script for Scientific Python Packages

More Libraries can be found in the "awesome machine learning" repo & in related specializations

Data Structures & Analysis Packages Flexible and powerful data analysis / manipulation library with labeled data structures objects, statistical functions, etc pandas & Tutorials Python for Data Analysis / Book

Machine Learning Packages scikit-learn - Tools for Data Mining & Analysis

Networks Packages networkx - Network Modeling & Viz

Statistical Packages PyMC - Bayesian Inference & Markov Chain Monte Carlo sampling toolkit Statsmodels - Python module that allows users to explore data, estimate statistical models, and perform statistical tests PyMVPA - Multivariate Pattern Analysis in Python

Natural Language Processing & Understanding NLTK - Natural Language Toolkit Gensim - Python library for topic modeling, document indexing and similarity retrieval with large corpora. Target audience is the natural language processing (NLP) and information retrieval (IR) community.

Data APIs twython - Python wrapper for the Twitter API

Visualization Packages matplotlib - well-integrated with analysis and data manipulation packages like numpy and pandas Seaborn - a high-level statistical visualization package built on top of matplotlib



Datasets are now here

R resources are now here

Data Science as a Profession

Doing Data Science: Straight Talk from the Frontline O'Reilly / Book $25

The Data Science Handbook: Advice and Insights from 25 Amazing Data Scientists Book $22

Capstone Project

Resources

Read

Watch

Learn

Metacademy - Search for a concept you want to learn

Coursera - Online university courses

Wolfram Alpha - The smart number and info cruncher

Khan Academy - High quality, free learning videos

Notation

Non-Open-Source books, courses, and resources are noted with $ .

Contribute

Please Contribute -- this is Open Source!

Follow me on Twitter @clarecorthell