Big Data and Education



A Massive Online Open Textbook (MOOT)

2nd Edition

by Ryan Baker

in cooperation between Teachers College, Columbia University and the Columbia Center for New Media Teaching and Learning

Chapter 1: Prediction Modeling

Video 1: Introduction [YouTube] [pdf]

Video 2: Regressors [YouTube] [pdf]

Video 3: Classifiers part 1 [YouTube] [pdf]

Video 4: Classifiers part 2 [YouTube] [pdf]

Video 5: Case study in classification [YouTube] [pdf]

RapidMiner Walkthrough [pdf] [data set]

Assignment 1: Prediction Modeling [EdX] (requires registration and course enrollment)

Chapter 2: Model Goodness and Validation

Video 1: Detector confidence [YouTube] [pdf]

Video 2: Diagnostic metrics: kappa and accuracy [YouTube] [pdf]

Video 3: Diagnostic metrics: ROC and A' [YouTube] [pdf]

Video 4: Diagnostic metrics: Correlation and RMSE [YouTube] [pdf]

Video 5: Cross-validation and over-fitting [YouTube] [pdf]

Video 6: Other validity considerations [YouTube] [pdf]

Assignment 2: Diagnostic Metrics [EdX] (requires registration and course enrollment)

Chapter 3: Behavior Detection

Video 1: Ground truth [YouTube] [pdf]

Video 2: Data synchronization [YouTube] [pdf]

Video 3: Feature engineering [YouTube] [pdf]

Video 4: Automated feature generation and selection [YouTube] [pdf]

Video 5: Knowledge engineering and data mining [YouTube] [pdf]

Assignment 3: Data Aggregation [EdX] (requires registration and course enrollment)

Chapter 4: Knowledge Inference

Video 1: Knowledge Inference [YouTube] [pdf]

Video 2: Bayesian Knowledge Tracing [YouTube] [pdf]

Video 3: Performance Factors Analysis [YouTube] [pdf]

Video 4: Item Response Theory [YouTube] [pdf]

Video 5: Advanced Bayesian Knowledge Tracing [YouTube] [pdf]

Assignment 4: Bayesian Knowledge Tracing [EdX] (requires registration and course enrollment)

Chapter 5: Relationship Mining

Video 1: Correlation Mining [YouTube] [pdf]

Video 2: Causal Mining [YouTube] [pdf]

Video 3: Association Rule Mining [YouTube] [pdf]

Video 4: Sequential Pattern Mining [YouTube] [pdf]

Video 5: Network Analysis [YouTube] [pdf]

Assignment 5: Correlation Mining [EdX] (requires registration and course enrollment)

Chapter 6: Visualization

Video 1: Introduction to Educational Visualization and Learning Curves [YouTube] [pdf]

Video 2: Moment-by-Moment Learning Graphs [YouTube] [pdf]

Video 3: Scatter Plots, Heat Maps, and Parameter Space Maps [YouTube] [pdf]

Video 4: State Space Networks [YouTube] [pdf]

Video 5: Other Visualizations [YouTube] [pdf]

Assignment 6: Sequential Pattern Mining [EdX] (requires registration and course enrollment)

Chapter 7: Structure Discovery

Video 1: Clustering [YouTube] [pdf]

Video 2: Cluster Validation [YouTube] [pdf]

Video 3: Advanced Clustering Algorithms [YouTube] [pdf]

Video 4: Applications of Clustering in EDM [YouTube] [pdf]

Video 5: Factor Analysis [YouTube] [pdf]

Video 6: Knowledge Structure: Q-Matrixes [YouTube] [pdf]

Video 7: Knowledge Structures: Other Approaches [YouTube] [pdf]

Assignment 7: Clustering [EdX] (requires registration and course enrollment)

Chapter 8: Advanced Topics

Video 1: Discovery with Models [YouTube] [pptx]

Video 2: Discovery with Models Case Study [YouTube] [pptx]

Video 3: Text Mining [YouTube] [pptx]

Video 4: Hidden Markov Models [YouTube] [pptx]

Video 5: Conclusions and Future Directions [YouTube] [pptx]

Assignment 8: Social Network Analysis [EdX] (requires registration and course enrollment)

Acknowledgements: Sincerest thanks to Elle Wang, Michael Cennamo, Stephanie Ogden, Luc Paquette, Jose Diaz, Michael de Leon, Therese Condit, students who have recommended additions or corrections, and others.

These materials were created with generous support from the National Science Foundation (#DRL-1418378), and the Provost and President of Teachers College, Columbia University. The content represents the views of the author, and does not necessarily represent the views of the National Science Foundation.

Bugs? Errors? Email Ryan Baker.

Please cite this MOOT as Baker, R.S. (2015) Big Data and Education. 2nd Edition. New York, NY: Teachers College, Columbia University.

All materials here copyright Teachers College, Columbia University, and Columbia University, 2013-2015.



This work is licensed under a Creative Commons Attribution 4.0 International License.