Udacity Self Driving Car Nanodegree in 2020

My experience and review of the program

After many months of watching video lessons, completing quizzes and coding up projects, I have finally graduated from the Udacity Self Driving Car Nanodegree. It’s been quite the ride and I’ve learned a great amount about many of the core components that make an autonomous vehicle work.

The program is quite the investment of both time and money, so I wanted to share my experience with the program with anyone that is considering enrolling in it or wanting to learn more about it.

The Udacity Self-Driving Car Nanodegree has changed a lot over the years. It was originally launched in late 2016 and comprised 3 Terms, each taking about 3 months to complete. Each term would take a deep dive into related topics that are crucial components to build a self driving car.

Term 1: Machine Learning, Deep Learning, Computer Vision

Term 2: Sensor Fusion, Localization, Control

Term 3: Path Planning, Advanced Deep Learning, Functional Safety, System Integration

I was first enrolled in this program in February of 2017 and started going through Term 1. It was my first exposure to many Machine Learning and Deep Learning concepts and the content I was learning didn’t feel too far delayed from the cutting edge research that was happening in the field at the time. I actually found a job soon after finishing just the first term, applying many of the things I had just learned. Because my goal at the time was to leverage this Nanodegree to get a job, I cancelled my subscription soon after without finishing the last two terms.

Three years later I found myself in the middle of another job search and I decided to revisit the program. After enrolling I found that the core curriculum had changed extensively. The program is now only 2 three-month terms and twice the price. This last fact should be taken with a healthy serving of salt since Udacity often has sales where they offer 50% off or first month is free type deals. If you are only going to take away one thing from this post it should be this:

WAIT FOR A DEAL BEFORE ENROLLING IN ANY

UDACITY NANODEGREE PROGRAM

The discounted deals happen often, so be patient and try to get a good one. I managed to enroll during a 50% off period and ended up paying around the same as original price 3 years ago.

Overview of the Nanodegree

The current program is a restructured and streamlined version of the original. It has largely the same content but a significant part of it is now labelled as “Additional Content” which you are not required to complete in order to graduate from the program. The new program also includes a Mentor that is assigned to you and will help you with any questions or problems you face during the program. Mentors also schedule weekly check in meetings where you can talk 1-on-1 via a video chat about your progress and they are also available to answer questions over a chat during other times.

The current term structure look like this:

Term 1: Computer Vision, Deep Learning, and Sensor Fusion

Term 2: Localization, Path Planning, Control and System Integration

Advance Lane Finding Project

Term 1 is a really great introduction to Computer Vision concepts. It starts with simple concepts like edge detection and quickly gets you up and running applying those concepts and building projects with them. The first two projects consist of finding lane lines on a dash-cam type video of a car driving down the highway.

It’s really inspiring to see the output of your code correctly overlay the lines on the highway and you really get a sense of how it really is possible to build a self driving car. The first term continues with an introduction to deep neural networks starting with the very basics and then asking you to implement one of the first commercially successful deep neural network architectures (LeNet) but re-purposed to detect traffic signs.

Traffic Sign Classification Using LeNet

The last section of the first term breaks away from the computer vision side of things to talk about Sensor Fusion. Here you get a sense of how multiple sensors are used to make sense of the environment. This last section was a little underwhelming for me since you don’t get to work with real LIDAR or RADAR data which is instead “simulated” with points on a Cartesian plane. Most of this section revolves around Kalman Filters, which are admittedly a hard concept to wrap your head around (at least for me). Around this point is also when the program switches from Python to C++. This is not a small change and a few of the projects will require you to write extensive C++ code. While the programming language switch in the middle of the course might seem arbitrary at first, I imagine that this is done to familiarize students with C++ since it is extensively used for robotics and self-driving car applications. They do offer a free C++ course as well but completing this would likely add at least another week or two to your timeline.

Term 2 starts with an introduction into Localization, before jumping into Particle Filtering and a project where you implement a particle filter to localize a vehicle given some landmarks. The start of this term suffers from the same ailments as the end of the previous term. The concepts being discussed are really powerful and fascinating tools but the projects themselves are underwhelming.

Junior, the Stanford Self Driving Car, performing path planning. Video from Udacity: https://youtu.be/M7ZJ74RVHqo

The term continues by briefly converting a variety of interesting topics like search algorithms and trajectory generation. These lessons feature a lot of content from the Stanford Self Driving Cars, Stanley and Junior, which competed in the 2005 and 2007 DARPA Grand Challenge. Tying the concepts that you’ve just learned with videos and demos of a real self-driving car performing very well while using the same algorithms that you implemented is a very inspiring thing.

The next project is perhaps the most open-ended one because there is no “correct” way of approaching it. You are tasked with creating a path planning algorithm that a car can drive on a highway with traffic! This was my favorite project of the whole Nanodegree since it was rather open ended and you weren’t just following instructions on how to implement an algorithm but rather you could come up with your own.

Path Planning project

For this one I struggled with a rules based approach, trying to get the car to switch lanes, slow down and speed up at the right times. But after crashing into one too many cars I decided to revisit an earlier lesson on Cost Functions. In the end I substituted all of my rules based code for a single function that determined when the right time to switch lanes was. Seeing the car weave around traffic following the cost function I designed was a lot of fun.

The last stretch of the program goes into motion and PID controllers before introducing you to the last topic, System Integration. Here’s where it all comes together… sort of. The final project consist of using ROS (Robot Operating System) to interface different modules that are necessary to make the car drive itself. Much like in the previous project the car is tasked with following way-points around the track but this time it has the added challenge of having to detect traffic lights and classify their state correctly. This project is also fairly open ended when it comes to how to go about the traffic light detection and classification but it’s suggested that you use TensorFlow. The Nanodegree gives you some exposure to TensorFlow but it doesn’t walk you through the entire process of deploying a model, which makes this project a lot more challenging.

System Integration Project

This project also helps you get working knowledge of ROS and the lessons leading up to the project really give you a good grasp on it. An option given to students given that make it this far is to test their code on a real car! Sadly this option was not available to me since real car testing has been suspended for the time being due to COVID-19, but I hope to get the chance to do so in the future.

Conclusion

This newer version of the program is a welcomed change. A shortened core curriculum and the addition of a Mentor makes the program much more manageable than when it first launched. The shortened curriculum is not much of a downside since much of the old content is still made available to you. However, not all the changes are good, or rather the lack of them. Some of the content is now 3 years old and is outdated. The core concepts that you are learning are still very much relevant and in demand but the program no longer makes you feel that what you are learning is at the “cutting edge”. Perhaps where this is the most visible is in some of the projects with very outdated dependencies like Python2.7, Tensorflow 1.3 or ROS-Knitetic, none of which are supported by their original developers. This is really not a deal breaker since all of the software is still very much available and Udacity offers a Virtual Workspace environment with all the software already installed. The Virtual Workspace makes it so that all you need to complete the program is a computer with an internet connection. However, my personal preference is to do all the coding on my local machine with no latency issues, but with the added headache of setting everything up yourself.

Overall, completing the Udacity Self Driving Car Nanodegree has been a great learning experience. One of the best online learning experiences I’ve had. The program is very expensive and time consuming but totally worth the money if you find a good discount. This is advertised as an advance program and it really is, I would not recommend it if you are not already very proficient at both Python and C++. But if you are a Software Engineer looking to learn more about autonomous vehicles and self-driving cars, I can’t think of a better starting point.

If you made this far, THANKS!

Hopefully this write-up was helpful in determining whether this program is right for you. Please feel free to reach out to me on LinkedIn or Twitter. I’m happy to talk more about this program or self-driving cars in general.

If you want to see even more stories from me, check out my attempt at creating a DIY Lane Keeping system.