Q: What kind of testing are you doing to be confident at each stage that this system is safe or is functioning enough to be confident to put it in there, because it’s difficult to come up with I mean maybe it’s not difficult, what sort of testing you do to ensure that these systems are working well?

Elon Musk: Yeah it’s actually not all that difficult, we thought of obviously testing with Tesla you know the Tesla QA team. We are testing initially on a closed track with our test engineers, they all broaden it to a very limited set of a kind of alpha users including me so I usually have the latest software update about it day after our QA team has it.

And I am personally testing each element of the car then it goes to a local early access program which about a thousand customers/distributors around the world who are technically savvy and don’t mind you know or want to use the early software cognitive of potential issues you know so many different environments throughout the world, it’s just not possible for QA team to cover them all.

If that’s looking good then we will roll it out initially in shadow mode through the whole fleet by Shadow Mode actually means that the car is not actually not taking any action but it’s registering when it would take an action and when it would not take an action and when you compare that to cases where let’s say some body had and accident but you look at the vehicle logs and say well if the car had been in autonomous mode that accident would’ve been avoided, okay obviously is a plus and you can also say okay the car would have send something that would have resulted in an accident in that case it’s in the shadow mode and then that’s an issue that needs to be corrected.

And then you get into statistics and use the calls for false positives and false negatives, you gather enough of those over time until there is clearly a statistically significant sample set and at the point which is unequivocal that turning on an autonomy feature would improve safety, that is the point at which we allow to act to actually take action before that we do not the computer to take action.