Thanks Joe. I think you're right.



There are a few different dimensions to the issue which are easy to get mixed up. One is the size of the network, another is the amount of computation it takes to run the network in the car, and another is what resources (data/time/computation) is required to train the network. All of them grow, but to varying degrees.



The thing I find most impressive about the network I wrote about is the scale of resources needed to create it - that's the part of my post which strays into hyperbole. Second is the audaciousness of the architecture - it's ground breaking. The runtime resource requirement is also impressive, but it's the least shocking of the three. We know HW3 is coming and that it's a lot more powerful. This has to be because Tesla intends to run more resource intensive networks, so it's no surprise that the networks are getting more resource intensive.



Elon has specifically called out that the central improvement for HW3 is a custom NN processor, so it's certainly the NN processing which principally lacking in HW2/2.5. And there's this - running the same realtime control NN 10x faster doesn't get you much of an improvement in performance. If you need 10x faster hardware it's because you want to run a categorically more powerful network. That network might well look like the one I wrote about.



But there's probably a more basic misunderstanding here too - I think Elon is probably talking about a different network than the one I was talking about (as detailed in my previous comment above).

Click to expand...