$\begingroup$

No one here has addressed the practical issues of why someone studying TCS should learn programming.

If you are planning to do a PhD in TCS in a Computer Science department, there is a good chance you will need to take some non-Theory courses, and those will almost certainly be very programming-intensive. Depending on the program you are in, you may also need knowledge of non-Theory subjects to pass your qualifying exams.

When you finish your PhD, most job opportunities for TCS are in academia. If you work in academia, you will be expected to teach, and you might be expected to teach an intro-level undergrad CS class that will be more programming than theory. Even if you are teaching a theory class to undergrads, like say Algorithms, you can expect that your students will know more about programming than theory, and without knowing what your students know, it will be difficult for you to bridge the gaps in their understanding. I shudder at the thought of CS undergrads being taught by someone who does not know programming!

If you don't care about these practical concerns, then you can probably get by doing research without really knowing anything about programming. Certainly you have plenty of company in the TCS community, but mileage will vary depending on what exact area of Theory you're working in. For instance, if you're doing pure computational complexity theory, proving lower bounds on classes that no one has ever heard of, then it's likely that programming will be of no use to you. But if you're doing something more algorithmic, then I feel that being able to write good clean working code will strengthen your intuition if nothing else.

I recommend learning C (not C++). Pick-up a copy of K&R and read it front-to-back. C doesn't have many of the fancy features of modern languages, but it does have simple but elegant syntax and semantics, which you should be able to learn in entirety. However, even when you understand the language in entirety, it still takes practice to master writing good elegant bug-free code in C. Nevertheless, if you can master coding in C, you will be able to master any programming language you encounter. Furthermore, that discipline will help you think how the hardware thinks, which will be beneficial when designing algorithms.

Ideas like pointers are very important to anyone who does algorithm design, but unfortunately, languages like Java and Python obscure them from you, so that's why I don't recommend them as a first language to someone with a math background. OOP is more important for people who have to maintain huge software projects, not someone who is designing algorithms.