I think he's being blackmailed by Marley I mean he was raised from a young age to believe that destroying the wall was the only way. He was probably taught that the people there were evil, as mentioned in that scene where he kidnaps Ymir and Eren. He said that if he knew that the people here were good he would have never destroyed the wall.