Just the fact that there’s a Zelda timeline where they kinda Minority Report Ganondorf and take him out for a crime he had yet to commit based on the accusations of two ten year olds seems to imply the dude didn’t have much incentive to try to play nice with Hyrule to start with.

I’m gonna say for the sake of the joke and the ongoing character of beaten down petty old man MGDMT Ganondorf this is before the meeting that Link eavesdrops on. He can go to Hyrule castle more than once. Zelda can hold a grudge.