Patrick Dempsey has no regrets about leaving Grey’s Anatomy. If anything, the actor wishes he had checked out sooner.

“It had been long enough,” McDreamy’s onetime portrayer tells People.com. “It was time for me to move on with other things and other interests. I probably should have moved on a couple of years earlier. I stayed a bit longer than I should have.”

As Dempsey muses, “I think after a certain period of time, no matter how much money you make, you want control out of your own schedule.”

And when the time did come for him to walk away from the ABC drama in April 2015 (at the end of Season 11), Dempsey says series creator Shonda Rhimes did not block the exit. “It was clear we were both ready,” he offers.

Regarding Rhimes’ decision to kill Derek off by way of a spectacular car crash, Dempsey — whose first major post-Grey’s project, Bridget Jones’s Baby, opens Sept. 16 — says, “I thought it was really interesting. It definitely woke people up. And it was beautifully symbolic in many ways.”