The actor talks about the expectations of being the eldest of his brothers, how he and his sisters were turned into the household staff and the delights of working less to be at home more

My paternal grandfather was a bit of a pirate, a bit of a tough guy. You could call him intriguing – though that is a generous way to describe him. He was a lawyer who was disbarred on a technicality. After that, he wound up having a very unhappy life. My grandmother was like a saint. She was a great cook, a nurse, and she worked all hours. She was thoughtful, loving and kind. She raised her three sons on her earnings, without a husband around much of the time.

Regardless of what my grandfather’s demons were, he eventually worked all that out. Long story short, when my dad got married and had a big family, six kids, my grandparents were available to us in a “grandparently” way. The most memorable times of my childhood were going to their house. My grandfather died when I was 10, which seemed very old to me, but he wasn’t that old … My God, he was only nine years older than I am now, but he was a heavy smoker and a heavy, daily drinker.

My father was very close to his mother. He was obsessed with her, but did not like his father: they had a nodding, begrudging relationship. They were two very different people. But in the end my dad made peace with him.

My father lost years of his life providing for his family. He was salt of the earth: dependable, a high-school teacher; he coached football. He never did anything for himself. When I was a kid, my dad and I had a lot of time together. He was a tough disciplinarian, but he enjoyed having four sons who were all pretty athletic. But when his parents died, especially his mother, he changed – he was not the same again. And there was a real delineation after his mother died. He was 40 or 41 and he became very mute, very depressed. As a teacher, he knew what the kids were into and I think he was always worried about what trouble we would get into. When you have four sons, the surface area is increased quite a bit.

My mom, Carol, was very stressed a lot of the time – six kids and no money. The house was a mess all the time, because she just didn’t have the energy to clean up. When I was young, I did not have a good relationship with my mother because to make her life easier she turned my sisters and me into the household staff.

I was my father’s oldest son and my father was the oldest son, and more was expected of me. It was less to do with me being some exceptional person than just my birth position. I was the only one of my siblings who had a full private college education. I don’t think any of them thought: “Oh God, he’s going to spend all ‘my’ college money”; they were OK with it.

Some films didn’t have the budget for my family to travel with me so they were fast nos

I had just turned 25 when my dad died of cancer. He was 55. I had started working in this business [acting], but of course, you skip 10 years and my life became completely unrecognisable. What I care about is not so much that my dad didn’t get to see my career, as much as what the results of my career might have been for him. I was sad that I hadn’t been able to pay him back and thank him for all he had done for us. My father had a great dream about travelling, he talked about it a lot. But he couldn’t because he didn’t have any money or the time, with six kids.

Work is what I do and I enjoy it, but it is not everything to me. So when I met my wife, Hilaria, and we had our kids, I said, “I’m gonna go all in on my family, our relationship and our home” … and it was easy. Nowadays, my work is completely tailored around my family and around me being at home. I have given myself a tremendous gift by doing less work. I’ve had a good number of opportunities to do films that would involve travel and some of them didn’t have the budget for my family to travel with me so they were fast nos. I’ve got a menu of different jobs … with some Trump every now and then! Everyone’s loving it [his appearances on Saturday Night Live as Trump] … my brother Stephen, not so much. He’s a big rightwing Republican.

We had three children in three years, which has its benefits and its drawbacks – the drawback is that we are exhausted all the time. We’re on a constant rotation of sleepless little children, but I’m the luckiest person I know because my kids are healthy – I am very grateful for my career and I am not dismissing it, but the kids are the most important thing. Family is all there is.

• DreamWorks Animation’s The Boss Baby is available on Blu-ray and DVD from 31 July, from Twentieth Century Fox Home Entertainment