If discretion is the better part of valor, Outlander‘s David Berry deserves some kind of medal for our recent chat.

The Australian actor portrays the Starz historical drama’s Lord John Grey, an earnest British soldier and closeted gay man who — like many characters in the drama — quietly yearns for James Alexander Malcolm MacKenzie Fraser, played by Sam Heughan. Warning: Book (and potential Season 4) spoilers ahead. If you don’t want to know, skip down to the Q&A below.

Because Season 4 of the TV adaptation will pull heavily from Drums of Autumn, the fourth novel in Diana Gabaldon’s Outlander series, it’s likely that we’ll see Lord John (and William!) arrive in the Thirteen Colonies before season’s end — and, thanks to circumstances beyond their control, they’ll stay longer than he originally intends. If the series closely follows the novel, we also know that Lord John also will play a rather important part in another Fraser’s storyline as the episodes continue.

But Berry can’t talk about much of that right now. Production secrets and such. So while we had him on the phone ahead of the show’s Season 3 Blu-ray, DVD and digital release Tuesday, we settled for the next best thing: grilling him about gazing at Heughan, as well as getting his take on a few other Season 3 moments. Thankfully, he was a good sport.

TVLINE | I have heard that the time between your Outlander audition and shooting your first scene as Lord John was incredibly quick. Has Season 4 so far seemed like a leisurely cakewalk in comparison?

I don’t know how much I’m allowed to say on Season 4, but I can say that anything that comes after my entrance to Season 3, yeah, that has definitely been… [Laughs] I don’t think you can get much harder than that. So, yeah, absolutely.

It wasn’t easy, but I can look at it with a sense of amusement now, and I think it probably helped the character in some ways. Because Lord John is always getting himself into situations he’s not always well equipped for, and just kind of treading water and trying to keep his head above water and then finding some kind of clever solution out of it. I think I found some kind of clever solution out of it. At least I managed to keep my head above water.

TVLINE | I read an interview where you said you’ve read a lot of Diana Gabaldon’s Lord John novels and novellas. Have they given you any insights into the character that helped you portray him in Season 3?

Yeah. When I first stepped into the role, I hadn’t read anything, and then I went ahead and read as much as I could. And I don’t know, what did I really glean from it? You know so much is already there on the page. A lot of it is lifted from the books. When you read around it you get a better sense of the stuff that isn’t written there. One thing I guess: It informed my performance in the Governor’s ball sequences. [An extended] interaction between John and Claire that didn’t make it into the episode, that I still wanted to play a sense of that relationship or infuse it with a bit of that. That tonality that there’s tension between John and Claire, it isn’t really explicitly mentioned in the scripts per se, but it’s sort of in the unspoken dialogue between the three characters, John, Jamie, and Claire, and that history in the book helped with that performance.

TVLINE | Lord John is a gay man at a time when that is something he can’t be publicly, so it feels like there is so much in what he doesn’t say, the way he looks at people — Jamie, specifically. Have you and Sam had any conversations about your characters might communicate without words? Were you given any direction about that kind of thing?

No. I don’t think you need much direction to look at Sam longingly. [Laughs] He’s a beautiful man. I can say that as a straight man… I imagine John with a sense of desire, but also in a sense of longing for love, and I think that’s what really connects people. When you see it it’s a longing for love and that’s someone you can universally empathize with, and that’s what I like about the character, so that doesn’t really take a lot of direction it just takes a lot of trying to get inside the thoughts and things of a character like John. I think we can all relate to that.

TVLINE | I loved the scene toward the end of the season where, in one fell swoop, John nullifies Jamie’s arrest, dresses down the naval officer and forwards the plot. It also gave us a chance to see him a little bit more commanding than we had seen him previously. I know you can’t speak very freely about Season 4, but are there opportunities coming up where we’ll see that side of him again?

Well, that was an interesting scene, because I actually only had really a night to learn it, and I was thinking, ‘Oh, my gosh, I’m never going to be able remember all these lines,’ which is frequently the case in Outlander. [Laughs] So I learnt it overnight and just gave it my best shot and I was pleased to see it came up so well. That was probably a nightmare for the editor to put together. I don’t think I ever did a full take where I said all the lines out. But what was great about it is that it just gave a sense that John is his own person and has his own agency and does command a little power. He’s not just a guy who is longing for unrequited love. In his own right he could probably have the pick of the bunch if he wanted, being the powerful guy he is.

It’s through his heart that he chooses to protect Jamie and to be that guy and to use his power, never abuse it — not like that at all — to help others, and that’s a bit of his conflict as well. I think that Capt. Leonard really had it coming. I think the audience really wanted to see that character get beaten down, but everything kind of lined up on my side to make it work, except for learning the lines. [Laughs]

TVLINE | In Book 4, Drums of Autumn, John comes to America. Again, I’m not going to ask you to talk about stuff you can’t speak about, but it has always struck me that John and Jamie live in a time when it is not easy to have a friendship that spans years and continents, and yet they manage to continue that bond and have it grow closer as the years pass. What do you think it is about them that allows for this to happen?

Well, they have a shared history. I think when a good friendship has some kind of shared trauma or shared experience that really binds two people closely together, and in this case being Ardsmuir. And then of course, Helwater, where Jamie entrusts John with his son, William, and that, of course, is one of the most compelling reasons to stay in touch with any friends… They also have a simpatico of a shared intellect and honor and just the way that they believe that in the moral forces of this world, whatever they are, however hard they are to see in this very brutal, 18th-century world. I think they share those things.

…I try to keep in touch with Sam as much as I can. I think it’s fair to say that Sam and I have a pretty good friendship, and I hope if you interviewed him he would say the same. It’s interesting being away from the Outlander set — myself as David. I see some kind of correlation between the character and myself. I can envision in my mind, Sam and Cat[riona Balfe] are over there in Scotland doing their thing, and David’s over here in Australia… We share a relationship where we’re tethered by this show and occasionally we’ll keep in contact with each other. That, in some strange way, informs the performance that in turn gives when we arrive and see each other, because just like John, you know, he’s kind of in and out of their lives, and that plays really nicely with the way things are set up at the moment.