Joe Morton is a veteran of the stage and screen who broke through with his silent leading role in John Sayles’s 1984 film Brother From Another Planet (which, he told Vulture, is currently in development as a television show). Most recently, Morton has been playing Eli, Olivia Pope’s father, on Scandal, with delicious Shakespearean flair since season three, when he burst into his daughter’s life, come hell or high water. He even won himself an Emmy in the process.

This season on Scandal, we’ve seen Eli Pope cornered for the first time. In past years, even when he was down, you got the sense that Eli still had the upper hand — so much so that he called President Fitz a “boy” in one of the most memorable monologues of the show. He was smarter, faster, and more ruthless than his enemies, and there would be hell to pay if anyone tried to bring him down. Vulture spoke to Morton about why Papa Pope resonates with audiences, the speech he gave to Olivia about being “twice as good,” and the time a network executive threatened to bury a TV show if the producers cast him as the lead.

I watched Brother From Another Planet a couple of years ago again at an Afrofuturism screening. It’s such an important film, and also a kind of weird one. What drew you to it?

Well, I suppose it’s weird. Two things drew me to the film: One, it was written by John Sayles. John is an extraordinary filmmaker, probably one of the original independent American filmmakers. And two, it was the subject of the film, about an escaped alien from another planet who was being chased — and clearly there was slavery on the other planet — by slave-catchers [and ends up in Harlem]. So that was another reason why I wanted to do the film. And in many ways, it was akin to some experiences I had as a younger person. My father was in the service. His job was to integrate the armed forces overseas. So that meant we showed up at military bases in Okinawa or Germany, racially unannounced. That made me, in that particular society if you will, the outsider. And in this case, the brother from another planet is the outsider. The brilliance of the film is that although the alien looks like he knows what’s going on, in terms of being in Harlem, he doesn’t. So it’s a way for the audience to look at Harlem, at some African-American experiences, via the eyes of a stranger. I thought that was all very brilliant.

It also feels particularly relevant right now, especially with regards to what’s happening with immigration and ICE raids.

Right. There is some talk of returning to Brother From Another Planet as a television series. And one of the things I actually did want to talk about was just that: If we do Brother From Another Planet, there are a group of aliens who really don’t have papers. So it could be a very interesting thing, as you say, in terms of what’s going on these days.

Would you be in it?

They haven’t quite figured it out. I think John would like me to be in it. I think the woman who’s producing it would like me to be in it. This is very, very early stages. Just the idea would be really interesting — dealing with an alien from outer space who now has to deal with immigration laws in America.

So far on Scandal, Eli has always been one step ahead of everyone else. But in this episode, we start to see him as the one who’s actually being run, as opposed to running other people. What’s it like to play the prey as opposed to the predator?

He says it’s not so much that he’s the predator, he’s just very smart prey, trying to keep his species alive. And I think that’s how he’s always looked at himself — as very smart prey. And you’re right. Now, there seems to be another entity that has power over him in a way that no one has been able to hold power over Rowan in the past. It will be very interesting to see how all this plays out.

Why do you think, as a somewhat villainous character, he resonates with so many people? Because I know I really love him.

I think many villains have the burden of not being very human. On some level, though, if you look at somebody like Hannibal Lecter, there’s something about that character that you like, even though he’s a madman. And I think the same thing is true here. Maybe that quality is just that relationship he has with his daughter, that that really is the crux of his being. That he has raised her, as he says in this episode, to be a warrior. That she even should come up against him, but at the same time, he loves her. He loves her unconditionally. So he will do anything he thinks is necessary to get her the things that are important to her. And I think it’s that quality that makes it easier to abide by all the other things that he does.

Something that really resonates with me about your character is his desire to be free. Do you feel like he can ever be free?

I think the greatest lesson that power has to teach us is: Once you’ve had it, once you are a part of it, you’re never free. If you’re someone who seeks that sort of thing all the time, you’ll never be free. So for Rowan, that’s always going to be the monkey on his back.

Are there parts of you that identify with Rowan?

I identify with the love for his daughter. I have two daughters and a son. And this is an odd kind of identification, I suppose — Rowan being a black man who has that kind of power that, in the real world, we understand doesn’t exist. But on some level, people like Malcolm X and Martin Luther King, they had that kind of power because they swayed people with their voices and their actions. And in some way, I think in that kind of strange triangle — between me, those two that I just mentioned, and Rowan — there is something I identify with. And I think maybe vis-à-vis my father.

Recently, I went back to Hofstra University, where I went to school, and part of what I talked about was Shakespeare. The first Shakespeare I ever saw was Othello. And it was Othello being played by Laurence Olivier. That was the first Shakespeare I had ever seen, and what was wonderful about it was watching a black man, who was so powerful, and powerful enough really to hold his own among this all-white community that surrounded him. And so, in that way, again, I identify with Rowan.

For me, one of the most memorable lines of TV happened in the season-three premiere when we first really meet Rowan, and he tells Olivia that he taught her to be “twice as good to get half of what they have.” What was the dynamic between you and Kerry Washington when you were filming that scene?

That was the very first thing we shot at the beginning of season three. Remember, season two ended with the reveal that, in fact, I was her father. Season three then opened up with a clear understanding of what their relationship is. Shooting that scene was great. It was wonderful. On one hand, it felt a little bit like an audition. I think Shonda [Rhimes] decided to play around with the idea that I am a Shakespearean actor and that I have done a lot of stage work — well, everybody in the cast has, actually. So what she was doing was something very unusual for television, which is to give a character a page-and-a-half monologue as a way to then reveal what the relationship between Olivia and Eli is. The thing about being twice as good — after that scene played, I can’t tell you the number of African-American publications that then wrote reams and reams of stuff about how many times they heard that same line from their father or mother.

I’m Korean-American, and my parents would tell me that, too. Do you feel like that was true for you, working in Hollywood and making your career as an actor?

Oh, it was absolutely true. A film like Brother From Another Planet definitely put me in people’s vision, if you will, but it didn’t necessarily make life any easier, because the problem — which in those days was a lack of material — still existed. The fact that Brother did very well, and I continued on in my career, was a number of different things happening all at once. One, was my own ability to prove to someone, No, this character can be played by a black man. Two, I had agents who believed in me who would push that kind of thing. And then, three, the world, slowly enough, began to change.

You’ve written about the lack of opportunities for black actors. Viola Davis also talked about it beautifully when she won her Emmy. Have you two commiserated about that?

No, unfortunately. I never get a chance to see her even though we shoot at the same studio. It is, and has been, our experience that in order to get the role, to get the Emmy, as she said in her speech, the opportunity has to be there first. The difference between Viola and Kerry is that Kerry sort of embodies the American, classic idea of beauty. Kerry is just gorgeous, with or without makeup. Viola just has a different look. I think Viola is a beautiful woman. But for the longest time, because she’s so much darker and because she’s so much more emotional in so many different ways when she works, I think that’s why she ended up playing mostly nurses and maids, cops, and the like. Because America just didn’t look at her as a beautiful woman.

Now, this has to do with what goes on still in film. The difficulty for African-American actors in film is that most of that star system is based on good-looking, young white folks. And it’s just hard; it’s harder to break into the film world, whereas television has now changed its parameters. Television has decided that it will actually try to, as much as possible, make the world that it creates look like what the world actually looks like. I think that’s one reason why TV these days seems to be doing so well. So, yeah, I think many of the problems still exist today. People like Viola Davis are breaking the mold. People even like Kerry Washington. Kerry Washington is the first black woman to be a lead in a dramatic series in over 45 years. So, as all of those molds get broken, it makes it easier for the younger actors who are coming up behind us.

Could you talk about the opportunities that you missed out on because of racism?

One story in particular: I won’t give you any names, but I came into Hollywood, it was pilot season. I got an audition for what is now a very popular procedural TV show. The producer of that show said, “Oh, this will be perfect for you,” and actually offered me the gig. When he went back to the network and told them that he wanted me to be the lead in this series, they told him that the only thing they would give him was to put that show in a graveyard spot so it would never succeed, if he put a black man in the lead. This was maybe 20 years ago.

On the other side of that same coin, Thomas Carter put me in the lead of two series. One was called Equal Justice, a law show. And the other was called Under One Roof, which was a generational family show. It was me and my family and my father and my sister living all in the same house, which was really lovely. What was sad about that series is that we did really well critically, but the network we were on at the time, CBS, didn’t really push the show. They took us off the air. I know that lots of church groups and black community groups wrote letters to try to get the show put back on, and it never happened. So that is two different ways of how racism affects this business. I think the second example of a network taking something off simply because they don’t know how to promote it, because it’s black, happens less now, because TV has more experience viewing that kind of thing.

The New York Post reported that you’re going to be in the first episode of the new show Inside the Black Box, where you would be discussing these experiences. How did you come to do that?

It’s funny. It came around because the woman who actually put the show together is a friend of my daughter’s. So that’s why I did the show to begin with, to help them out. What’s great about the show is it will feature black actors, black directors, black producers, who will talk to a group of students, both black and white — but probably mostly students of color — about the business, about the craft, and then possibly even do a “master class.” Which is kind of what I did: I talked about my experiences with the business, and then at one point, one of the students got up and did one of my monologues. Then I gave her notes and she came back and did it again. So there was a before and after in terms of the master class. So that’s the way the show will present itself.

What’s your favorite monologue that you’ve done from Scandal?

I think my favorite will always be the “boys” speech. But recently, I did a speech to Olivia: We’re sitting out on that bench, and she asks me to be a dad because she was going through a crisis of “what’s the point of all this” when she was clearly wearing the white hat, so to speak. And I tell her about the number of lives I am responsible for. Not necessarily the number of people I have killed, but the number of lives I’m responsible for. And that speech ultimately gets to a place where I’m telling her everybody is worth saving, even the monsters. And the point of it all is her. She is the one who has to drag everyone else into the light. And I think that’s one of my favorite speeches as well.