Julian Fellowes was to the manor born. It just wasn’t a “Downton Abbey”-size manor.

Fellowes, of course, is the creator, sole writer, and executive producer of “Downton Abbey,” arguably the greatest global television phenomenon of the 21st century. Here in the U.S., where it is concluding its fifth season, it is the most popular series in the history of public broadcasting.

It depicts the lives of the Crawleys — Lord and Lady Grantham and their extended family — as well as those of their sizable service staff. Thus far it has followed events from 1912 (in season one) to the mid-1920s (in season five, currently airing), with real-world events — the sinking of the Titanic, World War I — making themselves felt in the storyline.

Fellowes spoke to me over the phone from Los Angeles about how his background influenced the hit show. And, yes, he said he understands the “manor born” analogy I begin our conversation with. But, he said, “These things are rather hard to define. I think my background is a little more interesting than that. My father came from one of those families, landed gentry. My mother didn’t. Her father was a civil servant, so in that regard, it was quite uneven. My father’s family disapproved of my mother, something I was dimly aware of as a child. That is probably the root of my interest in class.”

Image by Getty Images To the Emmy Born: Fellowes claims to have dated a daughter of one of England’s great Jewish families.

His own upbringing was “comfortable,” he said. “My father had a house in the country and a flat in London, and I had a pony.”

Still, despite the pony and the posh upbringing, Fellowes said he always carried with him a sense of being an outsider even as he became what was called a “deb’s delight,” an attractive escort for debutantes.

“I was always relatively unimportant,” he said. “I wasn’t a great heir and I didn’t have a great name, and no one felt the need to laugh at my jokes. I got into the habit of being an observer.”

His fascination with aristocracy included his own family. Fellowes spent a great deal of time with a great-aunt. “She was married before the First World War, had a great house in Hampshire, and it was all gone after the war. Her husband died of wounds, and soon the money was gone and the house was gone and she ended up in a flat. We talked a lot about those times, when she thought a way of life that would last forever slipped through her fingers.”

It is exactly this dynamic at the center of “Downton,” wherein the servants realize that they have more and the lords and ladies realize they have to do with less.

In addition to the great social upheaval taking place at this time, the series tackles controversial issues such as rape, homosexuality and intermarriage. “Some people get heated about these subjects,” said Fellowes. “I’m glad they do. Upper-class anti-Catholicism. I saw the end of it.”

Fellowes, whose family is Catholic, added, “Even as recently as when I was a young man, these families didn’t mind you dating their daughters or shooting their daughter’s pheasants, but they certainly didn’t want you marrying them.

“English aristocracy anti-Semitism has always interested me. It’s quite mild and so ingrained they’re sometimes hardly aware of it. Someone will say something like, ‘You’re rather Jewish looking.’ They don’t know they’re separating a whole category of people.”

“One of the first girls I went out with was a Jewish girl,” Fellowes said. “It was sort of my first experience of not being desirable. She belonged to one of the great Jewish families. I won’t name them. They certainly didn’t want a Catholic in the family. People always think the prejudice is one-way, but it can be two ways. I wanted to use that in the show.”

Spoiler Alert: This season, Lady Rose (Lily James), a distant cousin of Lord Grantham, meets and marries Ephraim Atticus Aldridge (Matt Barber), the son of a leader of the British Jewish community. Her parents don’t approve.

This isn’t the first time a Jew has invaded Crawley territory. Lady Grantham (Elizabeth McGovern) was born Cora Levinson, the daughter of a Jewish retail millionaire. Though raised Episcopalian, she was not welcomed by the clan and was accepted only because she brought with her a much needed cash infusion.

“I find anti-Semitism weirdly illogical,” Fellowes said. “The English Jewish community supports the arts, it supports science, it supports education. It has great family values. Everything people approve of is rooted in this culture.”

Fellowes was born in Cairo in 1949, where his diplomat father was stationed as second secretary in the British embassy. Unlike many diplomats at that time and place, his father “considered the birth of the state of Israel something of immense importance.”

“The arrival of Israel in the Middle East was the best thing that could have happened,” Fellowes said. “It brought all of these great brains from world Jewish communities, and they created this extraordinary modern state. When you think of what the Israelis have done, and they had no oil, it’s amazing.”

Fellowes, a character actor on British television, won an Academy Award for his screenplay for “Gosford Park,” a country-house mystery set in the 1930s . Almost a decade later, he was at dinner with a production company executive who asked if Fellowes was interested in revisiting “Gosford Park,” but for TV. At first Fellowes demurred, but then decided to shift the setting from 1932 England to 1912, giving him “20 years to play with. I thought and thought about it and started imagining characters. Funnily enough, Cora was the first one I thought of.”

“Downton” has become an international hit; I asked why he thought it was so successful. “That’s hard to answer,” he said. “I don’t really know.” Then he suggests that it might have something to do with the way the show is formatted.

“America re-invented television,” he said, specifically mentioning Aaron Sorkin’s “West Wing.” “That show raised the possibility that episodes didn’t have to be self-contained, that story arcs can travel over multiple episodes.”

I suggest that some might find the show interesting because it harkened back to a time of rules and protocols.

“There is reassurance in that,” Fellowes said. “I think we live in an unstable time, not just obviously politically but in terms of our social behavior. People don’t know what the rules are anymore. In fact, [the time of the show] was a period of great change. Women’s political power changed. Organized labor grew. It was a period of almost daily change. Nevertheless, that society believed in the rules. Of course people still broke the rules, but they knew what they were.”

Fellowes has yet to start working on an American “Downton,” a series set in the Gilded Age. But that doesn’t mean he’s been lollygagging around. In his spare time, he’s written the book for the Broadway-bound musical, “The School of Rock.”

It seems like he’s doing a whole lot but he said he’s simply taking his father’s advice: “If you have the misfortune to be born into a generation which must earn its living, you might as well do something amusing.”

Curt Schleier writes frequently about the arts for the Forward.