Pete Buttigieg believes he probably won’t be the first gay president if he’s elected in 2020.

In an interview with Axios on HBO, the South Bend, Ind. mayor was asked how he’d respond to critics who call him too young, too liberal and too gay.

“We have had excellent presidents who have been young. We have had excellent presidents who have been liberal. I would imagine we’ve probably had excellent presidents who were gay — we just didn’t know which ones,” Buttigeig said.

He added: “I mean, statistically, it’s almost certain.”

Some of the most popular theories about whether there has ever been a gay president surround President James Buchanan, one-term president from 1857-1861. He is reported to have been close to another successful U.S. politician, William Rufus King. Others suggest President Abraham Lincoln might have been attracted to men.

But historians point out that the cultural differences between the 19th century and today are so vast when it comes to sexuality, that it’s all but impossible to say that the U.S. has had a gay president–by the modern definition.

This means that affectionate letters and other correspondence between men that might seem romantic to 21st century readers had very different connotations in the 19th century.

“Most historians of sexuality would tell you that the identity categories, the terms and the ideas that we use to organize things like sexual behavior, sexual identity, sexual desire, change over time,” Rachel Cleves, a professor of history at the University of Victoria in Canada, says. “It might be more accurate to say that there may have been presidents in the past who desired men, or had sexual experiences with men… they wouldn’t have necessarily identified as gay in a 21st century sense.”

Thomas Balcerski, a professor of history at Eastern Connecticut State University, has studied Buchanan and his relationship with Senator and Vice President William Rufus King for at least seven years. He’s critical of Buttigieg’s theory that the U.S. likely already had a gay president.

“I think it’s an early answer, if I may say, to the question of how a candidate like Pete Buttigieg can deflect what will be political heat towards sexual politics, identity politics,” Balcerski says.

For Buttigieg to say that another president may have been gay in the way that it’s understood in the 21st century, “it’s just farcical,” Balcerski says.

How likely is it that the U.S. has had a gay president?

Though theories exist about Buchanan and Lincoln, both Balcerski and Cleves make it clear that the modern way of thinking about sexuality can’t be applied to the era that 19th century presidents lived in. Even if the two had some sort of attraction to other men, they wouldn’t have identified as gay because that notion of living as an out gay man didn’t exist, both historians say.

The term homosexual, for instance, didn’t exist until 1868.

“If what [Buttigieg’s] remarks were that it’s likely that… someone in the past probably experienced same-sex desire, or same-sex attraction, or same-sex encounters, I would say yeah, I would agree with him,” Cleves says.

Obviously there were men who were attracted to other men or had sexual experiences with other men, she says.

“I think that there were always gray areas and nuances and plenty of possibilities for men in the late 19th and 20th century to have sex with other men and not consider themselves gay,” Cleves says.

But by the standard of modern gay identity, no president would fit into that category.

“[Buttigieg] lives in the 21st century where he can be married, he can be happy, contented, he can be proud of his identity,” Balcerski says. “That was not possible until, frankly, about the year he was born.”

Both Balcerski and Cleves say gay identity needs to be seen through the lens of the era. Depending on the time and place, affection for members of the same sex was normal and was seen as having nothing to do with sexuality.

“In some ways, the first half of the 19th century was much more open towards physical touch between men and expressions of love,” Cleves says. “Did James Buchanan or Abraham Lincoln identify as gay in the modern sense of the term? That would be very difficult. But did they have long standing relations with other men? In both cases I think the answer is yes, they had deeply loving and physically intimate—although I don’t know that they were sexual—relationships with other men.”

This type of intimacy didn’t carry through to the 20th century. The 19th century was an era before homosexuality was medically classified as a disorder or perversion.

“I think the 19th century, interestingly, can give us some hope,” Balcerski says. “I find that the early 19th century, ironically, almost presents an era where the same kinds of pathologies [against gay men] that will be present all the way through the pre-Stonewall era aren’t quite in place yet.”

Did James Buchanan and William Rufus King have a romantic relationship?

Buchanan was known as the bachelor president. He remains the only U.S. president to never marry. As a senator, he shared same quarters for 10 years with King, who is also famous for having served as vice president for 31 days before his death. However, during this period in Washington D.C., it was common for groups of senators and congressmen to live together.

Balcerski’s book Bosom Friends: The Intimate World of James Buchanan and William Rufus King, publishing in July, takes a deep look into the relationship between the two men, which Balcerski considers a friendship. Other historians like Jim Loewin, author of Lies my Teacher Told Me, believe letters written by Buchanan and King point to a romantic and sexual relationship.

Proponents of the theory point to a passage in a letter Buchanan wrote in 1844:

“I am now ‘solitary and alone,’ having no companion in the house with me. I have gone a wooing to several gentlemen, but have not succeeded with any one of them. I feel that it is not good for man to be alone; and should not be astonished to find myself married to some old maid who can nurse me when I am sick, provide good dinners for me when I am well, and not expect from me any very ardent or romantic affection.”

Balcerski says it’s important to chronicle gay history, but he says popular theories about the likes of Buchanan and Lincoln are too simplistic.

“As we become more and more interested in broadening the test, so to speak, of LGBT history, what we find is we’ve taken people, who for one reason or another, don’t fit into a normative pattern of heterosexual marriage and we basically put this label on them as gay,” Balcerski says.

Write to Jasmine Aguilera at jasmine.aguilera@time.com.