In his 1987 book, The Art of the Deal, Donald Trump recalled punching his second-grade music teacher in the face: “Because I didn’t think he knew anything about music.”

The alleged recipient of the punch, who taught Trump at the private Kew-Forest school in Queens, New York, would later insist it never happened.

Even so, it seems Trump is still an unpopular figure among the school’s staff.

“I don’t think any of us that work here are proud that he went here,” one employee told the Guardian on Monday morning.

The woman requested anonymity because “some of the parents here are gung-ho for Trump”. She said she did not plan to vote for Kew-Forest’s most famous alumnus.

“Unequivocally, absolutely not. Because of who he is. His stance on women, on immigrants, on vets, on the disabled, and just Trump as a person,” she said.

Trump’s father pulled him out of the Kew-Forest school when he was 13 years old after numerous misdemeanors. Trump was sent to the New York military academy, located 70 miles north in Cornwall-on-Hudson, in upstate New York.

For Trump it meant leaving behind both his school and the Trump family home, located in the upper-middle-class neighborhood of Jamaica Estates.

Donald Trump’s childhood home in the Jamaica Estates neighborhood of Queens. Photograph: Alamy

If that experience was jarring for the teenage Trump, a jarring experience for the septuagenarian Trump might be learning that he isn’t very popular in his old neighborhood.

“The shit he’s said – it’s just so disrespectful,” said Steven Morello, a 19-year-old college student. He lives around half a mile from the old Trump homestead.

“Like getting rid of all the Muslims. I know it’s not going to happen, he’s not going to get all Muslims kicked out. But still.”

Morello said that a lot of Muslims live in Jamaica Estates. The neighborhood was whiter when Trump was growing up here, but 45% of residents were foreign-born according to the 2000 census, with substantial Bangladeshi, Filipino and Haitian populations.

Trump’s childhood home was built by his real estate mogul father and is still standing, although it is no longer in the Trump family. The home, on Midland Parkway, is a two-story redbrick with four large white columns on front of the porch. It has 23 rooms and nine bathrooms, according to the New York Times.

No one answered the door on Monday, although a Range Rover was parked in the driveway.

Like many houses in the neighborhood, Trump’s former residence is vaulted on the street, with stairs winding up to the doorway. The imposing white columns are a typically extravagant detail adorning many houses in the neighborhood.

Jamaica Estates, where Donald Trump grew up. Photograph: Mapbox, OpenStreetMap

Down from the old Trump place there’s a house with a huge spire on one corner, and a stone folly on the lawn. Opposite, one home has a 20ft glass window looming over the street. One house had a courtyard. Many had lamps designed to look like old-fashioned gas lights.

There were few people walking around Midland Parkway. The only real signs of life were landscaping trucks. It seemed Monday was lawn maintenance day, with five separate crews blowing leaves off lawns.

One crew was working on the house next to Trump’s former home. A man named Miguel, who was unloading a lawn-mower from a trailer, was surprised to learn Trump had grown up in the area. He described Trump as “no good”.

“He doesn’t like Mexicans,” Miguel said. He said he was from Guatemala and won’t be voting as he doesn’t “have papers”.

There were no Trump signs on the businessman’s childhood street, although it seemed as if the neighborhood was doing its best to ignore the election altogether.

The Guardian only saw one Clinton sign, and not many people in the area were keen to talk about Trump. Even if they were aware that he was from the neighborhood.

“Oh, I know exactly where he grew up,” said a man called Jean, who asked his last name not be used. He said he lived in the neighborhood, although not on Midland Parkway itself.

“I’m not voting for either of them,” Jean, 61, said. “One of them is a liar. The other is crazy.”

Jean said he believed Trump to be the crazy one. He took issue with Trump’s stance on immigration.

“It’s a country that was based on immigration. How can you ban people coming in? It’s impossible to do it.”

Jean said he was originally from Haiti. The Guardian asked how he felt about Trump’s comments on Hispanic people.

“It hurts,” Jean said.

• This article was amended on 8 November 2016 to replace a photograph of Trump’s childhood home. An earlier version showed a house Trump lived in before the family moved to the one described in the article.