The office should be a place where employees can focus on getting work done, but a good share of lesbian, gay, bisexual and transgender workers say they're getting harassed on the job.

About four out of 10 LGBT workers report feeling bullied at work, or 11 percentage points higher than the national average of all workers combined, according to a new study from CareerBuilder. The survey was released on GLAAD's Spirit Day, which seeks to raise awareness of the bullying LGBTQ people face.

The majority of those who said they were bullied said it was by one person, while about 13 percent said it happened in a group setting. The number of LGBTQ workers with concerns about workplace bullying has grown since the 2016 election, said David Kilmnick, CEO of The LGBT Network, a nonprofit that serves the gay and transgender community on New York's Long Island and the borough of Queens.

Get Breaking News Delivered to Your Inbox

President Donald Trump earlier this year announced a ban on transgender service members in the military, while the White House's LGBT rights page disappeared after he took office in January.

"It has such a negative impact, not only to the LGBT person, which is a given. If you're being bullied for who you are, it will impact how one feels safe in one's workplace. But there are so many other people, non-LGBT folks, that don't want to hear this," Kilmnick said.

His nonprofit has received about 150 calls this year regarding workplace bullying, or triple what it normally receives, he added. The callers report being openly mocked for their looks -- such as being criticized for not being feminine enough for people who identify as a woman -- or being attacked over equal rights.

"Comments are made that their marriage isn't as real or worthy as others," Kilmnick said.

About four out of 10 of those surveyed in the CareerBuilder poll said they were picked on for their personal attributes, such as their appearance or gender, while 47 percent said they were gossiped about. LGBT workers also said they were ignored, falsely accused of mistakes or judged on different standards than other workers.

The survey was conducted earlier this year by Harris Poll, which surveyed 3,420 employees, including 238 LGBT workers.

The impact on workers can be long-lasting, CareerBuilder said. About one out of five LGBT workers who said they were bullied at work had health-related problems because of it, while 15 percent said they had called in sick because of the bullying.

Workers who are experiencing bullying should take notes to document their interactions with the bully and keep them in a safe place, CareerBuilder said. Sometimes standing up to the bully can work, with about 20 percent of bullied workers saying the behavior stopped after they confronted their aggressor.

Taking the issue to HR might not always solve the problem because not all employers are prepared to handle workplace bullying, Kilmnick said. He added that his group is stepping up workplace training to help businesses build a more inclusive workplace.

"In a way, what we are dealing with now feels like what we dealt with 20 years ago," he said. "It's not a good thing, and we should be beyond this. But when we look at what we've overcome as a community, we have not only survived through those times, we've thrived."