KALAMAZOO, Mich. -- Family, friends and colleagues of a Kalamazoo, Michigan, doctor are heartbroken four days after Immigration and Customs Enforcement (ICE) agents arrested him. They say they're working to ensure he can remain in the U.S., CBS affiliate WWMT-TV reports.

Lukasz Niec's family said they are unsure of how long he will be detained. The 43-year-old immigrated to the U.S. from Poland when he was 3 years old, and his wife said he holds a permanent green card. It's unclear when Niec will face a judge.

"We need him here, and we're lonely without him," Rachelle Burkhart-Niec told WWMT. She said her husband was taken from her family when three ICE agents arrested Niec at their home Tuesday.

Get Breaking News Delivered to Your Inbox

"I received a phone call from Lukasz that he had been detained," she said. "I kept saying over and over to stop pranking me."

Lukasz Niec WWMT-TV

Those who know Niec say he's a loving father and a well-known physician of internal medicine at Bronson Methodist Hospital in Kalamazoo. Now they're worried he could be deported back to Poland.

"He cannot be deported back to Poland, a country he doesn't know, a country he has no family at," his sister Iwona Niec-Villaire told the station.

She believes Niec's arrest stems from a 1992 misdemeanor arrest for property damage when he was 17, but his family didn't know it could be used to deport him from the U.S.

"Now, they're using this expunged case that's stamped 'non-public record' against him," Niec-Villaire said. "This is a man who's needed in the community and not detained in Calhoun County Jail."

Several of his colleagues wrote letters to an immigration judge in support of Niec.

"He's a leading physician among the team," Dr. Hussein Akl said. "He helps and supports. He's just one of the better people you can know and connect with."

"It seems like every night we're seeing innocent people being ripped away from their family for some vague immigration laws and a confusing immigration system that hasn't been fixed," Dr. Michael Raphelson said.

A spokesman for Detroit's ICE field office did not respond to WWMT-TV's request for comment.