Story highlights Residents in Atlanta's affluent Buckhead community are worried

They have organized a protest in front of a mansion that's for sale

They say Bieber's "eclectic" lifestyle is not right for their neighborhood

Bieber's camp has not commented on any rumored move

Harold White jumped into action when he heard a rumor that Justin Bieber might be eyeing a mansion in Atlanta and could become his neighbor in the affluent Buckhead neighborhood.

The retiree and longtime resident is worried about unwanted attention should the singer indeed move to Buckhead, known for its vast old-money mansions. White has organized a protest for 6 a.m. Monday in front of an on-the-market mansion that is part of the Bieber speculation.

No one in Bieber's camp has commented on the rumored move, which began with a TMZ report . Still, the very idea of Bieber deciding to make the Georgia capital another home has proved enough to get residents such as White up in arms.

"We're concerned he'll bring the wrong type of element into a quiet, residential area," he said. "It is our position that a person with his means could certainly find a neighborhood more suited to his "eclectic" lifestyle.

White's wife started a Facebook page to organize the protest.

It warns that "Justin Bieber's relocation to Atlanta can be nothing but bad for our children, as well as the community. Some can't even let their children play in the driveway without fear; he has raced vehicles under the influence, before. What's to say he won't do it again?"

Bieber has had several encounters with law enforcement authorities in recent months. He was recently arrested in Toronto, accused of assaulting his limousine driver in December.

He also faces charges of drunken driving , resisting arrest and driving on an expired license after being stopped by police in Miami Beach, Florida, last month.