(CNN) A flag bearing a swastika flying outside a house in a small Colorado community was taken down after dozens of residents showed up outside the home in protest.

When a neighbor displayed a Nazi flag and a Confederate flag outside his home on Election Day, November 6, residents of Fruita, Colorado, organized a peaceful protest calling for the homeowner to take down the flags, CNN affiliate KKCO reported.

Protest organizer Joe Gibson said he hoped people expressed their anger and pain by protesting against the flags being put up.

"Really, I just want a future where my kids can grow up without Nazis and hate on our streets," Gibson told KKCO.

But not everyone felt the same. The protest was met with another protest, with residents vouching for the family's right to fly the flag.

Read More