On Thursday, the attorney general, Christian Porter, released the religious discrimination bill for consultation, and I’m worried. It seems far-right commentators are banking on using people of colour as a wedge, pitting the right to practise our religions and cultures against our LGBTQ identities.

I do not want a repeat of the same-sex marriage “debate”.

On the day the plebiscite results were announced, I had two responses. I was standing backstage in Sydney, holding hands with a close friend, another ethnic queer. As the numbers were read out, I was elated – we did it! But then it sunk in: the places we live, the places we had grown up, had responded with “no”.

I remember the feeling of dread as I scrolled through social media and watched as the celebrations of “yes!” turned to “build a wall!”, “never going west of Parramatta ever again”, “western Sydney is a cesspit” – and while the white members of the queer community wouldn’t say it openly, we all thought we knew what they meant – those people of colour, they aren’t like us.

Conservative talking heads took to the media, salivating with delight. They had created the licence they wanted, to harp on about “homophobic Muslims” who don’t belong in Australia. Meanwhile, the majority of white queers were silent.

That’s the wedge, that’s how they divide us. And now they’re trying to do it again.

Right now the far right are using people of colour to wedge the door open on religious discrimination, and they are ramping up their damaging hysteria about the trans community too.

A glance at the headlines will tell you that people of colour and people of minority faiths in Australia are under attack. White supremacy is at play in this country’s immigration, law and order, defence, education, and employment laws.

Racism permeates every facet of our society, and people of colour, black people and Indigenous people bear the brunt of its weight. In this context, we need our white allies in the queer community to stand with us as we call for balanced religious discrimination protection. As far-right extremists target our places of worship to enact white supremacist terror, as women are harassed for choosing to wear religious clothing, and as we are continuously marginalised for our differences, we need protections that shield us from discrimination as we live our lives without harming others.

What we don’t need is a religious discrimination bill that can be used as a sword to target, vilify and harass others – fringe groups like the Australian Christian Lobby must not be given a platform to spread hate. The ACL wants to make this “debate” as reductive as possible. It wants to make it about Christians v queers in the frame of White Australia, where people of colour are used as political fodder.

Communities of colour are immensely diverse – we come from all over the world and we have a wealth of experiences that form our views. I grew up in outer western Sydney, an area of about 2 million people, where, because of decades of government neglect, infrastructure development lags behind population growth, where at your public school your desks are held together with chewing gum. It’s where so many black and brown families are, because it’s what they can afford, and it’s where our communities are.

We are not to blame for the isolation our communities face, shunned and very deliberately othered through the social and economic policies of White Australia. For many, the only place to find community and connection is at a religious institution, and maybe that institution is a conservative one.

When you’re part of a diaspora, sometimes the only way to build networks with people who share your experiences is through your suburban religious and cultural institution. A mosque, a synagogue, a temple or a gurdwara are not just places of worship. Annual holy and cultural events such as Diwali – the Hindu festival of lights – are celebrated at suburban temples around the country. Imagine being away from your friends and family during the biggest cultural festival of the year, yours is the only house on the street putting up festive lights during Diwali, and your neighbours ask you why you’re ready for Christmas in October. You go to the temple and there are hundreds of others who have come for the same feeling as you – community.

Sikhs have a tradition of giving out a three-course meal, soup kitchen style, to everyone who attends a service at the gurdwara. After a service you find a spot to sit and because there are so many people, you’ll end up sitting next to a stranger and end up making a new friend over a delicious, home-cooked meal.

For so many migrants and children of migrants, their religious institution is also a cultural one. It’s where they go to meet people who speak the same language, who love to cook the same food, who understand the significance of certain days on the calendar and with whom they can share understanding. That’s why we need a religious discrimination act that balances and protects ALL the parts of who we are – queer and religious.

Buying into the Christian right’s frame gives them the fight they want, and it’s one they win in the Liberal party room.

Uniting for a balanced act strengthens us, and it’s the first step in decolonising LGBT advocacy in Australia. Now is the time to show up and work with us to build the change we need.

• Neha Madhok is the national co-director at Democracy in Colour, a racial justice organisation led by and for people of colour. She worked on the yes campaign with Australian Marriage Equality

• Comments on this piece will be premoderated