Gender-neutral lavatories introduced by "woke" local authorities and public bodies make women feel uncomfortable and should be replaced with single-sex facilities, a Tory peer has said.

Lord Lucas told a House of Lords debate that half of all lavatories and changing rooms in public buildings should be for women only, reversing a trend towards letting men and women share public lavatories.

Increasing numbers of publicly accessible lavatories are being converted into facilities that can be used by all. Last autumn, the Old Vic theatre in London converted all its male and female lavatories to be gender-neutral.

However, Lord Lucas said many women did not want to share lavatories with men. Blaming a culture of "wokeness", he said: "Women do not want to wash bloody underwear in front of men. What is the justification for making women feel unsafe where now they feel safe?

"Some institutions have converted their ladies and gents communal toilet facilities to gender-neutral. Others have converted changing rooms similarly. Is this desirable or justified? What research as to people's needs is it based on? Has anyone – and women in particular – been consulted?

"Who does this change disadvantage? Women who do not, for reasons of discomfort or religion, wish to find themselves in an enclosed, unobserved space with men."

Lord Lucas, a hereditary Conservative peer, said he had decided to raise the issue after using the gender neutral lavatories at the Westminster offices of the Department for Education (DfE).

He said: "Men [like me] who do not wish to spook such women... I found using the gender neutral toilets at the DfE deeply unpleasant.

Lord Lucas added that "women who are unappreciative of the not infrequent lack of toilet hygiene and/or sexual behaviours and/or violent behaviours in men" and "women who need the place as a refuge", for example in a nightclub, should be able to use a women-only facility.

He called on the Government to explain the terms of the existing law better so that the managers of public buildings recognised they were able, in law, to make lavatories or changing rooms male or female only.

The peer pointed out that the Equality Act 2010 made it lawful to exclude men from "women-only spaces" in buildings, and said: "Organisations that take money from the public – public institutions or theatres like the Old Vic – should provide women with women's toilets to the extent that there is a demand for them.

"In institutions such as theatres, where the queues for the ladies are notoriously longer than those for the gents, going entirely gender-neutral creates more spaces where men feel comfortable and fewer that suit women – an entirely perverse outcome."

During the debate, former Tory Home Office minister Lord Blencathra, who uses a wheelchair, challenged the Government to "stand up" to transsexual activists.

Saying he agreed about the inadequacy of public toilets, he added: "There are over 70,000 public buildings which wheelchair users cannot get into, let alone have the luxury of deciding which toilet we use.

"So my blunt message to the Government tonight is this – when will you stand up to the small, militant, transgender fascist lobby and say the rights of 32 million real women and 800,000 wheelchair users are more important than the rights of tens of thousands who identify as transgender?"

He said that "sex is biological and binary and it is not a social construct" and called on the Government to maintain support for a current system that allows people to change gender over a two-year period rather than drop it in favour of "the absurdity of self-identification".

The DfE said it had not made any estimates of the "percentage of visitors who may feel excluded by separate communal male and female toilets, and of both toilets being gender-neutral".

Lord Agnew of Oulton, an education minister, told peers last summer: "The department has not identified any particular group of visitors that might feel excluded by separate communal male and female toilets or by both toilets being gender-neutral.

"The current approach for visitors supports individual choice rather than presupposing that particular groups of individuals have set or predetermined preferences. Visitors have not fed back any concerns or sense of exclusion regarding the provision."

The Old Vic has said four out of five of its visitors either agreed, or neither agreed nor disagreed, with the re-designation of its lavatories as gender-neutral.