An alarming number of Australians believe that violence towards women and rape can be excused and blame the victim, particularly if she is drunk, a new report shows.

About one in five Australians agree that a woman is partly responsible for rape if she is intoxicated, and one in six agree with the notion that when women say "no" they actually mean "yes".

About 80 per cent of people agreed that it was hard to understand why women stayed in violent relationships.

A nationwide VicHealth survey of 17,500 people about their attitudes towards violence against women discovered that although most people understood that family violence was a crime, a sizeable proportion believed there were circumstances in which it could be excused.

About four people in every 10 agreed that rape was the result of men not being able to control their need for sex, while more than half agreed that women could leave a violent relationship "if they really wanted to".