"Your vagina will not be penetrated. Your vagina is a temple." With these words, Sleeping Beauty establishes the ground rules and sets the scene for a bizarre sexual nightmare. The film is set in Australia but conceived in a high, contemporary European style: it is perpetually a surprise to realise the dialogue is in English and not French or Austrian-accented German.

It is technically elegant, with vehemence and control, though often preposterous, with the imagined classiness of high-end prostitution and art-porn cliches of secret sexiness in grand chateaux: shades of Eyes Wide Shut.

Author-turned-director Julia Leigh has certainly made an assured debut, which evidently owes nothing to Jane Campion who has "presented" this movie in some kind of Executive Mentor capacity. Instead, Leigh aims for the occult ritual of Buñuel and the formal exactitude of Haneke: rigorously framed and composed shots.

Emily Browning gives a fierce and powerful performance, which should put her in the running for the festival's best actress prize; she is Lucy, a student who takes part-time jobs to pay her tuition. These include medical experimentation – an almost unwatchable, and crucially penetrative process in which a tube is inserted into her gullet – and occasional prostitution. From here, Lucy is inducted into a niche sex industry: elderly gentlemen in tuxedos hold dinner parties and pay beautiful young women to wait on them, almost naked. Her success leads to a lucrative "sleeping beauty" gig. She is drugged and these ageing, sagging sensualists can do whatever they wish to her naked body, except penetration. But Lucy becomes obsessed with knowing what is being done to her.

The movie's emotional seriousness consists in Lucy's unhappy backstory: it is not merely a need for cash that has led her to this, but a kind of trauma and self-hate. She evidently had a painful and complex relationship with an agoraphobic drug addict called Birdmann (Ewen Leslie); he was and is clearly in love with her and, on her many visits, her affectionate friendship is complicated by unexpressed guilt at not feeling the same way or wanting to sleep with him.

All of this Browning suggests in her sulky pouts, her suppressed tears, her angry blankness: especially when talking to her alcoholic mother on the phone. It all leads to the grotesque situation of lying in a drugged stupor for men who wish to indulge in a necrophile-rape fantasy. The sleeping beauty of the original fable fainted on pricking her finger on a spinning wheel, an image of a moment's daring combined with one of women's work and torpid docility. Lucy is certainly doing women's work: but what is she doing? Will she wake up to what is happening?

The stomach-turning "sex" scenes themselves brutally show how unlovely is the ageing male body: and the penises here are pitifully unable even to attempt penetration. To some, the situation may call to mind Almodovar's Talk to Her or perhaps the denouement of Dennis Potter's Brimstone and Treacle. But there is force and originality in Leigh's work. Sleeping Beauty is an impressive technical display, though no more than the sum of its parts.