"If all I was trying do was shock," says Jonathan Anderson, "I'd make it really shocking. It would be easy."

We're in Dalston, London, home to many a twentysomething fashion designer, sitting in the JW Anderson archive, which houses all the patterns and clothes from the label's five-year history. Upstairs in the studio, fittings are under way on the women's resort collection as Anderson's design team pin a model into a toile. They're in the final phase of the new menswear, too, for spring/summer 2014, a collection that will be shown in a fortnight's time at men's fashion week in London. But in the basement we're talking about autumn/winter 2013 – which, thanks to the temporal shiftiness of the fashion calendar, is the one that will sell this summer.

"It caused violent reactions," says Anderson, gleefully. No one was surprised when it appeared in the Daily Mail's savaging of London's more imaginative menswear shows. "I thought that was great! It meant the collection was challenging perceptions. But some journalists [he means fashion journalists, the sort that understand him and stood by him from day one] really couldn't handle it either."

Anderson has never shown much interest in the gulf between menswear and womenswear – he is not in the business of offering dinner suits or cocktail dresses. He studied menswear at London College of Fashion after a childhood interest in theatre led him to focus on costume. He launched as a menswear brand, but over the years JW Anderson has presented kimono-style coats that look a bit like dresses, jumpers hanging from the waist that look a bit like skirts and shirts in lace so sheer they look a bit blousy. Those who champion his work tend to use adjectives such as androgynous or gender-bending, neither of which accurately convey the cool neutrality of Anderson's approach to gender. He prefers the word unisex. As a teenager, he says, he was obsessed with CK One, Calvin Klein's unisex scent that hung heavy over dancefloors in the mid-90s: he still buys a bottle when waiting for a flight. "People forget about unisex, but I think it's very relevant now." Few labels show clothes where, season after season, men's and women's collections are so similar. Women buying his clothes led Anderson to venture into womenswear in 2010.

Suits you: models form the latest JW Anderson catwalk shows. Photograph: Observer

Anderson's take on unisex in this autumn's menswear collection even unsettled some of his fans. There were leather shift dresses, shoulder-baring tops that wrapped around the chest like bustiers, the many ruffles, and hems on short shorts, tops of riding boots and on gloves, making them look like luxury Marigolds. Elements from womenswear were grafted in with an uncompromising bluntness that emasculated the male models.

"I love how groinless and flat-fitting it was," says Anderson. The shorts, for example, were made using patterns drafted for women, hence their castrating cut. Male designers tend to create clothes that play up masculinity on the understanding that sex sells, but satisfying obvious desires doesn't appeal to Anderson. "I think that's what unnerved people: there was no gay fantasy there."

Anderson takes fewer risks when it comes to his own appearance. What the 29-year-old wears today – a grey jumper over a black T-shirt, blue jeans and Converse – is pretty much his uniform. His T-shirts are by Sunspel, the British manufacturers of cotton basics that supplied longjohns to the British empire: 'They make the best in the world." Anderson is the label's creative director, so you might expect him to say that, but he was only offered the post after visiting its Nottinghamshire factories as an enthusiast, soliciting their input on a JW Anderson collection. "What they make is so painstakingly done and flawless. I love the modernity of something that pure."

Sunspel is not the only contrasting brand the designer has collaborated with. During college Anderson worked as a window dresser at Prada and as designer he's overseen two Topshop collections. In May he unveiled his first for Versus, Versace's youth-focused second line. It was one of fashion's less obvious marriages, given Versus always traded in a sexiness that JW Anderson defines itself against. "It was good. It took me out of my comfort zone. At first I was, how do I do Versus in my way?"

For research, Anderson pored over Versus's back catalogue of promotional shoots from the 1990s, much of which ended up on his Instagram feed. "I think people forgot that brand, and how copied that imagery is. So the purpose for me was to articulate how important Versus was in fashion history." There was one particular image of a boy wearing a diamond tiara that epitomised Versus's appeal for Anderson and provided his take on the brand: "I just wanted to do that crown." Every boy in the Versus show walked out in a tiara.

As he talks passionately about Versus, it becomes clear what excites Anderson most about clothes is their status as objects, their totemic power to represent a moment and mood in fashion. "Yes, I do love an object. As humans we love what that object stands for." He remembers as a teenager seeing an image in a fashion magazine of "a massive wide-leg trouser on a kid on a swing. I was obsessed with those trousers and that character." He hopes his own clothes inspire a similarly heady mix of identification and desire. "Ultimately as a designer you want to create objects that take people somewhere else, that on a model will create a sense of character that makes people think, 'Oh my god, I wanna be that!'"

It's a loftier aspiration than churning out clothes that look flattering or sexy – and one the buyers for big stores have learned to appreciate. "We sold those looks, you know!" says Anderson, of the emasculating items from his autumn menswear collection. "I was surprised by how many ruffled shorts we sold."