“There’s a kind of natural urge to show off your body and sexuality and record it,” the photographer Terry Richardson told me in the autumn of 2000. “We all have our own internal porn star, and technology has allowed it to develop. You know, people flash the camera when they’re drunk at parties, there are couples filming themselves with camcorders, it’s like instant porn. That genuine, raw sexuality is breaking the veneer of polite behaviour all the time. I try to capture those moments.”

The words sound sinister after a week of re-reading the many allegations of sexual harrassment and assault made against Richardson since 2001. One man’s “genuine, raw sexuality”, we have been reminded, was for several women an exposed erect penis being waved in their faces.

And yet his proselytising was once taken seriously, and not only by the fashion industry. Curators at fine art galleries, broadsheet newspaper editors and highbrow book publishers all nodded along sagely. In 2000 I was interviewing him for a long story about pornography’s influence on fashion for the Daily Telegraph. I had been well placed to watch the phenomenon develop, having in the mid-90s edited the Face magazine, which ran some of his early pictures. Some of the less obvious reasons for the fashion industry’s delayed reaction to the allegations might be buried in that period.

I met him three times, all before the first harrassment allegation against him, in 2001 and, so far as I’m aware, before he started publishing photographs of himself having sex with models. There was gossip about his shoots, but mostly concerning celebrities; I recall stylists recounting rumours of a bisexual orgy with supermodels in his New York studio, and jokingly referring to his pictures of this carry on as “Terry’s pension”. The truth of that tale has not been proven, but I think the men on the staff wanted it to be true because it brightened his aura of interesting glamour.

Facebook Twitter Pinterest Richardson with the fashion designer Stella McCartney in New York in June 2017. Photograph: Schildhorn/BFA/REX/Shutterstock

Richardson felt like an exciting person to work with. His father, Bob, had helped to invent modern fashion photography in the 60s, and when Terry was four, Bob left his mother for Anjelica Huston. Terry’s mother then went out with Jimi Hendrix and, later, Keith Richards. Terry had grown up in Los Angeles, and had good stories.

Plus, he seemed endearingly humble and funny. I don’t know if that lasted (“I was a shy kid and now I’m this powerful guy with a boner, dominating all these girls,” he would tell an interviewer years later) but at the time all these qualities and connections meant he had an allure, to men at least. He has since said that he was using heroin and drinking heavily before 2001. I can only say that it didn’t seem apparent to me, and I didn’t hear it mentioned.

It also helped his cause that he could talk about his work as a social and artistic project, with a unifying aim and purpose. “I’m interested in trying to capture something spontaneous,” he said. “Trying to capture those unpremeditated moments when people’s sexualities come up to the surface.” He talked about his images being “fashion pictures that mix art and erotic photography”, and about his shoots as artistic “happenings” that he faciliated while other people actually pressed the camera shutter. His tongue may have been partly in his cheek, but part of him seemed to believe it.

Benedikt Taschen, the German gallery owner and publisher of art, photography and erotica books, bought in to this, publishing the infamous 2004 Terryworld book, with its images of Richardson having sex with models (“Who took 1970s porn aesthetic and made it fashion chic?” asked the blurb). Jenna Sauers, who wrote about the accusations in Jezebel, claims she was contacted by an anonymous model who said “I mean, his assistants were like, “Do you think all these celebrities would take pictures with him if it was porn?” “Which all meant,” one fashion journalist who spent time with him and his people in the 2000s told me, “that he could do what he wanted with girls, and if they said they didn’t like it, he could say well, you don’t get it, do you? You’re dull. That alone can be quite persuasive and manipulative.”

Richardson seemed part of – though not defined by – a trend for referencing grainy, low-budget 70s pornography that originated with photographers including Juergen Teller and Steven Meisel, and extended to fine art, Hollywood, pop music and advertising.

One of Richardson’s early catalogues for Sisley was based on a story about male porn stars and their girlfriends; Terryworld was edited by Dian Hanson, a well-known editor Taschen recruited from porn magazines. The trend was partly driven by a desire to create a raw visual style as opposed to the good taste of the 80s, but it went along with a sort of detached, ironic attitude that purported to see porn as consequence-free images.

In reality, it meant it was easier to ask models to take their tops off, and porn mags could be scattered around bars and offices, which was intimidating for women. It shows what happens when women don’t feel they can speak out and be listened to, and, as the actress Kate Hardie recently pointed out, the need for men to recognise connections between product content and workplace experience.

At the start of the 2000s, Richardson’s own aesthetic and reputation really took off as, partly under his influence, fashionable young urban people adopted what the critic Mark Greif calls the white hipster style. As Greif has pointed out, this look – trucker caps, cap-sleeve T-shirts, Aviators, tattoos, porny moustaches, cheap tinned beer as a style statement – appropriated the taste of 1970s American white suburbs and “trailer trash”. And while this culture could seem terribly ironic and kitsch, it served as a cover for some 70s-style racism and sexism.

Richardson became that most prized thing in fashion – someone who influenced and even determined tastes.

This was, after all, the heyday of Vice, for which Richardson was now shooting and defining a visual style. Vice, which in 2009 published pictures of one of its interns who had been persuaded to be photographed naked by Ryan McGinley, was co-founded by Gavin McInnes, now a prominent figure on the US alt-right who founded the Proud Boys men’s organisation. This year he used his slot on the rightwing Rebel Media website to blame the death of civil rights activist Heather Heyer at Charlottesville on “feminism”. He was also credited as an author of Taschen’s 25th edition of Terryworld.

I have no idea whether McInnes’s political ideas are connected with Richardson’s work, but it raises questions about the risks of ironic detachment and “consequence-free” aesthetics. There was little wondering at the time because, with his noughties ascendency, Richardson became that most prized thing in fashion – someone who, in an industry where billions of dollars ride on six-monthly shifts in taste, influenced and even determined those tastes.

There is another point. In the Face’s office in the 1990s, issues were sometimes raised about the treatment of young models in Paris and Milan. In 1997 we ran a cover story by the writer Chris Heath in which the model Karen Elson spoke frankly about her experiences of those of her friends. Of working in Milan, she said: “There are a lot of blond-hair, blue eyes very stunningly beautiful girls out there, and these men just say the most hideous things. Literally pull the pants down, do the thing, say ‘Amore – watch this.’ There were little 15- or 14-year-olds from Poland or from England or wherever, and they’d be out with these playboys in Italy, being taken out, wined and dined. And then guess what they expected in return?”

The story attracted angry complaints from Elson’s agency, Models One, and at one point during the interviews we had the impression that it was trying to sabotage it. The agency was angry, chiefly because we had said that it had repeatedly lied to us during the arrangements, but I found it striking that they didn’t mention Elson’s complaints. And while agencies often protested that they couldn’t protect models all the time, they could get pretty protective when their own reputations were threatened.

If “everyone knew” about the Richardson allegations, so did the agencies who continued to send young women to his castings without – say the women – adequately preparing them. Most people in the industry agree that in recent years agencies’ responsibilities have been taken more seriously, particularly in London. However, there are still stories of predators in Milanese bars and Parisian nightclubs, and unknown sleazebags and women from Gdansk are less sexy for the media than stories in which you can show pictures of Miley on her wrecking ball, or weird Uncle Terry gurning with Barack Obama. It would be heartening if, rather than acting as if Condé Nast distancing itself from Richardson will purge such problems, the debate moved on to some of that.

• The author was editor of the Face from 1995-98