When Jesy Nelson was 19 and working behind the bar at a pub in Dagenham, Essex, she remembers watching The X Factor on TV, and thinking: “I know I could win that.” In 2011, she did just that, as part of the girl group Little Mix – and thought: “This is the worst day of my life.”

Competing in Simon Cowell’s singing contest unleashed ceaseless criticism of her appearance and weight (although rarely her voice). “All I cared about was what people were saying about me,” she says now.

Winning offered no respite. When Little Mix were crowned, the first Facebook message she saw was from a stranger. It read: “You are the ugliest thing I have ever seen in my life. You do not deserve to be in this girl band, you deserve to die.”

“I should have been on cloud nine,” she says. “I had Leigh-Anne [Pinnock, also of Little Mix] in my room being like: ‘This is the best!’ and I was like: ‘No, this isn’t.’”

Little Mix went on to become the biggest British girl group since the Spice Girls, but Nelson was consumed by the trolling and abuse on social media. Within two years of the finale, she had depression and an eating disorder and had attempted suicide.

The downward spiral and her eventual, slow recovery are the focus of an intensely personal BBC Three documentary, Jesy Nelson: Odd One Out. Before shooting it, she says, she had never spoken publicly about her struggles in the spotlight.

When we meet in a corner of BBC Broadcasting House in central London, Nelson, now 28, is friendly and glamorous, dressed in a double-breasted tangerine suit. It is the eighth anniversary of her X Factor debut and #8YearsofLittleMix has been trending on Twitter all morning, thanks to their fans, the “Mixers”.

On The X Factor in 2011: (from left) Jade Thirlwall, Leigh-Anne Pinnock, Perrie Edwards and Jesy Nelson. Photograph: Ken McKay/Talkback Thames/Rex/Shutterstock

Within minutes of sitting down, she says that, had she known the consequences of appearing on The X Factor, she wouldn’t have done it: “I don’t think anything is worth your happiness, and it was a lot of my life that I won’t get back.”

As a child growing up in Romford, Essex, Nelson was intent on becoming a performer, be it singing, dancing or acting. “I didn’t really have any reason to not be confident,” she says.

In mid-2011, she auditioned for The X Factor as a solo entrant, and was eventually placed in a group with three others: Pinnock, Perrie Edwards and Jade Thirlwall, all aged between 18 and 20.

Back then, social media was not as inextricably linked with reality TV as it is now. In fact, that eighth series was the first where applicants could upload their audition videos to YouTube; Nelson didn’t even know what YouTube was. She remembers being wowed when all the contestants were given new Samsung phones and told to get on Twitter to build their fanbase.

On the first live show 12 weeks in, Little Mix (then Rhythmix – the name was changed later) performed Nicki Minaj’s Super Bass to gushing praise from judges Louis Walsh, Gary Barlow and their mentor Tulisa Contostavlos. It was “the best feeling in the world”, said Nelson through happy tears on stage.

That night, off-camera, the contestants gathered to watch themselves on YouTube. Someone pointed out the comment section. “I was very naive,” says Nelson. “I thought it would be people giving their opinion on our performance. But nearly every comment was about the way I looked: ‘She’s a fat ugly rat’; ‘How has she got in this girl group?’; ‘How is the fat one in this?’” She remembers the air being thick with tension – “because no one knew what to do or how to react”.

“I felt a rush of anxiety, because I’d never experienced anything like that in my life. People were saying my face was deformed – just the most horrific things. I felt like I was heartbroken. I remember ringing my mum and saying: ‘Mum, I want to go home, I don’t want to do it.’”

Jesy Nelson with Liz Ritchie in the forthcoming BBC documentary Odd One Out. Photograph: Rahul Bhatt/BBC/October Films

At about 1am, a member of The X Factor team found Nelson crying alone and asked why she was so upset. A couple of days later, she was asked to explain again – on camera. She didn’t want to do it. “They told me it wasn’t recorded, and it was.”

A few weeks later, the clip of Nelson in tears over “a few nasty comments” was broadcast before Little Mix’s performance, the reality TV playbook of “sad piano” switching to upbeat pop music when Thirlwall comforts her: an uplifting moment of girl power. From then on, that was Nelson’s public narrative.

She does not hold that clip, or the producers, responsible: “I think it would have always happened – that just added fuel to the fire.” From the start, relatability had been billed as a central tenet of Little Mix’s appeal. Contostavlos introduced them as “the girl group to represent ladies in this country”; she framed Nelson’s tears as evidence of Little Mix having “the same insecurities as every other girl”.

Nelson, however, was the only member even remotely close to the average UK woman at size 16. Although the four bandmates have always been friends – “that’s why we’re still together” – she felt singled out. “I was with three other girls to be compared to. I don’t think it would have been as bad if I’d been on my own.”

After the clip presented her as Little Mix’s weakest link, the abuse snowballed. “It was like as soon as people knew that it was really affecting me, they wanted to do it more.” Nelson had been bullied at school, to the point of stress-induced alopecia – “but this wasn’t playground stuff”.

She was shocked by the cruelty from adults – some clearly parents. “Obviously everyone sits in their living room and will see someone on TV and make a comment. But to actually pick up your phone and go: ‘I’m going to make sure this girl sees it’ – even if they didn’t think I was going to see it – you have no idea the effect that one comment will have.”

Nelson became “obsessed” with reading criticism. The praise didn’t register. “It only got worse when I got Twitter. And that led to the Daily Mail, and reading the [below the line] comments – the worst you can read about yourself. It was like I purposely wanted to hurt myself.”

“I had a routine of waking up, going on Twitter, searching for the worst things I could about myself. I’d type in the search bar: ‘Jesy fat’, or ‘Jesy ugly’, and see what would come up. Sometimes I didn’t even need to do that, I’d just write ‘Jesy’ and then I’d see all the horrible things. Everyone told me to ignore it – but it was like an addiction.”

At one event, Nicola Roberts of Girls Aloud – who had seen the clip of her crying – took Nelson aside. “She said: ‘Can I just give you one bit of advice? Please don’t read stuff about you. It’s the worst thing you could do.’”

Little Mix in Sydney, Australia, in 2013: (from left) Perrie Edwards, Jade Thirlwall, Leigh-Anne Pinnock and Jesy Nelson. Photograph: Newspix/Rex

Nelson rolls her eyes self-mockingly. “But did I listen? No.”

Contestants had been told help was available if they were struggling, but Nelson had learned that talking only made the problem worse. “I don’t think any of the team really knew how upset it was making me – it’s just go-go-go, from the car into hair and makeup, then rehearsals.”

It was also a popularity contest. “We just wanted to make everyone happy, and we wanted everyone to like us.”

In December 2011, Little Mix became the first group to win The X Factor. Their debut single entered the charts at No 1 seven months later; DNA, their first album, was released in November 2012. Scrutiny of Nelson only increased amid the pressure to maintain momentum.

Although she tried not to discuss it, she feels the abuse came to define her public image. “I’d become a bit of a joke. People would make memes, chopping my head off in a group photo and putting a monster or ET on there. I’d be in live Q&As and these things would pop up and I’d have to just sit there.”

Interviewers asked her how she dealt with it; fans said they looked up to her. She was depressed and in denial: she refused antidepressants, and therapy didn’t help. “Our schedule was so gruelling. I was going to see a therapist at six o’clock in the morning, crying, and then going to a photoshoot.”

Meanwhile, in public, she was “giving speeches about being confident”. Little Mix, as the guardians of girl power, were not only supposed to represent every woman, but defend every woman.

“I felt I had to be this person who was like …” Nelson juts her jaw, sashays from side to side, a facsimile of her sassy music-video persona: “‘I don’t care what people are saying about me, I’m this strong woman.’ That was the role I had to take on in the group, when really I was an absolute mess.”

In the lead-up to TV performances or video shoots: “I’d starve myself … I’d drink Diet Coke for a solid four days and then, when I felt a bit dizzy, I’d eat a pack of ham because I knew it had no calories. Then I’d binge eat, then hate myself.”

Yet she did not see herself as having an eating disorder. “I could see that I was losing weight and sometimes I’d see a few good comments and that spiralled me to be like: ‘This is how I need to stay.’ No one cares whether your performance was good, or if you sounded great.”

Nelson in Odd One Out. Photograph: Jamie Simonds/BBC/October Films

Nelson started skipping events where she knew she would be photographed. On one magazine shoot, the wrong size clothes were provided. “I had a meltdown. I cried so much, I had to wear sunglasses. I did one photo, then left.” She hid her misery well, she says now. “I think people just thought I was a miserable bitch.”

Her lowest point was in the lead-up to Little Mix’s second album, Salute, in 2013. Her mum, Janice, increasingly desperate, told her she had to quit the band. Yet Nelson worried that leaving – or even taking a break – would draw more attention to herself. “Everyone’s going to ask why.”

In November 2013, Little Mix returned to The X Factor to perform their new single, Nelson notably slimmed down. Coverage centred on one tweet from Katie Hopkins: “Packet Mix have still got a chubber in their ranks. Less Little Mix. More Pick n Mix.”

Increasingly, Nelson felt trapped. “I felt that I physically couldn’t tolerate the pain any more.” She attempted suicide.

Nelson’s family, her management and the rest of the group knew – but “once it was spoken about, it wasn’t ever spoken about again”, she says. She was offered time off, but once more was too frightened of drawing attention to herself to take it.

The turning point came in February 2014, when Little Mix spent six weeks travelling across North America, opening for Demi Lovato. One day, on the bus, the dancers pulled her aside and told her she had to quit Twitter, likening it to a book filled with “loads of nasty things” that Nelson always had her nose in. She finally deleted her account.

“It was a long, hard process, because I didn’t want to help myself. But it wasn’t until I deleted Twitter that everything changed for me and I slowly started to feel normal again.” Through more regular therapy and talking to friends and family, eventually she was able to stop reading articles about herself, and distance herself from her public image even as Little Mix’s star continued to climb. In 2016, Glory Days became their first No 1 album in the UK.

Since February, Nelson has been dating the 2017 Love Island contestant Chris Hughes, who has defended her publicly from online trolling and who she says is a positive influence on her feelings about fame: “It’s nice to be around someone who doesn’t give a shit about all that stuff.”

Making the documentary also contributed; she lights up while talking about meeting a body-image specialist, Liz Ritchie, to help her understand her relationship with social media and the “mask” that she had developed to withstand the spotlight. Part of this involved going over footage from The X Factor, which was a difficult experience, but ultimately empowering.

“Don’t get me wrong, I still have days when I feel shit in myself but instead of beating myself up about it and being miserable, I think: ‘OK, I’m going to have my moment of being sad, and I’ll be over it.’ Before, I didn’t let myself be sad.”

Talking to other young people who have experienced online abuse made her feel less alone. “A lot of people think ‘stop moaning’, but until you’ve experienced it, it’s hard to understand – and it doesn’t just happen to people in the limelight. There’s so many people struggling with social media and online trolling. People need to know about the effects it has.”

The turnaround in five years, she agrees, is remarkable: now, as Little Mix work on their sixth album, Nelson is less conscious of her weight, her appearance, what she’s eating – even what is being said about her. To shoot the documentary, she returned to Twitter, and discovered some new slurs. “I didn’t even know some people said that about me, but it’s because I don’t look for it – and also, I. Don’t. Care,” she says, leaning forward in her chair.

“Now I’m mentally a lot happier, I just think people are always going to have an opinion. But I only care about mine.” She flashes a smile from beneath all her hair, happy but defiant – and for a moment she looks exactly like the girl in the music videos.

Jesy Nelson: Odd One Out will be available on 12 September from 10am on BBC Three on iPlayer and on BBC One at 9pm.