Image copyright Getty Images

When Foals told their fans there was a "pesky X Factor type on their heels", the dig didn't go unnoticed by the man in question.

"I was preparing for a civilised chart battle with the Oxford rockers," says James Arthur, with a smile.

His third album, You, went head-to-head with Foals Everything Not Saved Will Be Lost - Part 2 to play for this week's number one spot.

In the end, he lost out but says he'll wear the term "pesky" as a badge of honour.

Image copyright Getty Images Image caption Foals went head-to-head with James Arthur for the number one album

Since winning the X Factor in 2012, the singer from Middlesbrough has gone up, down and sideways through the music industry.

In short, he had a huge single (Impossible), was later dropped by Simon Cowell's record label before being re-signed to it years later, and going on to have a number one comeback single in 2016 with Say You Won't Let Go.

James tells Radio 1 Newsbeat he's had to make it in the business "the hard way".

"I was poor when I started trying to make it. I didn't have any inheritance to make albums.

"People think doing the X Factor is easier, but it's hard. How many people stick around from that show."

It's clear from our conversation that he thinks people are snobby about him because of his appearance on the ITV talent show.

"Clearly there is (snobbishness)," he says. "I've paid my dues, I've played shows to one man and his dog and travelled around England trying to make it. I was in bands for years before X Factor.

"The show was a platform for me and I've used it well."

In a recent TV interview, James said he was disappointed he'd never been booked for festivals such as Glastonbury or Reading & Leeds, saying it was his "dream" to play there.

"I feel a bit embarrassed I've whinged about it. I've seen people saying that I should get my head out of my backside but I'm just telling it like it is.

"I've had a number one record, I am representing the UK around the world. I have the streams. The only reason I can see that I wouldn't be getting the accolades or offered to play Glastonbury is because it's not cool.

"Maybe it's an unpopular thing to do, to put someone who comes from that X Factor machine on the bill.

"It's what I wanted from being a kid. I picked up a guitar because I love Nirvana. It wasn't my path because I didn't have the resources to do so."

"It is what it is," he says.

Former X Factor star Lucy Spraggan, who appeared on the series in 2012 which Arthur won - told the Press Association she thinks she is the only act from the show to have ever played at Glastonbury. She first performed in 2017 and again in 2019.

Image copyright Getty Images Image caption James Arthur played Radio 1's Big Weekend in his home town of Middlesbrough

James' new album, You, has taken him far longer to complete than he would have liked.

For a start, he says the success of Say You Won't Let Go actually proved a hurdle to what came after.

"It was such a big song from that last album that it was hard to showcase anything else. I had this big, amazing ballad - an anomaly - in the way of everything else I wanted to show people."

The first single from this new album, Naked, was released in 2017.

"I went through a lot of stress making it," he says. "It's been sitting inside me. There's been a lot of toxic energy that I've just wanted to get out."

Image copyright Getty Images Image caption Throwback to when James Arthur was on X Factor with Rylan and Jahmene

His aim was to make something that resembled the "rawness" of his early career, something that fans saw in his very first X Factor audition, which has had more than 88 million views on YouTube.

"It seems people resonate with that more... I was determined that my new material came one-hundred per cent from the heart.

"I would just throw lyrics at the wall all day like a stream of consciousness."

His own mental health struggles have been discussed before, having previously told us that he had to manage his anxiety every day.

He says it's still an "ongoing thing" but says he's buzzing at the moment.

"I just love being out on the road. I love touring. When I'm healthy and things are going well then there's nothing that can bring me down - so I'm super happy."

Follow Newsbeat on Instagram, Facebook, Twitter and YouTube.

Listen to Newsbeat live at 12:45 and 17:45 weekdays - or listen back here.