It started during the 1999 eclipse. The year before, I had run away to Devon. Lots of bad things had happened there, but the main one was that I’d got fat. On my 27th birthday, I was 5ft 6in and weighed a hefty (as I bizarrely thought then) 9st 2lb. Worse: according to every news story I read, I was going to get an incurable disease and die. The most likely cause would be food; mad cow disease hadn’t gone away, and then there were pesticides and insecticides and growth hormones. When a friend told me he’d heard organic diets were cancer-preventing, I was in. As the skies darkened on 11 August, and birds began their evening song hours too early, I pledged that if I survived the solar eclipse, I would eat only organic food. I would stay healthy, and I would not die.

When organic food didn’t make my life perfect, I tried food combining (no protein with carbs). Then veganism. For 20 years now, I have cycled between diets and diet books, in search of the perfect hack for a good life: great health, better skin, the optimum weight and all, of course, with minimal impact on the environment. (Like so many women who dedicate their eating disorders to saving the planet, I need what I eat to be in some way an ethical choice.) I have been a vegetarian, a meat eater; I have gone paleo, keto, macrobiotic, pegan (look it up).

Now I am in my 40s, I have more than 100 different food rules that I have collected from various places. They range from the obvious (white wheat isn’t that healthy) to the ridiculous (tomatoes are very “yin”). Food rules are not hard to find. You can follow hashtags on Instagram such as #cleaneating (45.5m posts) or #vegan (85.3m) or #absaremadeinthekitchen (1.6m). There are thousands of YouTube What I Eat In A Day videos posted by famous models and influencers. A bit of Googling can put you in touch with Victoria’s Secret diets, ballerina diets, even diets to improve your mental health. You can learn why veganism is good for your heart, and why keto helps with anxiety and autism. The big question, though: which one to choose?

I am a novelist, but diet books are my embarrassing comfort read. I read them when I am bored, stressed, on a late train, facing a deadline. Recently I realised that I use them like porn, reading my favourites over and over again. As with porn, I quickly discovered a “type” of narrative that works best. Perhaps my ur-diet book is Robb Wolf’s The Paleo Solution, which begins with the author’s mysteriously sick parents, then describes how his own long period of vegetarianism left him almost dead at the age of 28, mainly as a consequence of his undiagnosed coeliac disease. Wolf began his new life with meat and a salad, and never looked back: “I was warmly satisfied, clear headed and I felt better than I had in years. After one meal.” I have read this passage over and over again, particularly when I am having a vegan crisis and need an excuse to try something else.

“Is it weird that I love diet books?” I sometimes ask people, experimentally. But no one else seems to have my problem, even though everyone is now an expert on diet. While most people take it all with a pinch of salt (not so bad for your blood pressure, it turns out), I take it all very, very seriously. I’m not sure I’ll ever be able to eat without a precise set of instructions. One major joy of my diet books is feeling that I belong, that I am part of a belief system, a member of a team. But the downside is, this stuff can send you nuts (which have more carbs than you’d think, but are a good source of selenium and fibre).

Most of the time, this obsession is fine. And then there are the dark times when I suddenly have to hard-delete every paleo book from my Kindle, only to buy them all again the next day. Or when I go through my Ocado favourites and remove anything not strictly vegan and spend hours creating the “perfect” clean order, only to then cancel the whole thing in the nick of time, just before it leaves the warehouse.

After one bowl of pasta and a dessert, I had a terrible panic attack. For hours I lay in bed hardly able to breathe

When I am busy with my writing, happy and distracted by life, I can stay on a mild version of one of my regimes for a few months. But when stressed, I start cycling through the possibilities like an anxious lab rat trying every door to find the one that leads out. I still want so badly to get it right. I want people to stop me in the street and say, “But you look amazing! And you seem so very, very healthy. What’s your secret?” I know a lot about every single possible answer. But at my worst moments, I have forced myself to believe in completely different paradigms, sometimes all in one day. Not so much believing six impossible things before breakfast as believing six impossible things about breakfast. Could I be a spiritual person while eating meat? Is our time on Earth supposed to be about kindness and the veneration of life, or is it better to accept the natural cycle: that we eat animals and are then returned into the earth to feed them? Was everything I ate in some way wrong?

As a fiction writer, it’s my job to wonder about, and dramatise, such things. But that drama has filtered through my life. During a pegan episode, I ended up hungry at a pasta restaurant with a limited menu. After one bowl of pasta and a dessert (well, why not?), I had a terrible panic attack. For countless hours I lay in bed sweating and hardly able to breathe, believing that gluten was attacking my insides, that I would develop an autoimmune disease, schizophrenia and lupus. At other times, I ate meat and fixated on the rotting corpse in my stomach, desperately wanting it out.

***

The term “orthorexia” was coined in 1997 by Dr Steven Bratman, author of the leading book on the subject, Health Food Junkies. I don’t have that book any longer: I Marie Kondo-ed it after it calmly told me that it was possible to die from orthorexia. This did not bring me joy. But it also confirmed that I had been suffering from it for several years. Despite increasing awareness, orthorexia is not listed in the International Classification of Diseases, or in the Diagnostic and Statistical Manual of Mental Disorders. However, it has made its way into the Oxford English Dictionary, and is increasingly ingrained in our culture.

Orthorexia is an obsession with “pure” or “healthy” food. It is treated as a type of anorexia because sufferers typically reduce their food intake over a period of time, often becoming raw-foodists, “juicearians” or even “sproutarians”. But it doesn’t have to be like that. Sufferers may appear to friends as simply “a bit fussy” or, like me, as “health-conscious”. And suffering inside doesn’t always show. A friend recently asked me if I (like her) felt like an impostor because I don’t look like someone who has an eating disorder. I knew what she meant. The standard eating disorder recovery “look” on Instagram is willowy and tragic, and I am a size 12-14.

In many ways, I don’t even stand out among the people I know. I have two close friends who struggle with irritable bowel syndrome. In the last two years, one went to a kinesiologist (they are the ones who make you hold a food item; if you feel weak, you are allergic to it), while the other had parasitic hookworms injected into her arm. Another close friend is low-carbing to combat her polycystic ovary syndrome, and another is on keto for his arthritis. We have all become persuaded we can self-medicate through food, and maybe this is even true. My friends are all neurotic, but they glow with health.

Whenever I sit down for dinner with my editor, we compare regimes. “Land animals,” he said last time, explaining what was taboo for him at that moment. “And dairy.” Another friend moves between low-GI, low-carb and fasting. We have both been doing 16:8, where you fast for 16 hours of every 24, for ages. We send each other pictures of sticks we have peed on to see if we are in ketosis, the point at which the body starts breaking fat down. Another friend is on the warrior diet, which involves eating one big meal a day. We all have proper jobs in health, publishing or the arts.

When I tell people I am recovering from an eating disorder called orthorexia, they usually mishear it as “authorexia”, and perhaps, for me, that’s what it is: an eating disorder based around storytelling. An eating disorder for authors, or at least encouraged by them. One macrobiotic writer suggests listening to Shakespeare if you have heart trouble, because the iambic pentameter will stabilise you. When it comes to the diet books, the different regimes, it is always the stories that do it: the familiar, comforting narrative arc of the lost soul who discovers the secret that leads to health, purity and beauty. The brave, bold young American who finds the truth that eluded his parents. The fat-adapted ultramarathon runners who do it all on a spoonful of coconut oil.

***

After my 1999 solar eclipse weight crisis, I bought two books about food combining in a Dartmouth bookshop. Before that, I’d rarely seen a diet book. When I was growing up, our house was full of books, but not that kind. After all, wasn’t dieting one of those twee things oppressed women did in the 50s because the patriarchy said they should?

The food-combining books didn’t just tell me not to eat protein and carbs together, advice I have tried to follow ever since (I still feel guilty when I fail). They told me what would happen if I did: along with indigestion and arthritis, I might get candida, anxiety, depression, IBS and chronic fatigue. One chapter was titled How Toxic Are You? The symptoms included being bored, tired, having aches and pains, few ideas and poor concentration. This was me! (And, of course, every writer I knew.) How lucky I’d found these books before I died of literally everything. But there were the success stories, too, and these became what I craved. Stories of people being brought back from the brink, people who had almost died and then been cured by avocado cream.

I bought more books, which could be unintentionally funny, too. One favourite talked knowledgably about the behaviour of animals in the wild. “When that lion ate the zebra, it wasn’t having a baked potato with it,” it pointed out. But, of course, I was not a lion. I was following all the instructions, but not losing any weight. In fact, I was putting weight on.

If I was not a lion, I wondered, what was I? A caveperson, came back the answer. A human with a genetic code that has not fundamentally changed for 10,000 years and whose system simply cannot compute “Haribo” or “bagel”. At first I ruled out “ancestral” diets because of all the meat, but Rose Elliot’s vegetarian low-carb book became the gateway that culminated a couple of years later in Wolf’s Paleo Solution. There were heady days of fried halloumi and eggs and mushrooms in butter, bowls of berries and cream. I loved the food, and most of my How Toxic Are You? symptoms went. I stopped feeling nervous all the time. But I didn’t lose any weight, so I went vegan again.

For the next few years I swung between extremes so frequently that from a distance I might have appeared moderate. My weight increased until it hit 11st (70kg), and then refused to budge. On the outside I looked like a M-L person, a Topshop size 14, able to fit into 30in-waist jeans from anywhere but All Saints. Inside, I was suffering all the time. Pictures taken at literary festivals showed me with a double chin and flabby arms, looming over a lectern like Jabba the Hutt. Me! In my head I still looked like Kate Moss in the 90s.

I started looking for clues about what other women ate: the ones who still have flat stomachs in their 40s, who still look like pictures in magazines and who don’t seem to use three filters at once on Instagram (like I do). I am ashamed to say that when I spotted Arundhati Roy looking glowing in a hotel restaurant at the Edinburgh International Book Festival this year, I watched to see what she ate, and was disappointed by a cappuccino. (But was it soya, almond or cow’s milk?) Enabled by a world that is still fixated on women’s bodies, on slow days I would do online research to determine which of my regimes actually worked on other people. The fact that they’d failed to work on me was my fault, not the fault of the diet, of course.

It is unlikely I will ever be able to get out of my head all the stories I believe about food and illness

Had they failed? Perhaps I sound mentally unstable, but physically I am as strong as an ox. Whenever I am ill – which, perhaps due to all the kale and vitamins, is very rarely – I think back to the last thing I ate and blame it on that. But could it be that these “healthy” diets actually work? Maybe I am meant to be a size 12-14 person who gets one (if that) cold a year. Of course I am constantly on guard against any sort of illness, ready at all times with my turmeric and green tea and manuka honey. And I don’t want to admit I feel healthy in case I tempt fate and am diagnosed with an incurable disease.

I’d love to say that all this is in the past, but it is unlikely I will ever be able to get out of my head all the stories I believe about food and illness. In that sense, I’ll be authorexic for ever, even if at times I have got my orthorexia more or less under control. I spend most of my time now on a lowish carb diet simply because it makes me feel better mentally and comes with less plastic packaging, and I’ve almost, but not quite, come to terms with my size and shape. But I can still be easily triggered: a column on being vegan for the environment can turn me for a week or two; coverage of the health benefits of Tom Watson’s keto diet will exert an influence. If I had seen Roy’s breakfast, I would have adopted it.

Meanwhile, my Instagram heaves with body positivity. I love Lizzo and Ashley Graham, and all the many muscly heroines of CrossFit. But they are young and beautiful, and I am 47. They are naturally glorious, while I still need help. I just don’t have that jawline – though a part of me thinks that there is a diet to fix that. One of my favourite new books is about intuitive eating, where you eat what you want (yes, including Big Macs and Twirls) on the basis that your body will eventually ask only for what it needs. As one of my friends despairingly asked, “How is that even possible?”

Last week I was in the supermarket trying to decide between organic brazil nuts and nonorganic pistachio nuts (this is close to my natural state) when I became aware of two teenage girls in the same aisle. They were browsing the “Free From” foods. They looked perfectly healthy and robust: everything you should be at 15. They were surrounded, as I was, by abundance: cheap, plentiful food. “Let’s get some gluten-free stuff,” one said to the other. “Apparently it helps you lose weight.” I wanted to interrupt, to say something to them, but what? That it’s not true, and all that gluten-free stuff is just full of sugar? That they really don’t need to lose any weight? What would I have told my younger self, before I embarked on my orthorexic journey? I don’t honestly know. So I chose the brazil nuts and left, hoping they would simply grow out of it, but knowing, given the size and power of the wellness industry, with all its compelling stories, that they probably won’t.

• Oligarchy, a novel by Scarlett Thomas, is published on 7 November (Canongate, £14.99). To order a copy for £11.99, go to guardianbookshop.com or call 0330 333 6846.

If you would like a comment on this piece to be considered for inclusion on Weekend magazine’s letters page in print, please email weekend@theguardian.com, including your name and address (not for publication).