Steve has been with me for the past 50 years and Ron for 47. Neither is the man I am married to, nor have I seen or spoken to either since our love affairs ended in my 20s.

All the same, there is no denying they have both messed with my marriage to Olly, the man who has been by my side for the past 40 years.

I found myself thinking about them both as I read recent research that suggested women who played the field before marriage are unhappier with their lot than those who entered matrimony virginal.

Angela Neustatter has often questioned what life would have been like had she married another man

My first reaction was: why on earth would that be? I have always believed a bit of experience, in both love and sex, to be an asset to understanding what we really want when selecting a partner for life.

Having no history to draw on — settling down with no idea what else the world has to offer — seems a recipe for disaster, not satisfaction.

And yet there is no denying that my past lovers have made their presence felt in my marriage — at times, even making me question my commitment.

Olly and I have, along with a lot of fun, weathered many disagreements, and even contemplated separation after a mid-life hiccup saw all passion wilt.

Although we’re still together, would Olly and I have been happier, our journey to this point easier, had he been my one and only?

Not that I had notched up a great many scalps on my bedpost before we met when I was 29 — but there were a few, and, most significantly, Steve and Ron were both serious contenders for my heart.

Although my relationships with the two ended very differently — I left Steve in London for a job in Manchester when I was 24, with great pain; Ron found fidelity too hard and broke my heart by ending our 18-month affair abruptly — they both left me with an undeniably tantalising sense of ‘what if’.

Neither maintained any physical presence in my life, but memories of how good things had been with them at our best always lived at the back of my mind, like old love letters or black-and-white photographs gathered at the bottom of a box.

They faded during the early years with Olly, and were covered over by fresh memories made with him. We were absorbed in our new-found love, revelling in the unearthing of shared interests and enjoying our sexual chemistry, leaving no room for thoughts of anyone else in my mind.

So it is for many women in the early years after they tie the knot. But, as time passes, things change. The early days of romantic enchantment tend not to be the stuff of life ever after.

Evenings channelling Elizabeth Barrett Browning and counting the ways we love each other are displaced — particularly after the arrival of children — by the urgency of deciding whether to replace the boiler, or whose turn it is to make dinner.

All such things can bind us together, weaving tightly the tough fabric of a joined life. But the danger of life’s predictable routine — the groundhog day familiarity, talking to each other so matter-of-factly, being sparing with the compliments — lies in allowing the mind to wander.

Angela with her former lover Steve. She says there is no denying that her past lovers have made their presence felt in her marriage — at times, even making her question her commitment to husband Olly

It is at such times that women, far more than men, according to Susan Nolen-Hoeksema, Yale psychologist and author of Women Who Think Too Much, are likely to find themselves ruminating on how life might have panned out with the past loves that seemed so magical in their time. Whether the thrill might have endured and could have made for an infinitely more satisfying relationship than the one we have now.

These fantasies, as I see it, are like powerful ghosts, haunting the darker recesses of our psyche, ready to swoop in and cause trouble, when given the chance.

Certainly, my marriage became crowded with ghosts when, after my first decade with Olly, the little things that had once been no more than niggles began to take on greater significance. We seemed too often to slip on to opposite tracks.

Arguments and irritation replaced amiable conversation, sex felt more a chore than a delight, we were frequently exhausted by raising our two, growing sons while doing up a wreck of a house.

Sex felt more a chore than a delight with husband Olly, and they were exhausted by raising two, growing sons while doing up a wreck of a house

All that had been so loveable in our partnership — impromptu, romantic weekends away, gifts given on whim, hugs whenever we passed on the stairs, conversations peppered with unsolicited ‘I love yous’ — had been replaced by a tetchy co-existence.

After one explosive row — undoubtedly over something banal, such as Olly not paying sufficient attention to what I was saying — I took myself for a walk and, suddenly, it was as though Steve were there: tall and angular, his long legs striding beside mine, his deep, drawling voice asking me what was wrong, telling me how he desired me, his intelligent face breaking often into a broad smile.

Once the gates had opened, the happy memories came in a flood. I remembered the hurt, baby pigeon we had adopted in a shared fit of impulsive sentimentality, and nursed back to flight. The novels he presented me with, filled with flowers he had pressed himself.

The bottle of bubbles he had opened, apropos of nothing but his insistence that it was time to celebrate us. The evenings spent making passionate love. Memories so powerful, so tangible that I felt an urgent yearning to be back in that time and his arms.

Angela with her husband Olly, who she says was crowded out by memories of her past lovers

Why oh why had we let the relationship go? Surely the fault lines had not been so great.

I found myself ruminating on these memories over the days and weeks ahead, so that they became more real, more powerful, each time I conjured them up.

Steve began to appear whenever I was feeling low, sad or undervalued — he would have known how to soothe me, I felt, when to say sweet things that answered my longing to feel there was someone to lean on, to be looked after by. At other times, when Olly and I had been in a sexual drought zone, Ron would come sweeping in.

He had been one of those emotionally dangerous men of huge charisma and puckish good looks. He never quite felt reliable, yet gave me the intoxicating sense of inhabiting a private world, in which his penetrating, green eyes, teasing smile and utter absorption in our love were all that existed.

Although he ended our affair abruptly and cruelly — in a way that I now suspect would have been repeated in other ways in a real life together — tantalising memories pay no heed to reality.

Carefully edited by my subconscious, they left out his not-so-perfect moments and presented me only with the steamy ingredients that seemed to be missing from my marriage.

Such rumination seems a particularly female trait. Why do past loves not torment men in the same way?

Olly has ex-girlfriends he talks of warmly and with amusement, at times, but I have never felt he was inviting in the ghosts as I do — they are over and done with, locked away where they can do no harm.

Relate counsellor Christine Northam believes men are more inclined to take a practical approach to the past.

‘They can put emotional things into a box and shut down the lid so nothing seeps out into everyday life,’ she says. ‘Women can do a lot of good to their relationships by focusing on emotional needs and confronting them.

‘But letting fantasies take over can be a way of avoiding a problem that needs sorting out in a real- life marriage.

‘Are the fantasies telling you what is missing in your present relationship that you could work on?’

Once Angela stopped inviting her ghosts in, they seemed to fade away

My ghosts were undoubtedly crowding Olly out. Adrift in our marriage, getting on each other’s nerves, I knew I needed to do some hard thinking of a different kind.

Slipping so easily into ruminations and ‘what ifs’ was, I realised, a destructive self-indulgence, which stopped me making any real effort to unweave the unhappy web we were caught in. I was so focused on how Olly didn’t match up that I wasn’t wondering whether I did.

I had to grow up and let go, if our marriage was going to hold strong. I started by refusing to give the thoughts of Steve and Ron houseroom when they appeared, and forcing my mind to something else.

I laughed at Olly’s criticisms more often than I flared up; I made a point of talking about how I valued all we had built up together, what good friends we were, and how I relished spending shared time enjoying our grandchildren.

I was surprised by how well this worked, how once I stopped inviting my ghosts in, they seemed to fade away, and how Olly, in turn, became softer, less critical, more able to acknowledge how much we mattered to each other.

I am not sentimental enough to say we now have the perfect relationship, nor that Ron and Steve will never again get a nostalgic thought. There are times, even now, when I wish Olly were as artfully charming as Steve, or as dangerously seductive as Ron.

but, while I would not wish away my past, I no longer imagine that it holds anyone who would have been a better bet than my husband.

I can see clearly that his steady, reliable form of love, while not as demonstrative, has infinitely more value and will help us weather many more years together yet.

As we snuggle up in bed, our books on our knees, laughing at how cosy this feels after 40 shared years, I am glad that I allowed Olly’s substantial self to banish my ghosts.