When I heard the knock on my bedroom door, I was in my 50-shades-of-purple room at Casa Grande in Burgau, Portugal – a quirky bed-and-breakfast run by the quirkier matriarch, Sally Vincent, who expatriated from England 40 years prior. I was moping, chin on elbows, gazing out of the wood-panelled window at the bougainvillea drooping overhead.

“Hi,” Sally exclaimed excitedly, her tone du jour. “You ready to go?”

Burgau sits on the southern tip of the country, the second-to-last southernmost town in the sparsely populated Algarve region. The village has a permanent population of about 450 that barely doubles with tourists in the summer months, and is built of mostly white, sea-washed buildings smooshed together on narrow cobbled streets with a handful of bars and restaurants.

You may also be interested in:

• The kind people Europe forgot

• A village that needs approval to enter

• Where afternoon tea really came from

Long before Sally found herself the unwitting ‘Queen of Casa Grande’, she was a burgeoning young actress in 1950s London. She worked as a directors’ assistant at the famous London film production centre Pinewood Studios in between acting gigs, and brought scripts and coffee to the likes of Elizabeth Taylor and Charlton Heston. And when she was 21, she married – as many young women of that time did.

John Vincent was a bohemian adventurer, and the two left London in a whirlwind romance. They lived in Malta for a bit, then bought a rusty Land Rover to travel on a cargo ship full of goats to Libya. From there, they drove through the desert to Morocco, took the ferry to Spain and journeyed up through France, eventually returning to England where John worked in organising conventions. Not long after, he got a job offer to do the same in Chicago, and the two settled in the Midwest for five years. Sally followed as the dutiful wife and wasn’t sure if her acting could continue, but it turned out that being a British actress in the US made her a hot commodity. Their first two children were born there, but by the time they were two and four, the Vincents were feeling the pull back to Europe.

“Portugal was one of the least developed countries in Europe at the time,” 74-year-old Sally explained to me over tea and biscuits in her formal sitting room decorated with Victorian wallpaper and furniture to match. “We felt like rich Americans and went in search of a castle.” They drove around for a year in a converted campervan, landing in Burgau and spending all their savings on their ‘castle’ – a rundown mansion.

“There was no water, no electricity and we spent a year fixing it up,” she said.

After a hefty renovation and camping inside, the 1912 house began to resemble its former self. With no intention of returning to England, the Vincents settled into their Casa Grande and began renting it out as a B&B. Sally also ran a restaurant out of its converted adega (an old wine storehouse).

I came to know about Sally through her eldest son; we sat next to each other on a short flight from Las Vegas to Los Angeles, and as I try to often do, I struck up a conversation with my seatmate. After the normal Q&A session began to wane and more pointed questions began to fall out of my curious mouth, he mentioned growing up in southern Portugal. I was so intrigued by the five-minute version of his upbringing that I asked if he’d put me in touch with his mother. I already had plans to be in the United Kingdom for three months over the following summer, and I knew I’d need a beach vacation. The serendipity of it was not lost on me.

On my third night there, as I sat moping in my window seat because of some solo-travel loneliness, Sally took me and two other guest couples – one from Berlin and the other from the Netherlands – to hear fado, the traditional music and dinner experience that Portugal is famous for. Sally, who often plays tour guide with her guests despite not being paid extra for it, kept the conversation going when language barriers presented themselves. Her laugh was raucous and her curly silver hair looked like a halo in the dim restaurant light. It was hard not to believe Sally when she was speaking, no matter how outrageous the tale; her expert storytelling slid easily in between sets of soulful fado and sizzling Portuguese meat and seafood dishes.

She told us that a decade after their move to Burgau, John left her and their now four children. He had been going back and forth to the US for work and “created another life with someone else,” as Sally said pointedly. He left them with no warning, no savings, no financial assistance whatsoever and no goodbye to the children.

“By this time, the children had many friends and a good community here,” said Sally, who’s been the manor's matriarch ever since, “So they didn’t know the agony and the struggle of it – running a restaurant, running the guesthouse and bringing up four children.”

Casa Grande holds a maximum of 16 guests at a time, and it’s generally full between May and September every year. Each morning, Sally holds court around the 150-year-old dining-room table spattered with un-matching teacup sets and fresh fruits, meats and cheeses set out by her housekeeper, the 40-something Felicidad, who’s been working for Sally for more than 25 years. Guests hail from the US, the Netherlands, Germany, Spain, South Africa and beyond. And while breakfast isn't mandatory, it may as well be since you'll get a half-joking finger wag and a shake of the head if you're not in attendance. And I should know; Sally – who’s got seven granddaughters ranging from four months to 29 years, one of whom she said I reminded her of – gleefully tsk-tsked me the morning after the night of fado.

After dinner, my loneliness had led me to one of the three bars in town. Not surprisingly, Sally encouraged her guests to socialise – to party, if you will. I ordered my new favourite drink vinho verde (Portuguese white sparkling wine) as two young British men sat to my left and slapped down cards of a game I didn’t know. The taller of the two invited me to join, and then let me tag along on their three-bar bar-crawl. Over the next four days, as our time in Burgau overlapped, our one night grew into a solid holiday dalliance (and more than a year later we’re in love, happy partners on a new adventure living together in London).

Over that morning’s breakfast – where my hangover throbbed and my loneliness subsided – she regaled her guests with Casa Grande's ghost stories, guest gallivants and even buried treasure. This is a time-honoured tradition where she delights in telling of her A-list patrons, like the Queen’s beautician who, before her passing, summered there for 20 years during her annual two weeks off from royal duty. The billionaire hotelier Vikram Oberoi spent a month there with his family a few years back and never let on who he was or what he did; Sally suggested he come work for her because of his handy skills around the house. Billy Idol and his family summered there a couple of years ago, and his 86-year-old mother continues to visit. She’s been teaching Sally how to use Facebook.

Sally has no intention of stopping work at Casa Grande anytime soon, despite her age, but does admit that she’d like more help to run the business and perhaps open the restaurant again, which closed four years ago. For the time being, she brings together small groups of strangers who otherwise would never have met, much less shared stories of babies to come, old crushes to get over and other tales of loss and love – and new love, as it was with me – that flow easily around the breakfast table here.

“I went to a concert in Nottingham [about 10 years after John left us],” Sally told me as conversation wound down in the living room. “It was The Irish Tenors and they sang songs that he used to play and sing. It brought back all these amazing memories of the good times we'd had, because I'd just blocked them out.”

Spurred on by the memories and the music, not long after the concert Sally wrote to John and thanked him for the adventures and for their children.

It’s those magic moments that make the harder times worth it

“Apparently he cried,” Sally said, “And he called the children. He talked to me and it was just wonderful. It’s the music that did it. He’s passed away now, but I'm so glad that we made up and that I realised how wonderful our lives were and that [he brought us] to a brilliant place to bring up the children. If we had followed him when he left, then it would not have been as it was [and we] did really well.”

As Sally told me over FaceTime when we reconnected for this story from my flat in London with my partner, it’s those magic moments that make the harder times worth it. “Those magic moments,” she repeated, eyes twinkling.

Join over three million BBC Travel fans by liking us on Facebook, or follow us on Twitter and Instagram.

If you liked this story, sign up for the weekly bbc.com features newsletter called "If You Only Read 6 Things This Week". A handpicked selection of stories from BBC Future, Earth, Culture, Capital and Travel, delivered to your inbox every Friday.