Rare is the family whose fortunes run smooth forever; it is our misfortunes that test the bonds of blood and affection. “All happy families are alike,” wrote Tolstoy in Anna Karenina, “each unhappy family is unhappy in its own way.”

Over the past few years, I have met enough unhappy families to know that Tolstoy was right. The stories of their troubles fascinate us, in part because we crave assurance that our own are not unique.

In September 2009, I wrote on this same page of Guardian Family about Philomena Lee, an Irish woman who decades earlier had been forced by the Catholic church to give up her son for adoption. In the weeks that followed that article, I received scores of letters from Guardian readers with tales of similar tragedies in their family.

Most of them were from Irish and British women, who had given birth outside of marriage and had had their children taken from them, or from the children themselves who were looking for their lost mother. But there were messages, too, from families in the British Pakistani community, and their experiences were much more violent.

One woman from Yorkshire, whom I met through an intermediary, told me of her brother’s arranged marriage with a relative from Pakistan. The relationship had gone wrong and their father had taken the girl back to her native village and dumped her there. Her family had been dishonoured, so when the father next came to Pakistan the girl’s cousin murdered him.

The story opened my eyes to a fascinatingly unfamiliar world. A whole society seemed to have pigeonholed the raw emotions of family disharmony into a codified system of transgressions, each with its own level of appointed retribution.

I was riveted by the drama of it. But for several reasons, not least the absence of any obvious hero in the tale, I decided this was not a case I could devote my life to. The attraction of Philomena Lee’s story had been that a woman whom life had treated abominably – her family torn apart by prejudice and social hypocrisy – had responded with a nobility and generosity of soul that seemed the essence of human goodness.

I wanted desperately to find a similarly inspiring story in Pakistan, and when I met Ayesha Rahman I knew I had found it.

Ayesha was a second-generation British Pakistani woman; and she too had been through a devastating family tragedy. Her father, Ibrahim, had come to Britain from Karachi at the age of 16, settling in Burnley, Lancashire, where he found work in a textile mill. In Ayesha’s telling, he had been the perfect parent. Her eyes welled with tears as she spoke about his kindness to her as a child; about the toys and presents he had showered on her; and about his refusal to accept traditional prejudices that disadvantage girls in favour of male children.

Facebook Twitter Pinterest Judi Dench as Philomena Lee, centre, and Steve Coogan as Martin Sixsmith in the film Philomena. Photograph: Allstar/BBC Films/Sportsphoto Ltd.

Ibrahim ensured that Ayesha had a good education, and she rewarded him with outstanding academic results. She sailed through her A-levels and won a scholarship to Cambridge. For the next few years, the Rahman family seemed the archetypal immigrant success story.

Ayesha graduated, moved to London and set up her own IT consultancy with a group of friends from university. She made money and began to move in social circles her parents would never have dreamed possible.

To her regret, she felt she was losing contact with her roots. Families are close in the Pakistani community; clan cohesion a defence against outside hostility and an important factor in resolving differences and administering justice. Ayesha was caught in a no man’s land between the two societies she was heir to.

When I met her, four years ago, she was reeling from a blow that had thrown her life into turmoil. Her father, on a trip from Britain to Pakistan, had been murdered in his native village close to Karachi. Ayesha was racked by regrets. She had been away from Burnley so long, she said, that she had not even known Ibrahim was in Pakistan. She felt she had let him down.

Driven by her unwarranted sense of guilt, and by a daughter’s love for a cherished father, she was determined to discover what had happened. She had read my book about Philomena; she wanted me to help her track down the killers and then write about what we found.

I liked Ayesha. Her story was full of bad things but it had good characters in it. Like Philomena, she was a strong woman bent on righting a wrong. The background to the tale was dark, but the love and decency at its heart gave it a radiance that appealed to me. I agreed to help her with the investigation she was embarking on. I agreed to go with her to Pakistan. And I agreed to write her story.

In the fertile farmlands and dangerous shanty towns around Karachi, we tracked her father’s last days. We spoke to police who had been bribed by violent men to prevent us uncovering the truth. We met boastful criminals and corrupt politicians. And we heard stories about Ibrahim that were hard to reconcile with the picture of the loving father Ayesha had painted. A senior police inspector and a powerful mafia boss both claimed he had been mixed up in criminal activities involving drug smuggling, perhaps even people trafficking, and murder. When we challenged their claims, we were told to back off and threatened with reprisals if we didn’t.

The shared danger should have brought Ayesha and me together. But family dramas are supercharged with emotion. For an outsider, there is a minefield of sensitivities to be traversed; and in my relationship with Ayesha, I trod on a few mines.

We were looking for more than just the identity of Ibrahim’s killers. We wanted to know why he was killed. And we wanted to know who he was – honest family man, or someone with dangerous secrets hidden from the world? On that depended Ayesha’s sense of herself. She had grown up with an image of her father that had shaped her perception of the world. To discover now that it had been a lie would have been a heavy blow. Her father’s identity was at stake – and so was hers.

Martin Sixsmith: 'Scared by spin doctors? I've been in war zones' Read more

Differences arose between us. In retrospect not surprisingly, although I was at first surprised, she was determined that her father should be portrayed as a saint. I wanted to portray him as a man, and – as seemed to be the case – a man with many flaws. She wanted an idealised Ibrahim; I wanted him warts and all, even if it meant unmasking a life of evil deeds.

We argued. And as we argued, a second tragedy happened, this time to me. A message reached me at our Karachi hotel that my brother had killed himself. I was shaken. I dropped what I was doing and flew back to England at once. In the weeks that followed, I was beset by the unanswerable questions that suicide unleashes – why did he do it? Why didn’t I spot the extent of the trouble he was in? Why hadn’t I done more to help him? And how could I have been so selfish to fly to the other side of the world when my only brother needed me with him at the hardest moment of his life? I could think of nothing but the sorrow, regret and guilt that filled every moment of my day and night.

It was in those terrible weeks that something remarkable happened. Instead of reproaching me for abandoning her, Ayesha extended the hand of sympathy. She showed me a compassion that I had never shown to her. Just as she had come to doubt how well she had known her father, I was questioning if I had ever really known my brother, and why I hadn’t recognised the terrible agony that would drive him to kill himself.

Ayesha and I were now consumed by an obsessive search for the truth about someone we had loved, and that shared understanding brought us together. I went back to Pakistan. I resumed the investigation. And I discovered the identity of Ibrahim’s murderer and what sort of man he had been. I answered Ayesha’s questions about her father, but I am not sure I have answered those in my own life.

We can’t choose what the world throws at us, but we can choose how we respond to it. Both Philomena and Ayesha have responded to the reverses in their lives with extraordinary fortitude. Both stories had begun from the premise that I would be the one to help them, but with each of these two remarkable women, it is I who have emerged the richer for having known them.

Names and places have been changed

• Ayesha’s Gift: A Daughter’s Search for the Truth About Her Father by Martin Sixsmith (Simon & Schuster, £18.99). To order a copy for £14.99, saving more than 20%, go to bookshop.theguardian.com or call 0330 333 6846. Free UK p&p over £10, online orders only. Phone orders min. p&p of £1.99