In May 1974, Sarfraz Manzoor arrived from Pakistan with his mother and siblings to join his father in Luton, to build a new life. He reflects on the changes in his family – and in the world he grew up in – in the four decades since

How far we have come. Forty years ago this very week – 16 May 1974 – a plane that had set off from Lahore landed in London. Among the passengers was Rasool Bibi, a 41-year-old Pakistani woman, with her daughter, 12, and sons, aged 11 and two. It was their first plane journey and the first time any of them had left Pakistan.

I was the youngest of the children – I was almost three – and at the airport we waited nervously for our father to meet us and take us to our new home and our uncertain fate. In 1974, Harold Wilson was in Downing Street, Richard Nixon was vowing not to resign over Watergate and Abba had just won the Eurovision Song Contest with Waterloo.

My father, Mohammed Manzoor, had been in Britain for 11 years, visiting Pakistan only twice during that time. He had left my mother and his children in the hope of finding work in Britain that would enable him to send for us. When we joined him he was working on the production line at the Vauxhall car factory and that was how we came to live in Luton, in the Bury Park area.

I live in London now but return often to Luton. I recently visited Bury Park and stood on the rooftop of Nadeem Plaza, which offers one of the most striking views of the town.

Below me were streets lined with halal butchers and grocers, little changed from the days when I used to walk along them with my parents.

During the first few months, before we could afford secondhand beds, we slept on the floor or on two chairs turned to face each other. What might my parents have said had they predicted what would happen to them and us in the next four decades?

We didn't have a television set but the Saturday morning schedule for this weekend 40 years ago featured an animated series called Wait Till Your Father Gets Home in which "a conservative father butts heads with his family on various social attitudes of the day". That sounds like the story of my life. My father was an ambitious man – he had left his wife and children in Pakistan in the hope of a better life in Britain – but his ambition was tempered by reality. He could not imagine a world in which white people were friends and not just colleagues.

My family and home town did not equip me to talk to white people; my parents actively discouraged such interaction. My father and the other working class Pakistani men he fraternised with only knew work that was soul-destroying and low-paid. Work was endured not enjoyed; the same could also be said for marriage. I grew up in a world constrained by rigid certainties; life would follow a predetermined path and the route was largely determined by others.

Sarfraz as a boy with his sister Navelam, father Mohammed and mother Rasool Bibi.

Roads were closed because I was brown, because I was working class and because of my parents. It was against these limitations that I have spent the last 40 years resenting, raging, running and rebelling. My father came to Britain for economic reasons and I have struggled to process the unintended consequences of his decision.

As a little boy in the 70s, mine was an insular existence, enclosed within the Pakistani bubble of Bury Park and largely unaware that any other world existed. I was a teenager in the 80s, a decade of frustration as I realised I was different from my white friends who were allowed to have girlfriends and parties and free will. In the 90s, I left Luton and my family to study and live in Manchester only to be forced to return in the spring of 1995 for my father's death. I spent the next 10 years fearing an arranged marriage before finding the courage to reject it, and the fortune to find love.

This decade has been defined by marriage in 2010, and fatherhood the following year. When I first arrived in Britain my family consisted of my parents and their three children. In the years since there have been departures – the death of my father – and arrivals, and today the family includes my mother, her four children, their spouses and her seven grandchildren.

The family is bigger and broader: if someone had told my mother before she left Pakistan that two of her children would end up marrying white non-Pakistani partners, I doubt she would have believed it, or been able to conceive of it.

Sarfraz as a young man with his parents, Rasool Bibi and Mohammed.

Progress can be painful and one theme of the last 40 years is of absolute certainties being challenged and unravelled on the journey from resentment to forgiveness and acceptance, some further down the road than others. I was the first person in my family to marry someone who was not a Pakistani Muslim and I am now the only fully non-white person in my own family. My daughter, Laila, is mixed race but her skin tone owes a larger debt to me than my wife. The other day I was taking her clothes off in preparation for bath-time. She looked down at her body and said "I'm brown!" It was said with surprise and delight, an entirely innocent observation. I recall having the same thought when I was a teenager but it was freighted with foreboding. The difference says everything about how far we have come.

My father is buried in Luton and my mother lives less than a 10-minute drive from the very first house she lived in. When I was young I didn't want to spend time with my parents or their friends. It is one of the more surprising aspects about the passing of time to learn that not only do I love the things I once resented – I need them too. I need them because as the previous generation slips into the arms of history, my generation finds itself the keeper of memories, the teller of stories. It will be my job to tell Laila about her Pakistani grandad and grandma and what life was like way back then.

My parents and that first generation were, of course, pioneers but what I have only recently come to realise is that we are all pioneers. My parents were the bridge between Pakistan and Britain, the old world and the new. I am the bridge between the almost vanished world of my parents and the unfolding life of my daughter. We are all pioneers; and when freedom is not feared we are all bridges.

I look at Laila and I see distant echoes of my father and of me but I also see how different her world will be. Her history includes the history of myself and my parents but it is not dominated by it. There are fewer rigid certainties to constrain her future, she can believe that no road is closed to her.

This story began with my mother arriving at Heathrow airport and it ends, or perhaps begins again, with my daughter's birth in an east London hospital. She could only exist now. She is the past, the present and the future – a joyous hope-filled reminder of how far we have come.

Follow Sarfraz @sarfrazmanzoor