Born into almost unimaginable wealth and privilege, Lady Edwina Grosvenor is not someone you might expect to bump into on a prison landing. But over the past 12 years, the second daughter of Britain's richest landowner (the Duke of Westminster, 7th on Britain's Rich List with a fortune of around £7.3bn) has been quietly establishing a role for herself as a discreet but potent champion of prison reform. Her work, which began when she spent part of her gap year working in Kathmandu Central prison in Nepal when she was 18, has taken her into more than 50 prisons in the UK and abroad. Aged 22, she commissioned research into the multiple needs of ex-offenders. Since then she has sat in on prisoner adjudications, been involved in restorative justice sessions and spent a year as a support worker in the notorious Styal women's prison in Cheshire. As impressive as her prison-related activities are however, the big question for me is: why?

We meet for lunch in the Clink restaurant, the fine dining eatery at High Down prison in Surrey, which provides high-level catering training for prisoners. Grosvenor is a trustee of the Clink charity and has invested several hundred thousand pounds in the start up costs. This month saw the launch of another Clink restaurant, in HMP Cardiff, which unlike the Clink in High Down is open to the public. If all goes to plan, the concept will be rolled out in up to a dozen prisons across the country.

I ask her, does she know who Lady Bountiful is? Her blue eyes flash defensively and then she lets out a little laugh. "Yeah," she says. So is she Lady Bountiful? "That depends on people's views, really. My view is that I cannot escape the fact that I am a wealthy person and the daughter of a duke and duchess, because I am, and I'm not apologetic about that. You can be embarrassed about it or you can forget about it or you can be proud of it and I've done all those things. I'm now 30 and I feel more at peace with myself and I'm more confident about what I'm doing. At the end of the day I think people should be judged on what they do and how they do it."

And why they do it? "Yes, and why they do it." So why does she do it, this prison reform thing? "From a young age closed institutions fascinated me and things I didn't know about I wanted to know about."

She recounts her father taking her in her early teens to a drug rehabilitation centre in Liverpool where she met recovering addicts. Later her mother took her to a charity supporting troubled families. "It was something darker than I was used to," she says. "I remember feeling quite outraged that nobody at school had taught me anything about this other way of life. I wanted to know more."

Edwina Grosvenor aged two with her parents, the Duke and Duchess of Westminster, and her older sister Tamara. Photograph: Alpha Photo Press Agency Ltd.

School was Mostyn House, a modest, now defunct independent co-ed school in the Wirral. Her father went to Harrow, "and hated it", she says.

She and her three siblings all went to Mostyn House because her father didn't want them to suffer at boarding school as he had. Was she a good student? She laughs again. "I did better than my dad [who left Harrow with just one O level]. But at Mostyn we got so much more than just an academic experience – we had good mates, we were given shit and we gave shit back." Shit because she was a lady?

"Having a title there was definitely a novelty but the grief we got from other kids was nothing compared to what we got from some of the teachers. I remember one saying to me: 'Edwina, if I kidnapped you, how much ransom would your father pay?' That struck me as a very odd thing for a teacher to say."

Grosvenor was a goddaughter of Princess Diana and her father is a godfather to Prince William. But the family motto, virtus non stemma – it's what you do, not who you are connected to, that counts – seems to have had a particular significance for her. "It's really from my early teens that I remember thinking properly about these issues," she says. "When I learned about the destructive dangers of drugs, I remember thinking, 'how come that person has taken drugs and I haven't?' I became fascinated by the idea of people being born into different families and the way that that can determine their life journeys. After working in the prison in Nepal I knew I had found what I wanted to do."

Studying sociology and criminology at Northumbria University, she says, gave her little real insight: "I didn't learn much about criminals and learned nothing at all about prison. What I did learn was that academics like to put things into boxes." Her dissertation was on children reared in prison. "I think a lot of people are unaware that we do have babies in our prison system. There aren't very many mother and baby units. The differences in the west compared to the east are huge, but the question is the same – should we have babies in prison?"

She says she didn't really know the answer until she had a child of her own. In 2010 she quietly married television historian Dan Snow and last year their daughter Zia was born. "When you go into a mother and baby unit they are quite pleasant places, staff don't wear uniforms and cells are more like bedrooms. But having now got a baby daughter the whole process fills me with horror. I find it quite difficult. I almost don't want to go there, thinking about what it must be like to go through that, no matter what you have done."

She volunteered in Styal prison when it had one of the worst records for prisoner suicides in the country. "It was grim sometimes," she says, "but I learned so much." Later she worked in Garth category B adult male prison, helping with the Restorative Justice programme. "A lot of people think restorative justice is soft, but there was nothing soft about what I saw – the prisoners were terrified and humbled when they had to face members of the public who had been victims of crime."

One of the kindest, gentlest women I ever came across while serving my own prison sentence was raped by a prisoner in the prison chapel. Another female prison worker I knew was taken hostage and held captive for three days. Has Grosvenor ever felt in any danger in a prison? How does Snow feel about her going on to wings and landings? "In a way, naivety is bliss," she says. "I've read and I've heard stuff, but Dan is pretty relaxed about it. My parents and Dan are more proud of the fact that I'm actually doing something that I love doing and that means something to me. I love going to work – and they just see that. I've always got one eye on the possibility of things going wrong. One day something awful might happen to me, but if it ever happens I'll deal with it."

Wouldn't it be easier just to sit back and enjoy the wealthy lifestyle? Why persevere with the thanklessness that abounds in prison reform work? "If a family member of mine was attacked, if something awful happened to them, I'd want that offender to go inside, but I'd want him or her to come out less likely to hurt anyone else. A lot of people think you're investing your money in prison, in bad people. I get asked: 'Why aren't you helping sick children?' A lot of people talk about prison and prisoners without ever mentioning the victims. Anyone who helps an offender is doing it because they want to reduce the number of future victims."

Her most ambitious investment project to date is Pathways, a £10m community regeneration programme organised around St Anselm's church in Kennington, south London. The plan is to create a variety of sustainable businesses run by ex-offenders, including an 80-cover bistro and restaurant that will run along the same lines as the Clink. Grosvenor has spent half a million pounds developing the project, the plans of which were approved by the council earlier this year. "It's based around the church but it's not faith-based," she says. "Pathways really is about a community working together to provide employment, skills training and accommodation, not just for ex-offenders but also for any local young people who need that support."

What does she hope her work will achieve? Does she have an end goal? "I never came into this work with an agenda," she says. "It has just crept up on me, and for better or worse it has consumed me. Like anyone who has a job I have some really good days and I have some really shit days when I want everyone to get lost. But I'm never put off. In 20 or 30 years' time I want to be able to look back and say I did something worthwhile – I made a difference."