In the early 1970s, I spent five years going over to Ireland to photograph Travellers. It was a difficult time for them: they were victims of prejudice and struggling to maintain their nomadic lifestyle as the Irish government did its best to assimilate them into the settled community.

This image, taken in 1973, has always had special meaning for me. I had returned to Galway and was living in a worn-out minivan, trying to track down families I'd photographed on a previous trip. This was difficult as the authorities were continually moving them on. In general, Travellers didn't want people going into their sites, so it took time to get their trust. Once I got past that hurdle, I took my first pictures, but they no longer seemed relevant: they were stereotypical shots, full of dirt and grot.

At the time, most Travellers could neither read nor write, but they had developed an elaborate messaging system – word of mouth, objects left in places – to keep track of each other's whereabouts. I found the members of one large family camping up a narrow lane. It was cluttered with caravans, chickens, children and women. I chatted to them before the men and horses finally came back from town, where they'd been selling scrap. They were immediately mobbed by kids.

Sometimes, elements for a shot just fall into place. This man was fairly unaware of being photographed and didn't pose – yet with the baby in one hand and the horse held by the other, he seems to have his whole life in his hands. There is something almost religious about the image. Perhaps it's the composition: the trinity of figures and the baby's dummy dangling like a crucifix. His expression of calm strength and pride is a reminder of how tough life was for Travellers.

I was recently contacted by members of his family and now hope to catch up with them, 40 years on. With this shot and the rest of my project, I hope to counter the crass stereotypes about Travellers, which are still peddled today.

CV

Born: New York, 1947.

Studied: Guildford School of Photography.

Influences: W Eugene Smith and Dorothea Lange

High point: "My work allowing me to follow my curiosity and to learn more about other ways of living."

Low point: "Being held at gunpoint in Paris while taking a photograph."

Top tip: "The camera is only a tool – use it to develop and communicate your vision."

• Irish Tinkers, A Portrait of Irish Travellers in the 1970s by Janine Wiedel is available as an iBook. For more information visit: irishtinkers.com