Trico had left home at 17, taking the first ride he found on a fertiliser truck to Manzanillo. He had been on the road ever since and had an encyclopaedic knowledge of the little-known corners of Mexico. Planning a journey between two obscure villages perhaps 1,000km away, he could list by heart not only the highways and byways that would get us there, but the truck stops where we were guaranteed to find a long-distance lift, and the comedores where the blue maize quesadillas or the spiced pozole stew had received his particular seal of approval. Frequently, the señoras who bustled around these outposts remembered Trico as well as he remembered them, and would refuse to take our coins when we left. It was Trico's business to be memorable, and he did it with great aplomb.

The first time I met Trico, he was riding a unicycle along the top of a wall. This wasn't unusual for Trico; he saw the entire world as a complicated arrangement of surfaces to ride unicycles on. But at the time I didn't know that, and stopped walking for a few moments to watch him.

The wall ran along the edge of a small plaza in the centre of Creel – a village in the north-west sierra in the state of Chihuahua. It's an area where forested hills are carved up by jagged, red-walled canyons, and the sleepy alpine tranquillity is periodically shattered by incidents of gang violence. A massacre the summer before had cleared most tourists from the streets, and Trico performed more out of habit than enterprise, while his travel companions made hair braids for the local teenagers. Catching me watching, they called me over and offered me a swig of their beer.

They were malabaristas – itinerant circus performers who wander the streets of Mexico, hitchhiking from town to town and surviving on whatever donations they receive for their impromptu shows. I had become used to seeing them at traffic intersections across the country: jester-like figures who would skip out in front of the traffic as it waited at red lights, give a brief display of juggling or fire-spinning, then collect any coins the drivers offered them as the cars moved off on either side.

Cat Rainsford and jugglers in Mexico. Photograph: Sophie Pinchetti

I had read about them as well, though never more than a couple of sentences in the middle of some rumination on Mexican poverty. Malabaristas were portrayed as another symbol of desperation in a country where almost half the urban population worked on the black market and men joined the drug gangs because they saw no other way to make a decent living. Just impoverished kids trying to scrape a living off streets that were already overworked.

At first, their stories seemed to confirm this impression. Several hailed from the crime-torn border town of Ciudad Juárez, then notorious for being the most violent city in the world. They had witnessed assassinations, even lost parents. But when I probed them further, I realised they saw their lifestyle not as one of desperation, but of defiance.

"The things they learn in their travels, they take back to Juárez," Trico told me. "It's not only about the money. It's about showing that you don't have to live in fear. That you can still make people smile, even in a place like Juárez."

Their energy and conviction intrigued me. When travelling for long periods, it can be all too easy to fall in with other foreign travellers and stumble around in a closed group, insulating each other with a shared familiarity. You can strike out alone and try to figure out a country piece by piece, but you always feel like a stranger and feel increasingly disjointed yourself in the process.

I had done plenty of both over the previous months of travelling in Mexico. Trico and his friends were something different: escapists too, in a way, but also idealists, on a mission to explore the beautiful side of their troubled country and to give something back in the only way they knew. More than anything, it was this attitude that made me go with them.

After the chance meeting on the plaza, they invited me to spend a day or two with them. We hitched a ride in a battered pick-up truck to where the track to the hot springs of Recowata forked off the main road. Then we carried our packs on foot through the pine forest, emerging on the lip of a canyon from where a steep track zigzagged down to a cluster of turquoise volcanic pools cradled in the rock below. It was dark by the time we set up camp at the edge of the pools. Fireflies flashed among the trees. Up to his neck in the steaming water, Trico grinned up at the sweep of starlit sky framed between the cliffs.

"Ay, que rico", he announced. "No tenemos ni un peso, pero somos milionarios." (We don't have a single peso, but we are millionaires.) It was that sentence that turned a chance meeting into a journey that was to last me almost two years.

The original group soon fragmented, but Trico and I remained together, and new groups continually formed around us as we travelled down from the sierra and across the plains of northern Mexico. The sense of solidarity among the malabarista community was strong, and every group shared all food and possessions communally.

Over his years on the road Trico had become fully integrated into this network, and was acquainted with all the communal houses and squatted buildings where we could find a place to sleep for the night. For a while we performed outside restaurants along the coast of Nayarit, sleeping on beaches under trees crawling with giant iguanas. When business was good, we'd treat ourselves to nights in guesthouses. When it wasn't, we'd sleep in tents, shop doorways or plazas.

Any reluctance on my part was met with the admonishment not to be a fresa. Literally meaning "strawberry", the word "fresa" is used in Mexican slang to denote anyone spoiled or soft. Of all the wide and imaginative range of Mexican insults this, for them, was the worst. It was acceptable to be a large goat (cabrón) or even, on occasion, a pubic hair (pendejo), but to be a strawberry was unforgiveable.

As the months went on, I became integrated. I learned to fire dance and juggle. I learned the etiquette of the traffic intersections, which dictated that a performer must defer to the windscreen washers, though not to the peanut or sweet sellers. In villages too small for traffic intersections we would perform in the markets, juggling among bleeding slabs of meat and great hessian sacks of dried chillies, and the stallholders would laugh and throw us vegetables or blocks of cheese.

I crossed the country more times than I can remember, following a route around northern Mexico that was more of a scribble than a line. Northern Mexico tends to be a blank space on the tourist map, even other Mexicans tend to view it rather like the wild west, despite it being home to the country's richest industrial areas.

"Un pueblo," sniffed one of my more urbane friends in Mexico City when I spoke to him of Trico's home town of San Luis Potosí, which has a population of a million. A village.

Yet the region is much more diverse – geographically and culturally – than many give it credit for. The pine-fringed canyons of the Sierra Madre Occidentale are the homelands of the native Tarahumara people, famed for their long-distance running abilities. In the high-altitude plains and mountains of the eastern Sierra (the Sierra Orientale), the old silver-mining regions of Jalisco, Guanajuato and San Luis Potosí boast some of the country's finest colonial architecture, as well as some hauntingly beautiful ghost towns.

Real de Catorce, San Luis Potosi, Mexico. Photograph: Radius Images/Corbis

Real de Catorce is perhaps the most famous – a clutter of stone buildings and cobbled streets, it was semi-abandoned over a century ago when a crash in silver prices precipitated the closure of the local mines, but now has a few guesthouses, craft shops and cafes. Below lies the desert, austerely majestic in its vastness, and dotted with villages where men wear ten-gallon hats and tether horses to lampposts next to the pick-up trucks.

These enigmatic landscapes attract enigmatic people. Mexico was a hub for artists of the surrealist movement, including the English poet Edward James, who chose the village of Xilitla in La Huasteca Potosina or his home. Here, in the beautiful southern hinterland of the state of San Luis Potosi, the mountains and high-altitude desert that cover the north of the state give way to verdant subtropical forest, veined with cave systems and waterfalls, as the land plunges down towards the sea.

The castle embraces and is embraced by the surrounding jungle, its forms mirroring those of the gnarled trees that grow through it. Little waterfalls run through a bewildering tangle of arches, alcoves, and staircases that spiral to the sky without leading anywhere. I lay on a flat rock carved with an imprint in the exact shape of Edward James and gazed up, as he must once have done, at the sunlight filtering through the trees.

"They held the Mexican national circus convention in Xilitla last year," Trico told me. "You can imagine what this place looked like with unicyclists swarming all over it."Living hand-to-mouth, on the fringes of society, we frequently met people who had slipped through the cracks. Honguito, for instance, was a dumpy little man in a top hat and tailcoat who worked in a travelling fairground and spent his free time sitting in a storage tent to remind himself what darkness looked like. Hermano Sol wandered barefoot around the edge of the desert, talking to plants and preaching to strangers with the aura of a crazed prophet.

Even those with more conventional occupations often cherished intriguing beliefs, which we discovered through the stories of those we hitched rides with. On one journey, we were picked up on the edge of the remote patch of desert known as the Silent Zone by a young truck driver named Eduardo. It was dusk: the last warmth of sun was lingering in the ground like the embers of a fire, turning the earth red and cacti into barbed silhouettes.

Eduardo eyed us sceptically. "They always come at night," he sighed. "About this time, about this place. Bloody silent zone is full of ghosts."

The phenomenon didn't strike him as particularly strange, although he confessed to finding it disconcerting when his passengers disappeared with the first rays of the morning sun. On another trip, the earnest accountant who picked us up turned out to have a second identity as a self-mutilating moon-dancer at the pyramids of Teotihuacán, near Mexico City.

Mexico is a mestizo nation of mingled Spanish and indigenous blood, and its customs are similarly eclectic. In food, language and festivities a pre-Hispanic influence is continually bubbling to the surface. The malabaristas had a particular fascination with this suppressed half of their heritage, drawing inspiration from the myths and art of Mexico's surviving indigenous communities. We frequently made trips to Wirikuta, the sacred desert of the Huichol people, in the eastern sierra south of Real, high up in the central mountains and honeycombed with ancient footpaths leading up into stony ruins. Wirikuta looks like the seabed it once was, the hard-baked ground scattered with cacti like sea urchins, and carpeted with barbed gobernadora bushes that from a distance look deceptively like fronds of seaweed.

We would carry water and provisions, and camp in the desert for as long as they lasted. By day, we would trek towards the few clumps of mountains that stand out from the flatness of the horizon as if carelessly dropped there, every ridge and valley highlighted by the play of sunlight and shadow. By night, we would cook stews over an open fire, listening to the coyotes bickering in the darkness, and watch for shooting stars.

Despite their ancestral lands being encroached on by mining concessions and urbanisation, the Huichol have so far managed to preserve their traditional way of life almost intact. Every year they make a pilgrimage 500km across the desert to the mountain of El Quemado – the birthplace of the sun. Here they make offerings to the gods of the corn and the deer, and gather the hallucinogenic peyote cactus, whose ritual consumption is a cornerstone of their culture. We would come across them on the edge of villages, weaving intricate jewellery out of constellations of tiny coloured beads.

One Easter, in compliance with a spiritual bargain Trico had made with a local Catholic saint, we joined his family on their annual pilgrimage – a 200km walk from San Luis Potosí to the cathedral town of San Juan de los Lagos. Families from across the region had come together for the five-day journey, sleeping and cooking in the back of trucks driven in procession by the men whose greater age and distinction spared them the obligation of walking.

We caught up with Trico's family at a village about 40km along the route, where they were already settling down for the evening meal. Brothers, sisters, uncles, aunts, cousins, nieces and nephews were clustered cross-legged under a tarpaulin stretched from an enormous yellow truck to the top of an adjacent wall – plates on laps, elbows in each other's beans. Trico and I fought our way in and surrendered meekly to hugs, kisses, teasing and scolding. As the sun set the clamour died away, pans were washed and beds arranged. We slept in the yellow truck, the whole family packed in like cigarettes across a floor padded with foam mats.

The day began at 4am as we tramped blearily out of the village, hugging our arms over our chests against the chill early morning air. But as the sky brightened, the spirit of the walkers lifted, and the silent dawn trudge broke into gossip and laughter. As our procession snaked over the hills, to our right the desert swept out to a skyline patrolled by purple mountains. Every few hours we would come to a rest stop, where we massaged the knots out of each other's feet, and the women stripped down the surrounding cacti with long knives, gouging out the spines and grilling the flesh to serve for lunch. If not too tired, we would gossip and drink beer, or the fermented cactus juice known as pulque.

Finally, the clutter of buildings that was San Juan became visible on the horizon. And I looked before and behind us at the river of humanity plodding its way over the hills, aching and exhausted but together, and resolute in their faith that the chaparrita (Virgin Mary) would never abandon them.

Northern Mexico is, of course, a region in profound crisis. Fear lurks beneath the surface of everyday life, and few have escaped contact with the carnage of the drug wars. I was touched by it, more than once. Yet after two years in a society torn apart by greed, my clearest memories are of its effusive generosity. Travelling light, I collected few souvenirs. But one that I do keep, tucked in a notebook, is a single 100-peso note. In one corner, printed in letters almost too small to see, is a poem credited to Netzahualcoyotl, the pre-Hispanic philosopher-king.

I love the song of the mockingbird

Bird of four hundred voices

I love the colour of the jade-stone

And the intoxicating scent of flowers

But most of all I love my brother, man.