On a mountaintop 2,250 metres above sea level in Pshavi, in the Caucasus Mountains of Georgia, Ioseb Kochlishvili is performing a pagan ritual. Eighty-nine years old, and dressed in a battered suit jacket and corduroys, this is the first year he hasn't walked all the way up here to Iremtkalo, the "meadow of the deer". This time he came on horseback instead.

He leans on his walking stick and steps over the wall into the hallowed ground where only he is pure enough to go. He approaches a bell tower, a stone structure of about five metres, and rings the bell to tell the riders to get ready.

This year, only three people are taking part in the horse race, but it's still the highlight of the day. The three young men, tanked up on sacred beer ("It's technically against the rules to be sober," one of them tells me), gallop off to a semi-ruined shrine almost a mile away. There, they wait for the signal, and the hundred or so parishioners – vassals of Kopala, the mountain deity – assemble to get a view.Making incantations under his breath, Kochlishvili rings the bell three times and then clangs it like a fire alarm. On the third peal the riders take off along the grassy ridge towards the shrine. Riding bareback and barefoot, the race takes only a minute.

A villager tackles a vessel of holy beer. Drinking and toasting are all part of the ceremony (Alexander Bainbridge)

Raphael Kochlishvili, Ioseb's 22-year-old grandson, is the victor. He rides anticlockwise around the shrine three times, as ritual demands, and stands fully upright on the horse's back to receive congratulations. Leaping off like a trick rider, he proceeds to share the rest of the sacred beer, a flat, yeasty, yellow home brew, with his friends. Offering some to our party, a foreign journalist, a linguist and a photographer, we toast to his victory and to the glory of Kopala, and down the beer in as few gulps as possible.

Kochlishvili senior has been presiding over the ritual at Iremtkalo every summer since 1946. He is the khevisberi, or shrine-priest, more literally, "wise man of the valley". As khevisberi, he has a special, supernatural bond with Kopala, a primordial demon-killing hero, revered by the people of the region as a "son of God". Every July and August, Kochlishvili and the other khevisberis officiate in a ritual cycle of animal sacrifice, beer drinking and horse racing in forest groves and mountain pastures, maintaining an unbroken link with an almost forgotten past.

"The horse race is a very old law," Kochlishvili says. "I asked many people how old this custom was but nobody could give me an answer. Everybody says 'when I was 10 years old, I remember the horse race' and so on." In spite of only having three riders this year, he explains, the race must take place: it is a part of the ritual. "It is compulsory to hold it. Even if there are just two horses, you have to have the race."

In the pagan religion of Pshavi, following the rules is everything. Khevisberis like Kochlishvili are there to get the ritual right, not to theorise on its origins.

Shrine priest Ioseb Kochlishvili said he was visited in a dream and called to serve (Alexander Bainbridge)

"They do not have a personal knowledge like that," explains Dato Akriani, one of the tiny number of people who have moved from the lowlands up to Pshavi, and who was initiated into the cult of Kopala 20 years ago. "They are the true inheritors and passers-on of the tradition, but they cannot explain it metaphysically. They cannot tell you why they are doing this or that and what it means. They cannot touch bears or wolves, touch chicken or eggs, or touch a woman when she has her period, but if you ask them why, they don't know. It's supernatural, it's a mystery."

A thousand metres down the mountain, in the village of Shuapkho, another khevisberi honours a mountain deity in a different set of ritual mysteries. The biggest village in upland Pshavi, Shuapkho is home to about 50 people, but it's most famous resident is Iakhsar, friend to Kopala, son of God, and slayer of ogres.

Iakhsar's shrine is in a grove of oaks on a slope above the village. There are low walls and walnut trees beyond which pregnant or menstruating women are not permitted, followed by a disused Christian church built by the "Society to Spread Orthodoxy to the Caucasus" in the 19th century.

Near the top of the grove is the killing ground, sticky with the blood of victims. To the left, half a dozen sheep are being butchered, their carcasses hung from special iron racks set in the ground. To the right, men are lighting candles and offering alcoholic drinks on an altar built of granite slabs that seem to have grown out of the earth. In front, a bull calf is being led up for slaughter. Worth several hundred dollars, this is a huge offering. Pshavi is a deeply impoverished region – whoever is offering the bull calf is a very rich man.

Villagers gather to dance on the hillside (Alexander Bainbridge)

Our party has brought a sheep. It's not a bull, but it is a significant sacrifice nonetheless. In the shade of the oaks, Lazare Elizbarashvili, the khevisberi of Iakhsar, makes incantations over the victim. Holding beeswax candles and chanting a spell that must be intoned with precision, he blesses our offering and invokes the deity. Iveri, our host at the festival, and Mindia, his family's friend and helper, lead the sheep away to be killed with the sacred dagger. It's all over in seconds, and we toast the memory of Iakhsar.

A short while later Iveri's sister, the formidable Maia Tselauri, is preparing to boil the mutton for us to eat. Spreading a plastic tablecloth on the ground by the church, the blood sacrifice suddenly feels more like a school fete. As a woman, Maia is not allowed to approach the sacrificial altar, but as a year-round resident, school teacher and local government worker, she's one of the most important people in the village, and is happy to explain the festival to us.

"We come here to thank Iakhsar. He killed the ogres; he made the valley safe for humans. He fought against evil and we thank him for that. We give one shoulder of the sheep to the khevisberi, and the rest we feast on here, giving thanks."

The fact that Iakhsar and Kopala were holy people who killed the many-headed, > man-eating ogres that inhabited Pshavi in the mythical past, is basically the only biographical information we have on the deities. That this doesn't exactly fit into the canonical tradition of the Orthodox Church, the dominant religion of Georgia, doesn't bother Maia.

"The church is dedicated to St George," she says, pointing to the almost empty building behind her. "The shrine is dedicated to Iakhsar. It's like a parallel regime: God and Iakhsar, Christianity and paganism, living next to each other."

The parallel regime is apparent everywhere. People say "Amen" after toasting Iakhsar, and most would describe themselves as Christian. While children are initiated into Iakhsar's cult at the age of 10, many are baptised as infants as well. It is tempting to see this as a modern version of ancient Rome, where people would worship the state-approved gods and the emperors, but also the divinities in the local rivers and woods. "The lowlands have their traditions," says Maia, "and we have ours."

I suggest that some Orthodox Christians might look askance at the day's events. Maia smiles: "As long as the Church has no problem with the shrine, I have no problem with the Church."

Back on the mountaintop at Iremtkalo, Ioseb Kochlishvili is dozing on the wall of his shrine. Like every shrine-priest, he was visited in a dream and called to serve. And, like every khevisberi, he resisted, refusing to become Kopala's representative and give up on normal life.

Mindia toasts the memory of Iakhsar after the sheep sacrifice (Alexander Bainbridge)

"Boxing with God," as Kevin Tuite, professor of anthropology at Montreal University calls it, is the defining experience in becoming a khevisberi. You are haunted by dreams and hallucinations, the deity visits calamities on you and your family, and finally you submit.

"I'm in awe of them," says Tuite, one of the few foreign academics to have studied the religion of Pshavi. "They don't have a fork to go with their knife or a cup to go with their saucer, they live in what we would consider to be abysmal poverty, but they're charismatic and authoritative. They have a kind of gentle power."

Ioseb, even when taking an afternoon nap, exudes that gentle power. He probably did so even before he started the job, 69 years ago. No one knows who Kopala will choose next, but the position of khevisberi is often passed down through families.

Raphael, Ioseb's grandson and winner of the horse race, is a modern young man with a smartphone and a Facebook account, but he has his grandfather's bearing, and in some respects he appears old before his time.

"As long as I am physically able to do it I will maintain these traditions," he says. "All this was established before Christianity and since that moment it has remained, it hasn't collapsed. All these rituals and traditions and beliefs came from where we are now. They grew out of this place."