On the third leg of his tour, our writer explores the ancient walking trails of this mysterious Cyclades island, with its monasteries, sea cliffs and deserted hill villages • Previous stops: Ikaria , Kalymnos . Next: Folegandros

I’m getting the hang of Greek ferries. Turn up at the port early, check where the boat will dock, retire to a kafeneio, then be in the right place at the appointed time. The boat will slap down its ramp, disgorge passengers, ingest the next lot, and be gone. The entire process can take minutes. In this way I hop from Kalymnos to Kos, then take the three-hour voyage to Amorgos, at the same time shifting from the Dodecanese to the Cyclades.

Greek map Kevin Rushby’s Greek island trip has also taken in Ikaria and Kalymnos with Folegandros, Milos and Sifnos to come.

My third stop on this island-hopping adventure is an exciting outpost to see looming up in the early-evening haze. Imposing cliffs rise from a frothy skirt of spray then soar up through stony ramparts to peaks so high they have managed to conjure clouds from an empty sky. As we sail along the north coast, a few speckles of white houses appear, closely huddled together. This is an antique land, inhabited for around 6,000 years. The faces of the hills are lined with stone-walled terraces, occasionally interrupted by the sharp white paint of a simple barrel-roofed chapel. Between each bare headland is a dark, mysterious gorge.

I’m staying by Aegialis beach in the homely Pension Askas, set back from the coast in verdant gardens. My plan is to explore on day hikes and Marios, the owner, advises an early start. I set off at six, heading up the valley on a well-marked cobbled trail, one of the ancient paths that have given Amorgos a reputation for good walking. There’s no one around except a lone mule rider coming down from Langada village, a clutch of cuboid houses tightly woven together with steep alleyways, each step painted white, every window and door blue.

Facebook Twitter Pinterest Ioannis Theologus church’s white cat. Photograph: Kevin Rushby/The Guardian

Above the village, now on a stony trail, I spot a snake then the rare Eleonora’s falcon before arriving at Ioannis Theologus church. As well as being a sacred place, this tiny white gem is an intricate rain-catching device: each little patio draining into deep stone cisterns. No priest lives here now, only a white cat, perfectly camouflaged against the white-washed walls. There’s a key under a rock, but I can’t get it to work and have to be satisfied with a peek through the keyhole at the painted interior.

A mile further and I emerge on to a vast panorama of 600 metre-tall cliffs and wild sea. The path continues, snaking across a steep scree slope and clambering around cornices. I’ve been told that in strong winds or rain, continuing can be dangerous, but it’s a mild day … until I get halfway across. Then a violent blast of wind roars in. It’s the meltemi, the unruly summer wind, roaring up the cliff. I push on and reach my goal, the tiny chapel of Stavros, once home to monks who had fled Palestine in the ninth century seeking wild isolation. They definitely got what they wanted.

Facebook Twitter Pinterest A small church along the historical hiking trail to Panagia Chozoviotisa monastery. Photograph: Julien Garcia/Getty Images

I retrace my steps across the cliff and then pick out another route home: the hillsides are full of beautiful ancient pathways, sometimes choked up with spiky plants, sometimes defended by spider’s webs so thick it’s like walking into a loosely strung harp, but always leading to a tumble of stones and an abandoned cottage. Inside one I turn over a timber and there’s a rusted dining fork lying there, as if the last inhabitants had jumped up midway through breakfast and rushed away forever.

20 Greek islands … you may not have heard of Read more

Eventually I find a trail that gets me back to Langada and Niko’s taverna, a collection of colourful patios under shady vines where I grab a welcome cold drink. Later I swim and explore some smaller beaches. Dinner that night at Askas is a traditional oven-baked goat stew.

Next day I set out again, this time even earlier, heading up the mountainous spine of the island along a wonderful old mule trail past more deserted ruins. These houses have no windows, only a door and a fireplace, sometimes a little niche where a precious object might have sat. Inside one I find an old wooden chest, suitable for mule transport, and roof timbers as thick as your thigh, proof that this area was once thickly forested. In fact Amorgos was an important source of oak until a devastating forest fire in 1835.

Facebook Twitter Pinterest Typical transport in Amorgos’s narrow alleys.

I pass over a col and then down through wild gardens of thyme and juniper where clouds of butterflies accompany me. It’s still early and I’ve only seen one goatherd, but when I finally reach the monastery of Panagia Chozoviotisa I’m in for a shock.

This magnificent edifice, tucked neatly into the face of the tawny cliff 300 metres above the sea, is filled with tourists bussed up from the port of Katapola. I’m dusty and sweaty and the attendants take pity, ushering me into a kitchen and serving shots of psimeni raki, a spirit brewed by the three resident monks with lemon leaves, spices and sugar. It has legendary restorative powers, they assure me, although I reckon the large glasses of iced water also help.

Facebook Twitter Pinterest A small church below the village of Langada.

The monastery was reputedly built here after an icon of the Virgin Mary miraculously turned up in the 11th century, having travelled from Wadi Qilt near Jericho, a place I have visited and one that has a similar spectacular cliff location. From the monastery I march on with renewed vigour to nearby Chora, another wonderfully labyrinthine settlement with lots of tiny tavernas tucked into its shady alleyways.

Walking in Greece in high summer may not be ideal (May or October would be better) but draconian early starts and coordination with the local buses have convinced me it’s fine. Every island of any size seems to have good services – there is actually no need to hire a car. By late afternoon I’m lounging in the sea looking forward to the home cooking of the guesthouse. I have fallen heavily for this island, and am regretting not scheduling more time here: a few extra months would have been good.

• The trip was provided by On Foot Holidays; a four-night break, including B&B, route directions, maps, transfer and local telephone support costs £350pp; a seven-night break to Naxos and Amorgos is £485pp (both trips exclude flights and ferries). Ferries can be booked at ferries.gr. Further information at visitgreece.gr

Looking for a holiday with a difference? Browse Guardian Holidays to see a range of fantastic trips