Kalymnos, second hop on our tour of lesser-known islands, has many rock-climbing highlights – and some sweet spots for scuba diving • Read part 1 of Kevin’s odyssey : Ikaria

I’m sitting on the shady patio of the Paradisio cafe in Vlychadia, overlooking an azure bay. My wet swimming kit is drying on the fence. “I should explain the breakfast,” says Stethis, who has been my guide on an epic morning of hiking, caving and swimming. “Kalymnos people like to mix unexpected tastes. That tea is alisfakia – sage – with thyme honey. And that mizithra cheese goes with the watermelon.”

I try it. Great combination.

“And the coffee is bitter, but you’ve got this vanilla-scented sugar stick.”

I’m starting to get the hang of Kalymnos, the second stop on my quest to explore the Aegean’s lesser-known islands (which had begun in Ikaria, 2hr 40min away by ferry). This is an island that seems to love both the unexpected and the extreme.

Kalymnos was once the sponge-diving capital of the Mediterranean, but by the mid-1980s, a marine virus had destroyed that business. (The interesting nautical museum in the capital, Pothia, is filled with superb period photographs and artefacts.) Meanwhile, the nearby island of Kos had become a tourist honeypot, whose boat owners reportedly told visitors not to bother with Kalymnos because “it’s just rocks”.

Facebook Twitter Pinterest Kevin Rushby’s daughter Maddy climbing up to a cave at from the kayaks near Vathys

That didn’t deter Andrea Di Bari, an Italian climber who wanted rocks, big ones. He came over in the mid-1990s on a day trip, spotted the potential, and returned. By the early noughties, rock climbing had transformed the island’s reputation. I’ve come to test out this island of adventure.

I’m joined by my partner Sophie and daughter Maddy (16) who have come over on the ferry from Kos for a few days. We stay in the climbing capital, Massouri, in a cool and lovely apartment overlooking Telendos, a car-free island that I’m itching to explore. But we start by meeting Dimitris, a dive master who sails us east around the island. Dimitris comes from a long line of sponge divers, and points out the spot where, 50 years ago, his grandfather died while using a newfangled brass helmet and compressor. No one had told the divers about the dangers and complexities of using air underwater.

Today we are searching for dolphins and Dimitris assures us that his family connections will guarantee an encounter. Stopping above his father-in-law’s fish farm, he dives to 20 metres and releases a shoal of sea bass. I’m snorkelling above and see them come up, rolling their silvery sides enticingly, but there are no dolphins. The sea bass, sensing an opportunity, race away to freedom. Yesterday’s stormy winds, Dimitris thinks, have unfortunately pushed the dolphins out of their usual haunts.

Facebook Twitter Pinterest An octopus gets up close

We climb back into the boat and head into a narrow inlet where cliffs up to 15 metres high have become a popular spot for deep-water soloing. This involves diving into the sea, then grabbing the rock and attempting to climb to the cliff-top. A pair of old climbing shoes can be handy. Maddy skips the climbing element, walks up and chucks herself off the top. I fall. Deep water soloing is, I predict, a sport of the future, especially when the water is this warm.

We sail on round the east coast to the lovely village of Vathys, where we say goodbye to Dimitris and jump into kayaks with a new guide, Tasos. He paddles with us to another, much higher, cliff face and points out a cave about 50 metres up. After an ungainly transfer to land, we climb a narrow set of steps, push through a curtain of wild figs and descend into cool gloom. All sorts of ancient treasures have been found in this cave (they’re now in the remarkable Archaeological Museum of Kalymnos, also in Pothia). At the bottom is a pool of clear, cold water. Potsherds (broken ceramics) lie around the feet of great limestone pillars.

We retrace our steps and paddle out to a lovely deserted beach to picnic in the shade of a tree. Day one complete, we return to Massouri, where there are lots of excellent restaurants.

Facebook Twitter Pinterest A fishing boat arrives back at Pothia, much to the interest of a local cat

10 great Greek islands: readers’ travel tips Read more

Next morning Maddy and I are back with Dimitris for a shore dive which proves to be something of a classic: sponges, amphorae, a German second world war plane, bombs, a wrecked sponge boat, plenty of fish and an irate octopus that squirts angry ink, changes colour and escapes up a tiny rocky hole.

Finally, the bit I’ve been most looking forward to: the rock climbing. We are picked up by Loukas and Kostas. Is it going to be too hot? Loukas shakes his head. “We have climbing in the shade all year. May and October are best, but summer is fine.”

There are also so many routes that climbers can do 10 a day for a year and never repeat one. We head north to cliffs above the village of Arginonta. Kostas, Maddy and I climb. Sophie watches. There’s a dog barking down in the village.

Facebook Twitter Pinterest Kalymnos’s capital Pothia

“That’s Vito, my dog,” says Kostas. He points out his house. “He wants to play.”

Sophie heads off to find Vito. I manage to climb a grade harder than I’ve ever done before and then we head for the beach. That’s the joy of Kalymnos: hard, dry rock followed by cool wet sea. There we find Sophie in doggie heaven. Vito likes diving down, picking things off the seabed and bringing them back to be thrown. Climbing may be the new Kalymnos, but the sponge-diving heritage is still there.

• The trip was provided by Sunvil which has seven nights self-catering at Eftychia Residence in Massouri from £1,253pp including flights from Gatwick and transfers. The ferry from Ikaria to Kalymnos costs €32 one way, from Kos €11, book through ferries.gr. Six mornings climbing with an instructor (beginners and intermediates) cost €350pp; diving trips cost from €50pp, book at visit-kalymnos.gr

Looking for a holiday with a difference? Browse Guardian Holidays to see a range of fantastic trips