One false move, and it was "adeus". The path hadn't been designed with ramblers in mind.

Built along the lip of a levada (water channel), it was only a foot wide, but the drop was several hundred feet. The masons who built them diced with death (a gamble they occasionally lost). In old photographs, you can see them at work, dangling off the cliffs with chisels. But despite their view of eternity, they worked with astonishing precision: this levada, Dos Tornos, descended only a matter of inches in every mile.

Actually, it would have been a spectacular fall. I would have plummeted through a fantastically lush and improbable landscape (imagine Devon, half-capsized and sprouting giant houseplants and bananas). I'd have been fleetingly airborne among charming little creatures – variants of chaffinch and firecrest – not seen anywhere else. Then, I'd have hurtled past terraced fields, all quilted and vertical here, and still tended by mountaineers with hoes.

Eventually, I'd have made contact with the geology, something black and basaltic. But that might not have been the end of the journey. Boulders here are always on the move. Funchal, the main town, even has vast Victorian underpasses to funnel avalanches safely out to sea. The locals think nothing of such drama. This, after all, is their Madeira: fabulous, unusual, and not quite as sedate as people think.

This island has always intrigued me. On the world map, it looks like a full-stop after "Atlantic Ocean". It is, in fact, about twice the size of the Isle of Wight, although it rises to 6,106ft (1,861m). Even then, it's only the tip of a mountain, another 9,843ft (3,000m) of which falls away to the ocean floor.

All this, I now realise, explains the rain shadows, the volcanic soil, the fertility, and the 200 road tunnels. Madeira must be the only place in the world where – occasionally – you can drive from snow to pineapples in 15 minutes.

I had a hunch a child would also enjoy all this, and I was right. Our eight-year-old daughter, Lucy, was slightly overawed at first: the runway was built on enormous columns, like the Parthenon; Funchal was spilling out of the mountains; the streets were paved with lava; and someone offered us an enormous, blinking owl.

The town also had at least four yellow fortresses, and – every now and then – little brass bands came tootling though the lanes. The defeat of the pirates all seemed thrillingly recent.

By the second day, Lucy had the measure of the place. She'd ridden a few cable cars, spotted puppy-sized avocados and poked her head into the odd monster market (or were they fish?). It all seemed slightly make-believe. At one point, we plunged through the streets in huge wicker toboggans, covering a mile and a quarter (2km) in 10 utterly mind-jangling minutes.

Steering us were the two carreiros, wearing boots made of tyres. Ernest Hemingway had described this as one of the most interesting experiences of his life, which is roughly how Lucy saw it. Even better, it was winter in Britain, and swimming weather here.

Our first roost was above the town, in an old mansion. With its sumptuous antiques and drawing-room redoubt, Quinta da Bela Vista felt deliciously Georgian. The gardens were planted with trees from all over the New World, and I half-expected to find Jane Austen among them, enjoying the silence.

Britons have been staying in these hills for hundreds of years. Some came to ease their TB but most came for the wine. Captain Cook left with 3,300 gallons of Madeira, and even Napoleon sailed into exile with the equivalent of 557 bottles.

Once we went to watch the wine growing old. At downtown Blandy's ("Est 1811"), it's still a British hand on the tap. There, the wine matures in hot, stuffy lofts. It likes being treated rough, they explained (and that's what made it the empire-builders' tipple). But nothing happens quickly here. That week, they were bottling up the 1969 vintage, and – if you've a king's ransom to spare – you can still get a bottle of 1792.

After a few days, we moved down to the cliffs, to Reid's Palace. For 122 years, it's been one of the world's best-loved hotels. Churchill, Amy Johnson and Gregory Peck all besported themselves here once, around the pools. To me, it's like some fabulous stately ocean liner that's been embedded in the rock.

We went down six plush levels to bed, and the staff all looked exquisitely polished and naval. I wondered what they'd make of a child but they treated Lucy just as they had the crowned heads of Europe. She even had her own tiny robe and slippers, and a rather well-nourished teddy, now known as Winston.

From here, we set off, driving around the island. With all the tunnels, it was like racing from room to room. One minute we'd be in a ravine with hundreds of farms climbing the walls; the next we'd be among huge whale bones in Caniçal (until 1981, men had set out from here in open boats, armed only with harpoons).

Another time, we ended up on the roof of the island – the Paúl da Serra – a wild, boggy plateau, about 10 times the size of Hyde Park. It's still a hard life at altitude. More than a million Madeirans live abroad, and of the 270,000 remaining, a fifth still till the land. "We had donkeys once," they joke, "but we ate them."

During these hire-car adventures, the sea was seldom close. Some 80 per cent of Madeira's shoreline is sheer, and – at one point – we found ourselves on Europe's second-highest cliff (Cabo Girao: 1,903ft/580m), where we could see eagles catching the thermals, far below. All this made Madeira almost pirate-proof, and wherever there was an outlet through the rock (like Faial), there was a fortaleza and a battery of cannons.

The best place to swim is Porto Moniz, where the furious Atlantic gives way to weirdly serene volcanic pools. This is how it would be if you could swim on the moon.

It was even better without the car. Once we joined a boat trip, and soon found ourselves among a pod of dolphins. It was like bobbing around in a soup that's come to life. Another time, we rented horses, and rode up into the laurel forest above Santo da Serra.

I've never thought woods could be enchanting, but these were. We spotted giant dandelions and bright blue pigeons, and the trees were knobbly and human. Back in the Seventies, armed separatists had lived up here, and – whenever the army had appeared – they'd simply melted away, like elves.

Our last few days, we walked the levadas. We even moved up to a hotel with one running through its gardens (the supercool Choupana Hills). Farther up, the path became too sheer for Lucy (of the 870 miles/1,400km of levadas, 90 per cent are unwalkable) and so I had to crawl it myself.

In that gnarly, upturned world, even the plants had Hobbity names, such as link stink laurel and shrubby sow thistle. Our guide, Sandra, loved the levadas (Moorish technology, she said, introduced around 1450).

At lunchtime, we sat by a waterfall, and were covered in chaffinches, who'd come to help with the cheese. Ah, yes, I thought happily: the wild side of Madeira. If you don't find it, it will find you.

DID YOU KNOW?

Madeira translates roughly as 'Island of Wood'. It's named for the thick native laurel forest.

GETTING THERE

Monarch (monarch.co.uk) and easyJet (easyjet.com) fly from London to Funchal, from £260 return, and TAP (flytap.com) from £158 return.

PACKAGE

John Gimlette travelled as a guest of Original Travel (020 7978 7333; originaltravel.co.uk). It offers a five-night package, including return flight and transfers, from £555 per person based on two sharing.

Driving in central Funchal; it's a rat-run of steep, convoluted streets.

The food in the restaurants on Funchal's esplanade can be bland and overpriced. Head for the old town instead.

Take care on the more remote levadas. Hazards include landslips, getting lost and hypothermia.

Don't get caught out with casual dress. Some hotels, such as Reid's, still expect collar and tie.

THE INSIDE TRACK

Boat trips to see whales and dolphins run all year (a two-and-a-half-hour trip is priced from €48/£41; rota-dos-cetaceos.pt).

Join an embroidery class and discover this remarkable Madeiran art (00351 291 222 965; bordal.pt). Beware, a pillowcase can take up to three weeks to make.

Private walking talks with excellent English-speaking guides can be arranged through naturemeetings.com (291 524482; half-days trips cost from €27/£23 per adult).

The cable car to Monte runs daily (madeiracablecar.com; €10/£8.50 one-way). You can then come down by toboggan.

For an excellent introduction to Funchal, take an open-top bus tour (yellowbustours.com; €12/£10 per adult). Buses also tour the Christmas lights, and venture off down the island.

Hiring a small five-door car costs around €50/£43 a day.

Go riding with Paula and Paulo at Quinta do Riacho (966 720026; quintadoriacho.com).

For Madeira wine, visit Blandy’s old “lodge” at 28 Avenida Arriaga, Funchal. It also has a shop at the airport. Other appealing purchases in Madeira include honey cake and embroidery.

FURTHER INFORMATION

The local tourist authorities have useful websites: visit madeirapromotionbureau.com and visitmadeira.pt.

THE BEST HOTELS

The Choupana Hills Resort ££

Drawing on modern Asian influences, this designer hotel has wonderful views over Funchal. You needn’t go out to eat: the Xôpana restaurant is first-rate and rather funky. Try the cod, baked and served on a cobblestone (00351 206020; choupana hills.com; doubles from £104 per night).

The Quinta da Bela Vista ££

This beautifully restored 19th-century mansion offers a pleasant respite from hectic Funchal, far below. Most of the rooms are in more modern accommodation, around the gardens (706400; belavistamadeira.com; doubles from £141).

Reid’s Palace £££

Set in 10 acres of gardens, Reid’s is the epitome of old-fashioned style. Nothing’s too much trouble – there’s even a place for teenagers to chill out (291 717171; reidspalace.com; doubles from £256).

THE BEST RESTAURANTS

Muralha’s Bar £

A simple fish restaurant in the old whaling port of Caniçal, this is a favourite with locals. The fish is fresh, and it’s very good value at around €12/£10 a head (961468).

Quinta do Furão, Santana ££

With fantastic views down the north-west coast, this vineyard offers an unforgettable lunch. Try the baked limpets in garlic. Lunch costs around €25/£21 a head (Quinta do Furão, Santana; 570100; quintadofurao.com).

Riso ££

Right at the far end of old Funchal, Riso perches on a cliff, under a vast canvas canopy. Fabulous rice dishes from all over the world, costing around €20/£17 a head (280360).

O Jango ££

Deep in Funchal’s old fishing quarter, tables spill out into the street, and evenings can be lively. The seafood pasta is excellent. Dinner costs around €18/£15 a head (166 Rua de Santa Maria; 221280).