There is a playful antagonism between the inhabitants of New Zealand's two islands, North and South. If you're a North Islander: the South might have better views, but the North is superior because it has richer culture. If you're a South Islander: the North might have richer culture, but the South is superior because it has better views. It's a quarrel between substance and form, if you like, a question of emphasis – does a country's nature owe most to its history, or to its land? In both senses New Zealand is curiously compressed. The first Polynesian settlers landed less than 1,000 years ago, the first Europeans less than 300. Geographically, too, the land is compact: a five-hour drive over the spine of the Southern Alps will take you through a dozen entirely different landscapes – beach river valley marshland rainforest gorge foothill highland alps plains peninsula beach –and each with its own weather, its own skies, its own quality of light. (It is a strange thing how swiftly the forecast can change in the Pacific – dress for all weather, the backcountry guides advise you, and expect four seasons in a single day.)

The South is the more visually stunning, but the North is the more populous and cultivated: this is a contrast that recalls each island's proper name. The North Island is Te Ika A Maui, "the fish of Maui" (recounting the mythic tale of New Zealand's creation) where the South is Te Wai Pounamu, "the waters of greenstone" (describing the glassy stone, prized by Maori, that is found in the swift rivers and along the savage misted beaches of the lonely south). New Zealand national identity lies somewhere between these emphases, North and South: as a bicultural nation, it must identify both as "the place of this people" and as "the people of this place". In Maori the country's full name, Aotearoa, is a lovely kind of oxymoron: it translates as "the land of the long white cloud", as if clouds were properties of the earth, or served in some strange way to invoke it.

I grew up on the South Island of New Zealand, in a city chosen and beloved by my parents for its proximity to the mountains – Christchurch is two hours distant from the worn saddle of Arthur's Pass, the mountain village that was and is my father's spiritual touchstone, his chapel and cathedral in the wild. For many years while I was growing up my parents did not own a car. We rode around town on two tandem bicycles and one single (a source of considerable embarrassment to me at the time) and at weekends we would occasionally rent a car in order to drive into the alps, and go hiking.

My father is an expatriate American; he fell in love with New Zealand in his youth and never went home. As a child I didn't really comprehend my father's affection for the land, nor for the steep-sided alp to which he returns as to an altar: Avalanche Peak, a six-hour ascent above the cloud-filled valley of the pass. My sense of injustice about our family's "weirdness" in not owning a car was amplified by the fact that we did not own a television either – my parents were unapologetic about this, and told me very cheerfully that I would thank them for it when I was older, which was quite true. But at the time Dad's refrain "Nature looks more beautiful in the rain" was not met with good grace. Nor was his notion that a view was something gained through effort – scenery, for him, was something that ought to be deserved.

When we reached our summit, or whatever spot was deemed by my father to be of adequately punishing distance from the car to deserve lunch, Dad would invariably find he had forgotten his Swiss army knife (looking back, I begin to doubt he ever had one) and instead would cut cheese into slices with the edge of his credit card.

It is this kind of detail that I remember – the credit card, waxy and oiled along its edge – from our expeditions into the hills. I can recall the clean-smelling interiors of each rental car, always a different model and a slightly different shape; the empty glove box; the chipped toes of my boots; and how my hands became swollen and too weak to make a fist after a day of walking uphill. I remember, once, the rubber seal around the car door clipped into the shape of a postage stamp by alpine parrots looking for something to steal. But I don't remember the views – not as memories. In fact I am sure that I never experienced, as a child, any kind of encounter with the sublime, that catch in the throat, that tightness of the lungs, that sudden, roaring sense of one's extreme smallness in a huge, awful, beautiful world.

To experience sublime natural beauty is to confront the total inadequacy of language to describe what you see. Words cannot convey the scale of a view that is so stunning it is felt. In such moments natural beauty becomes a kind of devastation – it is pure encounter, too compressed in time and space to be properly contained. I do not feel the sublime when I look at a city, however impressive it might be in proportion and shadow, for the reason that a city is designed, in its substance it has been formally determined, and it has been named already by the fact of its creation. Words are adequate. I have never been moved to tears by a skyline, or a building, or a painted arch, but the sudden apparition of a peak from behind a sheet of mist is enough, now, to make me cry.

I think that a child does not feel the sublime because a child need not, perhaps cannot, confront the limitations of his or her language – language, for a child, is already miraculous, supple, generous in its association, tragic, hilarious, disproportionate and huge. Looking at a cloud-filled valley was less interesting to me (or at least, no more interesting to me) than looking at my father drag his thumb along the magnetic stripe of his credit card to wipe it clean.

When I was 14, my father took me on a tandem bicycle trip across the mountains. He had already taken my sister and then my brother, in his turn, and as the youngest, my trip came last. We were to cross the Lewis Pass, touch the Tasman Sea, and return over Arthur's in a loop. The trip would take four days. I remember with clarity the preparations for the journey – oiling the chain, strapping down our tent, fitting the road map into the laminated pocket on the front of the bike. But I remember, too, how hopeful I was that something out of the ordinary would happen; that we would discover something, or have to endure something, out of which might come a story.

My brother had described to me an event from his own trip several years prior. He had awoken early in the morning and witnessed firsthand the birth of a calf. He and my father had pitched their tent in the stolen corner of a farmer's lot, and so it was from inside the fence that my brother saw, not 10 feet away from him, the newborn calf slither on to the grass, unfurl its legs, and stand. The story had captivated me and stirred my jealousy to such a degree that I could recall the birth almost as a memory of my own – I wanted to return there, as to a favourite page in a favourite book.

It is curious to me how often we tend to describe the perfection and drama of the natural world, its sublime qualities, in metaphors of fakery or artificiality: "like a postcard", "like a painting", or latterly in New Zealand, "like a scene from The Lord of the Rings". The impulse, I think, comes from a wish to apologise for the limited capacity of the "real" world. To grow up is to confront the disappointments of language, in a way, and to suffer the divorce between what we experience and what we imagine to be real. I was preemptively disappointed, setting out on the tandem for the mauve shadow of the hills, to know that I would in all likelihood see no newborn calves, that our adventure would have a different character to the adventure undertaken by my brother and my father. I had settled, I think, into an adult frame of mind.

I drove through Arthur's Pass recently, and stopped to climb Avalanche Peak for the first time in several years. The ascent is taxing, rising sharply through beech forest to the sudden treeline and bare grassy peaks above. The summit offers a view across the blue ranges and snow-capped summits of the island's keel. The final length of the ridgeline stands as a rocky comb of shale against the sky, dropping down on either side to wide scree slopes and rocky bluffs and nothing. Across the valley to the west is the rumple of a high glacier, a face of snow; to the east, a horseshoe cup of grey and green. And yet it is hard to describe – indescribable, until you're up there, looking down – because the mountain is something other than its substance, something more.

Travel brochures try to capture the quality of New Zealand's panoramas with adjectives – "pristine", "untouched", "majestic". But the words seem cheap and insubstantial, however accurate they may be, in the face of the real thing. The language of description is always a matter of equivalence (a word equals the thing it describes) and so cannot contend with the sublime. But the language of paradox, oxymoron and subtle contradiction – the language of children – does better. Aotearoa is a land made perfect only by its opposites, the water and the air. It is both north and south at once. It is a land that casts its shadow on the clouds.