Holidaying at the seaside is one thing; living by or near the sea is quite another. One of the aspects of the Channel that I wanted to explore in the course of a long and sometimes leisurely cycle ride along it was what drew people to make their homes within sight, sound or smell of the sea.

The appeal of the seaside holiday is obvious. But those who take their week or fortnight, or flit to and from their holiday cottages, merely scrape the surface. If the weather is kind, they are happy. If it pours and blows a gale throughout, they come home resentful, as if they had somehow been cheated or let down.

For those who live by the sea, the weather is not really the issue – or, rather, they embrace it in all its moods. They tend to be less interested in the blatant pleasures of the summer season. In fact, they can't wait for the transients to go away, because then they can reclaim their territory. At last they can stroll along the beach or saunter along the esplanade without being jostled by half-dressed pleasure-seekers and having to dodge Frisbees and beach balls.

The magic of the sea lies partly with the water itself, obviously: the richness of variation it offers, the ceaseless flux of mood, colour and texture, the restlessness, the heartbeat of the waves. The 19th-century nature writer Richard Jefferies put it nicely in an essay called The Breeze on Beachy Head: “There is an infinite possibility about the sea ... it is not to be ordered ... it has a potency unfathomable ... there is something in it not quite grasped or understood.”

Another crucial dimension is that it liberates spirits. “I like the space,” an elderly resident of Sandgate, in Kent, told me. “With the land so steep behind, you need the openness in front.” A man I met beside the tiny harbour at Mullion in Cornwall had been born and bred in the village, gone away for many years, and was then drawn back. He extended an arm in the direction of France. “There's no end to your world here,” he said. “Does that make sense?”

It did. By that stage in my journey, I had begun to understand that power. Having breathed the air and watched the waves and listened to and felt the rhythms of this sea, I saw a kind of truth about it that had eluded me before. A big part of that is the availability of it, all the time, every day of every season. I talked to a retired couple in Bexhill, Kent – a seaside town notable for the abundance of such couples, as well as for the fabulous Art Deco De La Warr Pavilion – who walked a mile and a half back and forth along the front every day, weather permitting. “You can never get bored by the sea,” the lady said.

It follows that much of the Channel shore can be appreciated just as well – and often much better – in later autumn, winter and early spring as in the holiday season. Penzance, for instance, can look somewhat sad in the summer because of the unattractiveness of its beach and the unloved tattiness of parts of its seafront. But the town has much going for it (and will have more when its Art Deco Jubilee open-air swimming pool reopens after restoration). It has a superb position overlooking Mount's Bay, fine public buildings, elegant terraces and crescents, leafy, perfumed gardens, excellent pubs serving fish from Newlyn, and more besides. It also has a climate which, without warning, bestows what feels like an early summer's day when the calendar says it's winter.

Other towns need sunshine and warmth. I first visited Bournemouth in Dorset on a wet, blustery February and it was dismal. It was not until I returned to find its magnificent golden-sand beach glowing in the heat, and its clear blue waters beseeching me to come and swim, that I was able to see its virtues and understand its reputation.

A good few seaside towns struggle to look their best out of season, for obvious reasons, while some – Bognor springs to mind – seem to struggle at all times. Several are a mixture of the desperate and the elegant. Hastings in East Sussex has all kinds of social problems, and there are parts of its seafront that are in acute need of care and attention. But, as a whole, it seems to be emerging from a trough. Its old town is full of pleasant surprises, it retains a good number of its fine buildings (even if they do look somewhat scruffy) and the restoration of its peerless pier – due to reopen in a couple of months – should give it a great boost.

Hastings is one Channel port that retains an active fishing fleet. Another – surprisingly to me – is Eastbourne. I found it on a later winter's morning, mostly pulled up on the shingle awaiting better weather. But there were three blokes who'd been out collecting whelks from 800 pots, and were loading them on to a trailer. The board outside the fishmongers' advertised Dover sole, lemon sole, hake, brill, turbot – just about everything, in fact, that a fish-eater could desire.

The thing I most love about the seaside is its cheerful democracy. The snobs don't really like it, because of the generous way it spreads its special quality around. It is a realm of its own, with a different code, and no one can claim ownership of it. It is a playground for children, parents, grandparents, uncles, aunts and special friends. It accepts with equal ease the gregarious and the solitary. It inspires a distinct delight that can be found nowhere else. That is why so many live there, and so many more ensure that the pattern of their lives makes room for regular exposure to it.