For those of us outside the fusty world of philately, there are few reasons to get excited about stamps. Most childhood collectors grow out of the pastime as they do fossils or football stickers. To their annoyance, those who carry the habit into adulthood are considered a step above trainspotters in the hobbyist hierarchy. But before you turn the page on this story of stamps, just try, for a moment, to set aside prejudice and consider one sticky little square of paper that surely deserves to be coveted. The Penny Black, which first went on sale in London on 1 May 1840, is neither the rarest nor the most valuable stamp in the philatelist's album, but it is the most beautifully designed and nobly conceived. It also changed the world in ways that should excite us all.

Before 1840, there were no collectors to poke fun at because there were no stamps. To dis- cover how momentous the arrival of the Penny Black was, we must imagine posting a letter in, say, 1835, when you had to be rich – and patient – to use the Royal Mail. Delivery was charged according to miles travelled and the number of sheets of paper used, and it was the recipient who had to pay when the top-hatted "letter carrier" came knocking. If you lived in London and received a two-page letter from Cousin Hamish in Edinburgh, you would be stung for two shillings, or more than seven pounds in today's money.

Letter writers employed various ruses to save money, doing everything possible to cram more words onto a page. You wrote between or across the lines of your first page. Nobody bothered with heavy envelopes; instead, naked letters would be folded and sealed with wax. You then had to find a post office – there were no pillar boxes – and hope your addressee didn't live in one of the vast areas of rural Britain not served by the system. If you were lucky, your letter would arrive (it could take days) without being read or censored.

The state of mail had been causing concern throughout the 1830s, but it was Rowland Hill, an inventor, teacher and social reformer from Kidderminster, who proposed a workable plan for change. Worried that a dysfunctional, costly service would stifle communication just as Britain was in the swing of its second industrial revolution, he believed reform would ease the distribution of ideas and stimulate trade and commerce, delivering the same promise as the new railways.

Hill's proposal for a penny post, which meant any letter weighing less than half an ounce (14 grams) could be sent anywhere in Britain for about 30p in today's money, was so radical that the Postmaster General, Lord Lichfield said, "Of all the wild and visionary schemes which I ever heard of, it is the most extravagant." Lichfield spoke for an establishment not convinced of the need for poor people to post anything. But merchants and reformers backed Hill. Soon the government told him to make his scheme work. And that meant inventing a new type of currency.

Hill quickly settled on "a bit of paper... covered at the back with a glutinous wash which the user might, by applying a little moisture, attach to the back of a letter". Stamps would be printed in sheets of 240 that could be cut using scissors or a knife. Perforations would not arrive until 1854. The idea stuck, and in August 1939 the Treasury launched a design competition open to "all artists, men of science and the public in general". The new stamp would need to be resistant to forgery, and so it was a submission by one Mr Cheverton that Hill used as the basis for one of the most striking designs in history. Cheverton, who worked as a sculptor and an engineer, determined that a portrait of Queen Victoria, engraved for a commemorative coin when she was a 15-year-old princess, was detailed enough to make copying difficult, and recognisable enough to make fakes easy to spot. The words "Postage" and "One Penny" were added alongside flourishes and ornamental stars.

Nobody thought to add the word "Britain" – the stamps were intended for domestic use – but so powerful was the design that, apart from the arrival of new royal faces over the years, it never changed, and would come to symbolise the Empire. British stamps remain the world's only stamps that do not bear the name of their country of origin.

The Penny Black was an instant hit, and printers struggled to meet demand. By the end of 1840, more than 160m letters had been sent – more than double the previous year. By the turn of the century, the figure had rocketed to 2.3bn. It created more work for the post office, whose reform continued with the introduction of red letter boxes, new branches and more frequent deliveries, even to the remotest address, but its lasting impact on society was more remarkable.

Hill and his supporters had promised "moral" advantages from a cheaper post, which, they rightly predicted, would improve the "diffusion of knowledge". Suddenly, Cousin Hamish could be reached in a day or two and so, for the first time, Britain – and later the world – began to shrink. As literacy improved, sections of society that had long been disenfranchised found a voice.

Tristram Hunt, the historian and would-be MP for Stoke, values the "flourishing of correspondence" that followed the arrival of stamps. "While I was writing my biography of Friedrich Engels I could read the letters he and Marx sent between Manchester and London," he says. "They could write to each other three times a day, pinging ideas back and forth so that you can almost follow a real-time correspondence. For historians that's absolute gold."

The penny post also changed the nature of the letter. Weight-saving tricks such as cross-writing began to die out, while the arrival of envelopes built confidence among correspondents that mail would not be stolen or read. And so people wrote more private things – politically or commercially sensitive information, and love letters. "In the early days of the penny post, there was still concern about theft," Hunt says. "Engels would send Marx money by ripping up five-pound notes and sending the pieces in different letters. But the probity of the postal system became a great thing and soon it was part of Britishness to expect that your mail would not be tampered with."

So that's why we should get excited about stamps (or at least this one); the Penny Black's impact on society could only be compared with subsequent world-shrinking innovations such as the telegraph and the internet. When, then, did people who liked stamps enough to collect them become anoraks in the popular imagination? Pretty soon. In his 1988 work, Victorian Things, the historian, Asa Briggs, cites a report in Punch magazine in 1842 (just two years after the arrival of stamps) about a "new mania" that had "bitten the industriously idle ladies of England". A Times article in the same year reports a trend among such ladies to paper the walls of their dressing rooms with thousands of used Penny Blacks. The fad was widely ridiculed.

By the 1850s, dealers in coins and other curios saw potential in stamps. They included a 16-year-old Plymouth boy called Stanley Gibbons, whose name would become synonymous with collecting. Soon, "timbromanie", or stamp mania, swept Europe, as post offices all over the world followed Britain's lead with exotic stamps. Enthusiasts began to stick them into albums that could be viewed like mobile museums – or inspiring windows on the empire and the wider world.

The practice of philately, the study (rather than collection) of stamps, soon followed. Douglas Muir is curator of philately at the British Postal Museum and Archive, which is coordinating a year-long Festival of Stamps. He says the joy of stamps is about more than pasting and hoarding: "Stamps are microcosms of the age and place that reflect a great deal about the attitudes, history and geography of a country," he says. "You can say the same today. It's hard to compare the Penny Black to the Harry Potter stamps issued in 2007, but a hundred years from now, historians will look back and learn things about our popular culture."