I'm a photographer. No, let me rephrase that: I would like to be a photographer. In reality I'm merely an obsessive who takes lots of photographs in the hope that some day, just once, he will produce an image that is really, truly memorable. Like the images that Henri Cartier-Bresson captured, apparently effortlessly, in their thousands. Think, for example, of his famous picture of the guy leaping over a puddle; or the one of the two stout couples enjoying a picnic on the banks of the Marne; or his magical picture of a cheeky young boy carrying two bottles of red wine on the Rue Mouffetard in 1954. I like this last one particularly, because the lad in the photograph is about the same age as I was then and I often wonder if he's still around, and what he looks like now.

You can think about this obsessiveness, this quest for the one perfect picture, as a kind of illness. If so, then I've had it for more than half a century. And I'm not the only sufferer. Only the other day I was reading Larissa Macfarquhar's profile of Derek Parfit, the celebrated Oxford philosopher, who believes that most of the world looks better in reproduction than it does in life. Unlike me, though, Parfit has specialised. There were only 10 things in the world he wanted to photograph, writes Macfarquhar, "and they're all buildings: the best buildings in Venice – Palladio's two churches, the Doge's Palace, the buildings along the Grand Canal – and the best buildings in St Petersburg, the Winter Palace and the General Staff Building".

Sunday on the banks of the river Marne. 1938. Photograph: Henri Cartier-Bresson/Magnum Photos

Accordingly, between 1975 and 1998, Parfit spent about five weeks each year in Venice and St Petersburg. (That's the kind of thing you can do when you're a fellow of All Souls.) Like me, he dislikes the harshness of the midday sun, so he'd wait for morning or evening light. He would wait for hours, reading a book, for the right sort of light and the right sort of weather.

When he came home, Parfit developed his photographs and sorted them. "Of a thousand pictures," Macfarquhar writes, "he might keep three. When he decided that a picture was worth saving, he took it to a professional processor in London and had the processor hand-paint out all aspects of the image that he found distasteful, which meant all evidence of the 20th century – cars, telegraph wires, signposts – and usually all people. Then he had the colours repeatedly adjusted, although this was enormously expensive, until they were exactly what he wanted – which was a matter of fidelity not to the scene as it was but to an idea in his head."

Now that's a serious case. My condition is nothing like as bad as that. But I recognise the longing for perfection. Parfit contracted the illness because a rich uncle gave him an expensive camera (make unspecified). I caught it via a chance encounter when I was 13.

I was brought up in rural Ireland, which in the 1950s was a pretty sober society, priest-ridden and poor – not unlike Poland before the Berlin Wall came down. On Sunday afternoons, my parents insisted that the family go for a "drive" – an idea I found tedious in the manner of teenagers the world over. On one such Sunday we wound up in Killarney, Ireland's answer to the Lake District, and we were walking through the beautiful grounds of Muckross House when we came on a young woman sitting on a bench. She was in her 30s, neatly dressed and with a self-possessed air.

A photograph taken by John Naughton on 20 April 2013 in a caravan park in Kerry. John says: ‘The “austerity” regime imposed as a condition of the EU bailout was visible everywhere in Ireland at the time. The little boy was dejected because nobody would play football with him. It was one of those metaphorical moments.’

We sat on a nearby bench and my father engaged her in conversation, much to my embarrassment. It turned out that she was English and on her first visit to Ireland. Da asked if she was enjoying it. "Very much," she replied. What did she like about the country? "Oh," she said, "that's easy: the cloudscapes." She explained that she was a photographer and Ireland had very interesting light because of the way the sunlight was filtered through the clouds.

At this point I sat up and began paying attention. I had never heard this kind of talk before. "What sort of camera do you have?" I asked. She explained that she had two – "one for colour and one for black-and-white". I was astonished: in our world families had (at most) only one camera; and any photographs they took were in black and white. Seeing my amazement she asked if I would like to see one of her cameras. I nodded eagerly. She reached into her bag, took something out, leaned towards me and placed it in my outstretched hand.

I nearly dropped it! I was expecting something of the weight of a Box Brownie. Instead I found myself holding a silver-grey metallic object that looked more like a scientific instrument than any camera I'd ever seen. "It's a Leica," she said. "It's made in Germany."

The rest of that afternoon is lost in a haze. I do remember her talking about how one should use a yellow filter when photographing landscapes in black and white (it deepens the blue of the sky and makes clouds stand out), about framing and composition, and some stuff about focal lengths. But what I came away with were two ideas: one was that photography was something that was challenging, interesting and rewarding; the other was that if you wanted to do it properly you needed serious kit.

Oskar Barnack, inventor of the Leica camera, 100 years ago.

That kit was invented 100 years ago this year in Wetzlar, a small town in Germany, where a 35-year-old technician invented a camera that would shape the way we perceived the world for the rest of the 20th century.

His name was Oskar Barnack, and he worked for a company called Leitz which made microscopes for scientific research. He had been hired by Ernst Leitz, the proprietor of the company, in 1911 and by 1913 had risen to be its director of research and development. His abiding passion, however, was not microscopy but photography, an art form that at that time required not just technical skill but a physique strong enough to lug around a large plate camera and its load of 16.5cm x 21.6cm glass plates.

Barnack suffered from acute asthma and the weight of the kit caused him difficulty in breathing, so he set out to reduce the load. He first tried fitting four images on to a single glass plate, but abandoned that approach because the quality of the images was poor. (At that time photographic prints were mostly produced by contact printing from the negative and so quality was directly proportional to the size of the negative: the bigger the glass plate, the better the result.) Barnack concluded that lightweight photography would have to be done with something less dense than glass plates, and with smaller, lighter, cameras.

At this point, he had a stroke of luck. One of his colleagues, Emil Mechau, was working on a project to improve the performance of movie projectors, particularly the infuriating fluttering of the images when projected on to a screen. He was working with 35mm celluloid roll film – a format invented by Thomas Edison in the 1890s which eventually had become standard for the emerging motion-picture industry. Barnack had found the lightweight recording medium he sought. All that was needed was a camera that could handle it.

Barnack set about designing and building one. The prototype he came up with was made of metal (hitherto cameras were hand-built, often exquisitely, with hardwood). The camera took one picture at a time, the film being wound on manually by means of a sprocket wheel that engaged with the holes on the sides of the film strip. Because the film moved horizontally – rather than vertically as in a movie camera – he decided that the dimensions of each image should be 36 x 24mm, and that a roll of 36 images would fit in the camera body.

Thus were set the basic parameters of 35mm photography. There remained, however, one problem. Since the 36 x 24 images were tiny by the standards of the day, the only way to produce large images of acceptable quality would be to print them via an enlarger. The tiny images would have to be phenomenally sharp, which meant that they needed lenses of extraordinary optical quality. Here again Barnack was lucky: one of his colleagues at Leitz was a genius with optics named Max Berek, who designed a 50mm lens (the first Elmar) that delivered the kind of optical performance Barnack's camera needed.

The Leica 1, 1925, the world's first 35mm camera. Photograph: Science & Society Picture Librar/Getty Images

The first three prototypes of the camera were produced in late 1913 and early 1914. It was called the Ur-Leica (Lei from "Leitz" and Ca from "camera"). It was astonishingly small, fitting comfortably into one's hand, had a two-speed shutter, an automatic frame-counter and Berek's f3.5 Elmar lens (which collapsed into itself when not in use, making the camera even more compact). It was a breathtaking, revolutionary device that would change photography for ever.

It would be some time before the world found out about it, however. One of the first photographs Barnack took with the camera shows a spike-helmeted German soldier who has just affixed to a public building a copy of the Emperor's Order for total mobilisation. Germany, along with the rest of Europe, was descending into the first world war.

Leitz survived the war and the ensuing depression. The first commercial Leica – the Leica I – was launched at the Leipzig Fair in 1925. It was already much more sophisticated than the prototypes. It had a built-in optical viewfinder, shutter speeds ranging from 1/20th to 1/500th of a second, an accessory shoe – and Berek's Elmar lens. Just under 59,000 of the Leica I were made and those that survive are now among the photographic world's most coveted collectibles. Five years later, the first Leica with interchangeable lenses was introduced. The revolution was under way.

Leica cameras transformed the embryonic genre of photojournalism. Journalists had been using cameras almost from the dawn of photography: think of Roger Fenton documenting the Crimean war, Mathew Brady doing the same for the American civil war or Jacob Riis's photographs of the lives of the poor in the tenements of 1890s New York. These pioneers were constrained by the bulk of their equipment and their reportage was correspondingly static and formal. In most assignments, aspiring photojournalists stuck out like sore thumbs, or at any rate like the tripods they were obliged to use.

The Leica changed all that. Suddenly it was possible to be unobtrusive. The camera fitted in a coat pocket. It didn't need a tripod and was quick and quiet to operate. So photography became fluid, informal, intimate: the technology no longer got in the way of telling the story. So new kinds of storytelling evolved, published in the new illustrated magazines such as Picture Post and Life.

Alberto Korda with his 1960 portrait of Ernesto "Che" Guevara which became a global revolutionary icon. Photograph: Jose Luis Magana/AP

These publications developed new ways of laying out and presenting stories, creating the narrative not with slabs of text but with photographs, captions and short pieces of text. Freed by Barnack's 36-exposure rolls from the straitjacket imposed by glass plates and cut film, photographers were suddenly able to take as many shots as they needed, enabling editors back at base to choose from their contact sheets the images that best suited the narrative they were creating. The heyday of this kind of photojournalism was from 1925 until the 1960s, when the illustrated-magazine format began to wilt under the pressure of television news and features.

At the heart of photojournalism was the Leica. Almost all the great photojournalists of the period had at least one of them in his or her bag. (The only exception I can think of is our own Jane Bown: she always worked with Japanese SLRs.) Many of the images that became, in one way or another, iconic of the time were shot on Leicas: Richard Nixon jabbing his finger angrily at Nikita Khrushchev; Alberto Korda's photograph of Che Guevara; Cartier-Bresson's photograph of Gandhi's funeral pyre; Bert Hardy's image of the Queen attending the Paris Opera in 1957; Cartier-Bresson's photograph of a Gestapo informer being publicly exposed by a woman she had betrayed. And so on. Leica seeped into popular culture, such that when theatre critic Walter Kerr reviewed John Van Druten's I Am a Camera with the words "Me no Leica" everybody got the joke.

Leicas have never been cheap (the latest model in the M-series costs about $6,000 just for the camera body) but when you handle one you can see why. They are beautifully engineered precision instruments, and that kind of precision costs money. They have a reassuring heft and solidity, and shutter actions that are exquisitely balanced and quiet. (Even today some US courts define acceptable noise levels for courtroom photography in relation to the noise level of a Leica shutter.) And they go on for ever (my venerable M4-P dates from 1980 and still seems as good as new) – and Leitz will fix and service them if they falter.

Until the early 1970s the cameras contained no electronics – not even an exposure meter – which meant they were astonishingly robust. The playwright Arthur Miller once recounted an occasion when he and his wife, the photographer Inge Morath, were invited to dinner by Fidel Castro. "On arriving in the Palace of the Revolution," Miller recalled, "my wife was immediately required to give up her Leica before meeting Castro. The man taking the camera promptly dropped it from a high bin to the stone floor." Later on in the evening, however, an aide handed Castro a book of Morath's photographs. On seeing them Castro "promptly ordered an underling to return her camera to her. And he had no objection to her photographing him the rest of the evening." The Leica still worked flawlessly.

The other reason why Leicas are eye-wateringly expensive is the glassware. Leitz lenses are astonishingly good in terms of sharpness, resolution and colour rendition. The top-end Noctilux f0.95 50mm lens, for example, is capable of admitting more light than any other lens in the world. But at nearly £8,000 it can also deplete your current account faster than any other lens in existence. I know only one person who has this lens, and he long ago made so much money from technology companies that he doesn't notice the cost. But even a standard 50mm Summicron f/2 lens costs around £1,600.

These prices attract the derision of some amateur photographers, who see them as proof that Leica has sold out – abandoned the business of serious photography for the universe of luxury goods dominated by Louis Vuitton, Breitling et al; the world of the Financial Times's nauseating "How to Spend It" supplement. It is true that the red dot that was the badge for the Leica brand had become something of a fashion icon – to the point that serious photographers took to obscuring the dot with black tape. (In recent models, Leica has abandoned the dot.) But people who buy Leicas as fashion accessories often come unstuck, because you have to know what you're doing in order to use the M-series cameras. There's a lovely sequence of photographs online of Eric Clapton using an M8, for example. He takes the photograph, then looks in puzzlement at the LCD monitor and the camera until he eventually realises that the lens cap is still on. The Queen, meanwhile, has been an assured Leica user for decades. And she always takes the lens cap off.

Like other great engineering companies, Leica nearly missed the digital revolution. Initially, the new technology didn't seem to pose a challenge to high-end photography: the pixellated images produced by sensors the size of a baby's fingernail were too crude. But the bell began to toll for analogue photography in 2003 when Canon released the EOS 300, the first competent digital single lens reflex, and Nikon followed soon after with its D70. It was only a matter of time before larger sensors would start to produce images as good as those obtainable from film.

As the Japanese giants raced to introduce sensors that would match the size of Oskar Barnack's original 35mm frame, Leica looked like a rabbit transfixed in the headlights of an oncoming car. Instead of updating its M-range to take digital sensors, it fiddled about in an alliance with Panasonic to produce expensive but essentially derivative consumer cameras which were really just rebadged versions of Japanese originals. For a time it looked as though Leica would go the way of Kodak, another company that had dominated analogue photography but failed to master digital.

In the end, Leica was rescued from its near-death experience by a wealthy Austrian entrepreneur, Andreas Kaufmann, who gradually acquired a controlling stake in the company between 2002 and 2006 and turned it round. The first digital M-camera, the M8, was launched in 2006. It was a flawed product, but at least it showed that it was possible to combine Leica's traditional mechanical excellence with bigger sensors. And when the M9, with its full-frame 36 x 24 sensor, appeared in 2009 it was clear that the firm might weather the storm. Which it seems to have done: last year Leica reported annual revenues of around €300m and its 600 employees have moved back to a futuristic headquarters in Wetzlar, aided no doubt by the sacrifices of fanatics like me who took out second mortgages to buy M9s.

I bought my first Leica when I was a graduate student at Cambridge. It was a second-hand M2 with a 35mm Summilux lens and foolishly extravagant for a skint young scholar. In retrospect, though, it was one of the wisest purchases I ever made – not because it was an investment (though it could have been that) but because it taught me everything I know about photography. It forced me to think about what John Berger called "ways of seeing" rather than merely taking shots. It also pulled a comforting rug from under my feet: no longer could I blame my inferior work on the cheap lenses and crappy cameras that were all I could afford. With the same kit as Henri Cartier-Bresson, if I failed in the quest for the perfect picture then I only had myself to blame. Forty years on, that's still the position. Still, tomorrow's another day…

• This article was amended on 28 August 2014 to remove a Robert Capa photograph (and a reference to his use of a Leica on D-Day) and to correctly credit the quip "Me no Leica" to theatre critic Robert Kerr.