A "lost" first novel by Arthur Conan Doyle, the creator of Sherlock Holmes, is to be published tomorrow for the first time, 128 years after it was written. The Narrative of John Smith has been seen exclusively by The Independent on Sunday and provides a fascinating glimpse into the young writer's mind. However, it also reveals that, as a young man, he found the creation of first-rate fiction far from elementary.

The book's manuscript formed part of a collection of private papers that emerged at auction in 2004 and was bought by the British Library for nearly £1m. Written in four black notebooks, the 130-page work has now been transcribed and typeset for worldwide release to accompany an exhibition of Conan Doyle-abilia at the British Library.

Many years after writing The Narrative, Conan Doyle said that he would be horrified if the book ever appeared in print. But academics have defended the publication because of its contribution to understanding his later work. "This book gives us a unique insight into the developing creative mind of the writer," says Rachel Foss, one of the book's editors. "This is his first attempt to make the transition from a short-story writer to a novel writer."

The book is about a 50-year-old man who is stricken with gout and confined to his couch for a week. He then attempts to write a book, and expounds his views on topics such as medicine, religion, literature and interior design. Many of the opinions clearly chime with the author's, such as his belief in the importance of science and medicine, and his scepticism about religious dogma.

Conan Doyle was living and working as a doctor in Portsmouth when he embarked on the novel in 1883. His father had been taken ill due to alcoholism, and the 23-year-old had to support his mother and fund the education of his 10-year-old brother. He had started writing short stories and submitting them to magazines to supplement his income. But he was frustrated by the Victorian practice of omitting the author's name, especially when one of his works in The Cornhill was hailed as being by Robert Louis Stevenson. For that reason, he attempted a novel, which would have his name on the cover. He then suffered a major blow when the manuscript of The Narrative got lost in the post, never to be found again. So he rewrote it from memory, the result of which is thought to be the British Library's manuscript.

Although the novel suffers from a lack of plot, it does conjure a world of boarding houses and pipe-smoking, which fans of Sherlock Holmes will recognise. Conan Doyle called it a novel with a "personal-social-political complexion" and it hints at themes that would appear in the Holmes books, such as an interest in logical reasoning.

An introduction to the new edition says: "The Narrative is not successful fiction, but offers remarkable insight into the thinking and views of a raw young writer who would shortly create one of literature's most famous and durable characters, Sherlock Holmes." The book gives a flavour of the preoccupations of the time, such as the British empire, science and the rise of secularism. It is also remarkably prescient, foreseeing the rise of America and China as superpowers, the advent of aeroplanes and submarines, and even space exploration. Stephen Fry, who has also seen the book, hailed Conan Doyle's breadth of interests. "He was the first popular writer to tell the wider reading public about narcotics, the Ku Klux Klan, the mafia, the Mormons, American crime gangs, corrupt union bosses and much else besides. His boundless energy, enthusiasm and wide-ranging mind, not to mention the perfect, muscular and memorable prose, are all on display here in a work whose publication is very, very welcome indeed."

Four extracts from 'The Narrative of John Smith'

A description of Mrs Rundle, the landlady, a precursor to Sherlock Holmes's landlady, Mrs Hudson

Good Mrs Rundle, the lodging-house keeper, came up to lay my luncheon and is glad to see me so much better. I think I have mentioned that she is a widow handicapped in the race of life by three children. It is no wonder, poor soul, if her face is a little hard and so puckered up with wrinkles that it looks as though her skin had been made for a larger woman, and she had been compelled to take in tucks in it. Every one of those lines is a record of some fresh trial. Evil fate has scratched its memoranda all over her face as Robinson Crusoe cut the days and weeks into the post ... What a catalogue of little miseries, all of them real enough and grave enough to her at the time, are chronicled in that mesh work of wrinkles.

A visit from the retired Major, the narrator's neighbour

Who should come in after luncheon but my good neighbour from above – clad in a somewhat rusty tweed suit, but retaining the peculiar slinging gait and easy lounge of his class.

"Should have been in yesterday," said he heartily, "but I was busy packing my traps together in case of an emergency. I'm all right now," he added with a sigh of relief. "I could start at a couple of hours notice."

"Start!" I exclaimed. "Why, Major, you don't mean seriously to say that your services may be required in the field?' "And why not, sir – why not" asked my companion hotly... "But I had no idea—" said I a little timidly, for the Veteran was evidently very touchy upon the question of his fitness for hard work. "I had no idea that there were any complications between our government and foreign powers. I was under the impression that we were at peace with the whole world."

The Major produced a folded newspaper from his coat-tail pocket and after much fumbling and searching pounced upon a very small telegram in diminutive type which was stowed away under the meteorological chart in a back column. Crumpling the paper up so as to bring this item to the front, he inflated his chest and smiled at me with a smile of superior knowledge.

"At peace with the world," said he impressively, "listen to this. 'The Russian governor of Kashgaria has determined to send a brigade of Cossacks to the Kuldja frontier in order to check the depredations of the marauding Tartars.' There, what do you think of that!" roared the Major, slapping the paper down upon the table. "When telegrams like that appear in the public press it is time for officers of the reserve to pack their boxes..."

I suppose that every country is afflicted with ultra-patriots of this explosive type. Jingoism, Chauvinism, Panslavism, Spreadeagleism, it breaks out in nasty blotches all over the globe, and a very unhealthy irritative condition it is. The only thing to be said for it is that it is a shade or two better than the sordid preference of private to public interests which prevails in some other quarters. Here is this old gentleman, who is a kind-hearted man enough... howling out for a war which would put a third of the world into mourning, and all for the sake of some grievance which is so shadowy that it rests upon the supposition of a supposition. What makes him more dangerous is that he is in deadly earnest over it – so earnest that he is quite ready and even eager to risk his own life upon the quarrel. Imagine the danger of an autocratic system of government by which such a man as this might find himself at the head of a state with unrestrained powers of pursuing what he would call a spirited policy towards his neighbours.

The narrator looks 5,000 years into the future, and imagines how an archaeologist, Dr Dryasdust, might report his excavation of the site of London

A large building had recently been unearthed near the dried-up bed of the River Thames, and there could be no question from existing records that this was the seat of the law-making council among the Ancient Britons, or Anglicans as they are sometimes termed. Near this was a square brick building called the Aquarium, and serving, as the name implies, as a place of seclusion for habitual drunkards...

The bed of the Thames had been tunnelled under by a monarch named Brunel who is supposed by some historians to have succeeded Alfred the Great. The principal places of amusement were Kensington (from the German root "kennen – to know", so called with reference to certain schools of fine art and cookery) – and Hyde Park, the name of which appeared to Dr Dryasdust to suggest the possibility of treasures being concealed in it. These open spaces must, however, have been far from safe, as the bones of tigers, lions, and other large Carnivora have been discovered in the adjoining Regent's Park.

The lecturer, having briefly referred to the mysterious structures known as "pillar-boxes", which are scattered thickly over the city and which, he remarked, must be regarded either as religious in their origin, or else as marking the tombs of Anglican chiefs, passed on to the cylindrical piping... In a series of observations extending over several years he had discovered the important fact that these lines of tubing, if followed out, invariably led to large hollow metallic chambers which were connected with furnaces. No one who knew how addicted the Ancient Britons were to tobacco could doubt what this meant. Evidently, large quantities of this herb were burned in the central reservoir and the aromatic and narcotic vapour was carried through the tubes to the house of every citizen so that he might inhale it at will.

On the future

What is to be the end of it all? Since first a man scratched hieroglyphics on an ostracon, or scribbled with sepia upon a fragment of papyrus, the human race has been puzzling itself over that question... We may safely suppose that man will win fresh victories over mechanical and natural difficulties. That he will navigate the air with the same ease and certainty with which he now does the water, and that his ships will travel under the waves as well as over them. That life will be rendered more refined and more pleasant by countless inventions, and that preventative medicine and sanitary science will work such wonders that accident and old age will be the only causes of death. That the common sense of nations will abolish war, and the education and improved social condition of communities will effect a marvellous diminution in crime. That the forms of religion will be abandoned but the essence maintained, so that one universal creed will embrace the whole earth, which shall preach reverence to the great Creator and the pursuit of virtue, not from any hope of reward or fear of punishment, but from a high and noble love of the right and hatred of the wrong.

These are some of the changes which may be looked for. And then? Why, by that time, perhaps the solar system will be ripe for picking.