On and off over the 20-odd years that I have been trying to write journalism, I have carried around in my bag Joseph Mitchell's book, Up in the Old Hotel. Mitchell grew up in rural North Carolina at the beginning of the last century, came to New York as a young man to work as a crime correspondent in Harlem, and subsequently became known at the New Yorker magazine as the pioneer of a particular kind of reporting that owed something to Mark Twain: extended portraits of people and places at the margins of the city, told with all the patience of a novelist, and the precision of a newspaperman. Up in the Old Hotel collects all of the stories Mitchell wrote in this manner, for The New Yorker, from between 1943 and 1964. Until the publication of a Vintage edition, next week, admirers of Mitchell's writing have had to rely on an American import. The book, which runs to 707 pages, is not an insignificant piece of luggage, but it has anyway accompanied me on assignments to Islamabad and Dar es Salaam, as well as to Devon and Northumberland.

Quite often on these trips I have returned to memorable passages for inspiration or entertainment, or just for the sense of a friendly voice in a strange place. Once or twice, usually having reread favourite pieces, "Mr Hunter's Grave" or "Old Mr Flood", I've left a copy of the book in the bedside drawer of the more desperate hotel rooms, in the belief that later occupants might find far more early-hours solace and pleasure in it than the Gideons' Bible.

I'm at something of a loss to explain exactly why this collection of discursive profiles from a forgotten New York should have taken such a hold of me. Partly, it has to do with Mitchell's unsurpassed sense of sentence and paragraph – Salman Rushdie calls him the "buried treasure" of American writing. And partly it is his eye for character – Mitchell detailed men like Commodore Dutch, who for 40 years made his living by giving an annual ball for the benefit of himself; Arthur Samuel Colborne who devoted his life to the abolition of swearing in the city and, most famously, Joe Gould, "Professor Seagull", who squawked and screeched through Greenwich Village after the war, apparently writing its oral history, nine times the length of the Bible.

More than that, though, his profiles often have the apparatus of personal quests, and offer little insights into the sometimes lonely work of getting documentary stories on a printed page. He begins "Mr Hunter's Grave", a tale that expands to provide a whole illuminating history of the settlement of free black families around the oyster beds of New York, for example, like this: "When things get too much for me, I put a wild-flower book and a couple of sandwiches in my pockets, and go down to the South Shore of Staten Island and wander around awhile in one of the old cemeteries down there…"

Writing is presented as an escape from his own cares, and an excursion into the life of others, which has always seemed to me as good a way as any of thinking of it. Mitchell was scrupulous not to foreground his own opinion or to judge; the stories are, without exception, however, profound examples of the writer's ability to move and engage and amuse the reader, simply through the skill and care with which observations are made and quotations are put down. He thought of himself as a humble reporter, but he was a humbling one, too.

Over the course of the book you can feel Mitchell drawn to the places where his stories come alive – the city's saloon bars and graveyards and, most especially, its waterfront. As a book that reminds you that great cities live on water, this one sometimes brings to mind Conrad's London, but most of all Joyce's Dublin (if Mitchell carried a book in his own writer's bag, it was Finnegans Wake).

Like Joyce's sentences, Mitchell's often seemed to flow effortlessly and unstoppably, finding their own beautiful route through the lives and places he described. Given this facility, it was all the more puzzling, therefore, that after completing his tragicomic signature piece "Joe Gould's Secret" in which he revealed how Professor Seagull had left behind not a word of his fabled oral history, Mitchell himself stopped publishing altogether, at the age of 56. In the subsequent three decades of his life, he went into his office as usual at The New Yorker most weekdays, apparently working tirelessly on stories, and never submitted another word. So revered was he on the paper, that this long pause went unchallenged and unexplained – the enduring belief was that, if it ever arrived, the next story would be another masterpiece, but it never did.

As well as coming into the office, Mitchell continued to walk his city as he had always done in search not only of stories to tell, but increasingly of objects to salvage. These fragments – hotel door knobs and masonry and old jars from the bottom of the harbour - were stored in bags, and often carefully labelled. When Mitchell died in 1996, aged 87, he left most of this material to his two daughters, Nora and Elizabeth (his wife, Therese, had died in 1980). Last week, I spent a couple of days trying to speak to Nora Mitchell Sanborn, a retired probation officer, eventually tracking her down through a New Jersey barbershop in which she had staged an exhibition of her mother's photographs of New York of the 1930s and 40s.

When I caught up with her, she suggested she still found it easier to talk of her father in writing. My emailed questions ran in roughly the following order: Did Joseph take her with him on his quests around the city? Did he have particular habits? Was the great heart of his journalism also a characteristic of him as a father? Did he ever seem overwhelmed by doubt? What did he collect, and why, and what did her mother make of this obsession? Did she ever meet Professor Seagull? And what about the years after he wrote that story? Nora responded to these questions in a long and heartfelt email. I'll quote a little of what she said here, somewhat in the manner of Joseph Mitchell, as a single paragraph, and leave it at that.

"Yes," she explained, "I was aware of my father wandering around the city and he often took me with him. If he ran into someone who he wanted to talk to, he would, and I would stand around, shuffling my feet. After my mother died, I would drive him through the streets on Sundays, through blocks where he had lived in single rooms when he first came to New York, and he moved once a month to get to know the city. My father was canonical. He left the apartment at around 9 every morning and returned at 6. He always had a piece of New Yorker stock sort of yellowish paper in his breast jacket pocket – folded three times into a rectangle – and he would jot down anything he heard or saw that moved him. And yes, he was a very good father. We were never 'talked down to' and seldom chided. He was great company and acted like he thought we were. He collected some of everything. He would go out on Saturday mornings with a couple of shopping bags and return with building rubble, old nails, billheads from an abandoned building he had climbed through or silver-plated utensils from old New York hotels that he would buy at flea markets. (Among other things, my sister and I divided 240 pickle forks after he died). We have it all over our houses now. My mother was very long suffering and good natured. We lived in a small apartment and he would store everything, neatly and well marked, in corners and under things and over things and she once said that we were probably the only people in New York to have a skyscraper under the bed. I saw Joe Gould around the neighbourhood when I was little. Once he came up to us and my father, always courtly, introduced me. I still remember the scene and cringe. He really scared me. And it's true that, [after he wrote that story, for 30 years] my father was always writing. He would talk about certain projects and get involved in a million things. He had oceans of paper in many file cabinets, at home and at the office. Unfortunately these papers have been in storage since he died and in the charge of his executor [from whom his daughters are estranged]. We have not seen them… I missed one question – 'was I aware of him suffering doubt?' (It's funny I should have missed this one – a Freudian slip, I think. I can't stand the thought of his sadness or doubt). Yes, I was very aware of his melancholy and his sense of time and loss and change and how it affected his life and his writing. His past and his childhood and his parents were never far from his mind. He even mourned our childhood. He never recovered from my mother's death. Not to mention his migraine headaches and his ulcers. But New York was [by then] changing too much and too fast for him to get a grip on it… I wonder every day what he would think of it now."