On Sunday evening, the longest passenger train on Britain’s mainland rail network will pull out of Euston and head for the Scottish Lowlands. Several hundred customers, in 16 coaches, will be transported overnight with eight carriages arriving in Glasgow and eight in Edinburgh on Monday morning.

This is a regular, nightly service. But tonight’s Caledonian Sleeper trip will be different: travellers will be able to journey in double beds, use en-suite toilets, eat fine Highland food, enjoy modern air conditioning, and experience a wide range of whiskies in the train’s Club Car.

After two years’ delay, the much vaunted refit of the Scottish sleeper train will take to the rails: first with this week’s refurbished Lowlander services to Glasgow and Edinburgh and then, in June, with the Highlander services to Aberdeen, Inverness and Fort William.

“Once travellers set foot on the train, they are going to experience Scotland’s best, from luxury toiletries to good food and drink,” said Ryan Flaherty, managing director of Caledonian Sleeper.

Once travellers set foot on the train, they are going to experience Scotland's best Ryan Flaherty

“We are aiming to create a luxury hotel experience. After all, we couldn’t add any more coaches to the service because 16 is the upper limit for Euston’s longest platform – making the Caledonian Sleeper train the longest in the UK. So instead, we went for quality not quantity.”

It has cost £150m to build 75 new coaches, a remarkable change in fortunes for Scotland’s oft-threatened sleeper services. Once the acme of romance, overnight train travel – which featured in so many books and films, from John Buchan’s The 39 Steps to Alfred Hitchcock’s North by Northwest – dwindled in the latter half of the 20th century thanks to cheap air travel, and came close to extinction in the UK.

In 1995, it took a court order to halt British Rail from ending its sleeper service to Fort William, one of the world’s most beautiful rail lines; the London-Stranraer sleeper was axed in 1990; and Britain’s only other sleeper service – from London to Penzance – came near to closure a few years later.

Since then, the Scottish sleeper service has limped on using increasingly decrepit carriages with sticky carpets and malfunctioning air conditioning, experiences made worse when travellers found themselves sharing sleeping cabins with heavy smokers determined to flout trains’ strict no smoking policies or with cabin partners who urinated in their communal wash basin in the night. In another case, quoted by David Meara in his book Anglo-Scottish Sleepers, one woman awoke to find, on her pillow, a set of false teeth that had fallen from the gaping mouth of the stranger in the bunk above.

Facebook Twitter Pinterest Cary Grant and Eva Marie Saint bunk up in Alfred Hitchcock’s film North by Northwest, 1959. Photograph: Everett Collection/REX

Very often, these were tourists’ first experiences of Scotland, and although offset by cheery staff and glorious scenery on arrival, they left a lot to be desired. Hence the Scottish government’s decision to invest heavily in the Caledonian refit. From now on, visitors should arrive for walking or fishing or shooting holidays or trips to the Edinburgh Festival having travelled in style and without fear of night-time disruptions, said Flaherty. “Cabin-sharing with strangers is certainly a thing of the past,” he insisted. In addition, travellers will be able to board the Lowlander sleeper at 10pm – far earlier than they can at present – even though the train is not scheduled to depart until almost midnight. “Quite frankly hanging round Euston station at 10 or 11pm is not that pleasant, so we will give people a chance to settle in long before the trains head off for the north.”

Even better, travellers will be able to sleep in a double bed, the first time such a luxury has appeared on a regular UK train service. Food on board will feature the best of Scottish cuisine including plates of smoked venison and haggis, neeps and tatties, as well an improved range of whiskies. In addition, new engineering technology, in the form of special couplers, should stop trains shuddering when carriages are uncoupled and coupled en route to the north. “It’s going to be much more gentle and should be imperceptible to guests while they are asleep,” said Flaherty.

The investment may still seem unusual given that many European sleeper services are being abandoned, as high-speed trains are introduced, a point raised by transport analyst Christian Wolmar. “It is true that French and German sleeper services are being axed but in other countries, such as Sweden, they are being reintroduced. Some romance is being brought back to rail travel and Scotland has shown great imagination in taking part in that revival.”

Rail romance was described neatly by Paul Theroux in his classic The Great Railway Bazaar: “Anything is possible in a train: a great meal, a binge, a visit from card players, an intrigue, a good night’s sleep, and strangers’ monologues framed like Russian short stories.” Of these attractions, it will be the second last on that list that Caledonian Sleeper travellers will be anticipating.

• The first sleepers on British trains consisted of stretcher-like contraptions resting on facing seats in train compartments.

• In 1873, the North British Railway introduced seats that converted into beds. No bedding was supplied for this service - though an extra seat was provided for servants.

• In 1874, the Midland Railway imported the first Pullman sleeper carriages from the US. Travellers slept parallel to the direction of travel, in curtained-off cubicles.

• Separate cabins were introduced in the 1890s. Passengers slept crosswise to the direction of travel - as is still the case today.

Source: The Railways, by Simon Bradley