When he’s lying in the bath on the top floor of his house in Edinburgh, architect Richard Murphy is in a good position to survey the surrounding scene. The skirting board slides back to reveal a peephole slot down into the living room, a motor grinds into action and the ceiling opens up to the sky, while another lever is pulled and the entire corner of the building swings open, exposing his bathing body to the street. In this slipping, sliding transformer of a home, awarded House of the Year 2016 by the Royal Institute of British Architects this week, you have to be careful where you tread.

“I think I missed my vocation as a caravan designer,” says the portly 60-year-old architect, as he deftly slides along his bookcase on a steel ladder, before flipping open a secret hatch in the staircase to pick up the day’s post. On a six-metre deep site at the end of a sloping Georgian terrace in Edinburgh’s New Town, he has concocted a three-dimensional spatial riddle that speaks of a lifetime of coming up with ingenious solutions to tight historic contexts, carving out a richly layered world that revels in its playful architectural tricks. A friend of Murphy’s has dubbed it the “Rubik’s cube house”, but it feels more like climbing through a game of snakes and ladders: one wrong move and you could be catapulted out on to the street.

Window on the world... one of the Murphy House’s many openings. Photograph: Keith Hunter

The fun begins on the outside where, to eyes accustomed to the neighbourhood’s 19th-century townhouses, the building might not quite look finished. The corner of the house appears to have been unzipped, with big gaps between the stone blocks like missing teeth, an effect that becomes more pronounced at night when these gaping holes glow from within. A rubble wall – traditionally used for the cheaper, unseen facades around the back of the neighbouring terraces – runs along the side of the house, seemingly standing free from the building (the trick of a mirror, it turns out). The stone walls give way to layers of lighter materials, as if the home had been built inside an existing ruin, with a collage of glass blocks and timber screens, glazing and steel I-beams, along which garage doors and window shutters roll on little wheels. Even the bits that don’t move look like they could, or maybe once did, as if the whole house had been composed from a kit of parts that might one day be reconfigured inside the stone shell. It’s a busy bricolage of bits and bobs – and a bit of an eyeful.

“I suppose I’m a bit of a maximalist,” says Murphy, whose 20-strong practice has worked on everything from stealthy private homes to university campuses, and has just completed an extension to the first Carnegie library in nearby Dunfermline. “I try to wring every maximum possible opportunity for architecture out of a site. I freely admit that perhaps this house is ‘over-designed’.” It was too much for the planners, who recommended refusal, but were overturned by the committee, perhaps mindful that another one of Murphy’s award-winning mews houses had been turned down before, as the local newspaper pointed out with glee.

The outside might be odd, but the spatial sleights of hand begin to make more sense when you step inside. Entering the house, you’re greeted with a curvaceous wall of pink polished plaster, which leads you on to a narrow dog-leg staircase, lined with bookshelves and more mirrors (doubling Murphy’s decades-long collection of National Geographics), punctuated with little windows, framed through the gaps in the facade. The tight stair spills on to a multilevelled sitting-cum-dining-cum-kitchen space, a terraced tableau of the functions of living, disaggregated into their component parts, each pulled back into its own nook, framed with different-coloured plaster walls.

One of the Murphy House’s three bedrooms. Photograph: Keith Hunter

The walls don’t simply rise to the ceiling, but make their way there via a series of staggered ledges and shelves and clerestory glass panels, a measured sequence of layers and levels reminiscent of Frank Lloyd Wright’s interiors, always pausing to create little places for the body, or eye, to rest. Perhaps counter-intuitively for a tight site, the walls are castle-thick – up to 600mm (2ft) – but they are layered so as to make the rooms feel larger rather than more cramped, expanding to accommodate stairs and sliding shutters, or house a pulley to operate another hidden contraption.

The house, left, next to its neighbour, somewhat mirroring its stonework. Photograph: Keith Hunter

A big sliding glass door leads on to a small patio, designed as a homage to Murphy’s hero, the Italian architect Carlo Scarpa. It’s inspired by his garden at the Querini Stampalia in Venice, and uses exposed aggregate concrete walls and a frieze of metallic tiles sourced from the same manufacturer. Other key touchstones were the Maison de Verre in Paris by Pierre Chareau, a 1920s hymn to steel, glass blocks and mechanical fixtures, and the interlocking vertical spiral of rooms in Adolf Loos’s 1930 Villa Müller in Prague. In a similar vein, Murphy has somehow packed eight levels into the building’s four storeys, juggling three bedrooms, three bathrooms, a study, garage and the main living space in an interlocking puzzle, always prising open unexpected glimpses between the multiple levels.

Sir John Soane’s Museum in London, with its intricate universe of mirrors, views and pivoting walls, was clearly a key influence. As Murphy puts it, the project is “a quarter Soane, a quarter Scarpa, a quarter eco-house and a quarter Wallace and Gromit.” A lot of the complexity comes from the project’s long gestation: it won planning permission in 2007, but the recession put the brakes on, happily allowing four years of “fiddling about” with the design.

The result is more than just a collage of quotations and madcap inventions (including a soon-to-be-installed log-lift from the cellar). All the flaps and shutters create a home that feels truly responsive to the seasons and different modes of inhabitation. Citing another precedent, Murphy calls on Dutch architect Aldo van Eyck, who said that a house should be both a bird’s nest and a cave, an extrovert place in summer and a retreat in winter, which makes particular sense here. “In Edinburgh, we can have 20 hours of daylight a day or six,” he says, as the wintry leaden sky begins to fade to dusk. “The house needs to close down as much as open up.”

With that, he potters around the living room, pressing buttons and pulling at wires, and the space is instantly transformed from a light-flooded Scandinavian conservatory to a cosy timber-lined bothy.