At dusk we sat on our little rock, watching the lights coming on in distant houses across the water. We sipped at our tots of rum, peering out at phantasmal blurs that solidified for an instant into late-evening kayakers before vanishing once more into the gloaming. When, at length, we went to bed, we fell asleep to the plangent eep-eep-eep of oystercatchers skimming the gentle waves below us. This is not, it is fair to say, the sort of soundtrack one expects when visiting a capital city in Europe.

Glide into Oslo by train and, just before you reach the central station, you'll glimpse an array of tiny islands apparently bobbing about in Oslofjord. They're an enchanting sight with their miniature beaches, diminutive woods and brightly coloured little cabins, all just a hop and a skip from the city. I'd passed this way many years before and had thought then how heavenly it must be to live there.

So it was rather pleasing to find myself cruising across Oslofjord with a companion to one of those islands: Langøyene. Admittedly, we would be camping rather than inhabiting one of those charming cabins, but we really couldn't fault the price: it was free. Yes, that's right, our accommodation in a city routinely listed among the five most expensive in the world cost us nothing at all.

What's more, there's a frequent ferry service to Langøyene during the summer that takes about 15 minutes, making it eminently feasible to stay on the island and visit the city every day. I say island, but Langøyene – which means "long islands" – is actually two narrow isles that were joined together by (and here comes the romantic bit) a rubbish tip that was created between them. Thankfully, the dump has long since been grassed over and landscaped: an extra beach was created at one end, and the island is now dedicated to summertime fun.

When we arrived, however, Langøyene appeared to be populated by barnacle geese alone, with a huge flock happily grazing the verdant plain. There's no campsite as such – visitors just set up camp in little glades on the western side.

Having pitched our tents on a secluded outcrop near the water's edge, we set out to explore our new domain. We made a long, languorous circuit, following informal footpaths through woodland. Here we did, at last, see evidence of other people, or at least their tents. The tour took us a couple of hours, including stops for stone skimming and the investigation of a lighthouse scarcely taller than ourselves. We didn't take full advantage of the naturist beach, but we did wish we hadn't come too early to sample the delights of the island's summer-only bar, for here they serve an unexpected soupçon of city sophistication: the Langøyene Iste or "long islands iced tea". See what they did there?

Langøyene is the furthest-flung of half a dozen islands at the city end of Oslofjord. The remaining five are served by two ferries that spend all day doing little circuits of them so, frankly, it would be rude not to do some island-hopping.

The route 93 ferry calls first at diminutive Bleikøya – it's worth jumping off here for a wander around the exquisite wooden cabins where wealthier Osloites spend their summers. But greater treasures await at Gressholmen. Very sparsely populated and mainly given over to a nature reserve, this is a place where you can flop on to a beach or lose yourself among the woods and wild pansies with no inkling whatsoever that Oslo city centre is just two miles away. And, yes, those are wild spring onions growing everywhere. There are a couple of loos and, if you haven't brought your own victuals, a cafe open from noon until late (except Mondays).

Next stop is Lindøya, where the 93 will drop you off at its eastern end, leaving you to saunter on footpaths and tracks across the car-free island (again served by a cafe). Here you can marvel at the numerous hamlets of cabins and their proud owners tending their tiny gardens, before arriving at the western end to pick up the 92.

Those eager to see everything will want to stop off at Nakholmen, a smaller version of Lindøya. Otherwise, stay on the boat to Hovedøya ("main island"), a popular picnicking and bathing destination and possessor of a historical gem: the remains of a 12th-century Cistercian monastery founded by Lincolnshire monks. The beautiful barn of a cafe right next door makes for a perfect final watering hole before you head back to the hubbub of the city.

Take in some free attractions

While Frogner Park, with its weird and wonderful Vigeland sculptures, remains Oslo's best-known repository of open-air art (and is free), it now has a rival on a hillside overlooking the harbour. Opened last year, the 63-acre Ekebergparken (Kongsveien 21-23, ekebergparken.com) was a gift to the city from a local beer magnate and is packed with works by internationally renowned artists such as Dalí, Rodin and Bourgeois. Not to be missed is a disorientating piece by James Turrell (Sundays only, 12pm-6pm) in which visitors enter a tunnel beneath a reservoir to look into what is apparently infinite space. The park is just a few minutes' climb from the city centre on tram no 18 or 19, so grab a picnic and spend the afternoon there.

Pick up Visit Oslo's pocket guide for full details of the capital's free attractions, the cream of which are the Akershus Fortress Visitors' Centre (Bygning 18; nasjonalefestningsverk.no), the Oslo City Museum (Frognerveien 67; oslomuseum.no) and DogA, the Norwegian Centre for Design and Architecture (Hausmanns gate 16; doga.no). And rather than pay to get into the Munch Museum, see some of the troubled artist's paintings for free at the old University Aula on Karl Johans gate (summer only).

Meanwhile, jazz fiends should come during the Oslo Jazz Festival (11-16 August; oslojazz.no) when musicians spill out of the venues on to the streets to play for whoever's in earshot.

However, one of the most satisfying year-round free activities can be enjoyed at the Norwegian National Opera and Ballet (Kirsten Flagstads plass 1; operaen.no). The unconventional roof of the opera house dips all the way down to the fjord so, starting off from the lapping waters, you can walk right up it on to the top of the building to take in a view of the city.

Find a great cheap bar

The Underwater Pub (Dalsbergstien 4; underwaterpub.no) may not actually be underwater (it owes its name to its sub-aqua decor) but it does allow you to bring in your own food. Alternatively, you can ask the bar staff to get one of three local takeaway joints to deliver a meal to your table. For a truly joyous experience, go on a Tuesday or Thursday and nurse your half litre of the local Ringnes beer (about £7 – not bad for Oslo) while professional opera singers perform choice excerpts for your listening pleasure. Most of the action takes place at the top of the stairs of this split-level pub, so get there in the early evening to secure a table with a good view.

Stay in a cool hostel

The 60-room Oslo Hostel Central (Kongens gate 7, +47 23100800, hihostels.no) opened just last summer but has already been rated the second-best hostel in the world by Hostelling International backpackers. It's no wonder, either. Housed in an elegant red-brick end-of-terrace building and located in the heart of Oslo between the city's main drag and the Akershus Fortress, it offers private rooms as well as dorms, friendly multilingual staff on reception, and a decent breakfast to fill up on. The rooms are informed more by a stylish minimalism than any attempt to thrill, but are finished to such a high standard that the "s" in "hostel" becomes all but silent.

• Dormitory beds from £36 a night; private doubles from £79, breakfast included

Eat out on a budget

Lurking in the less-than-trendy St Hanshaugen district, Restaurant Schrøder (Waldemar Thranes gate 8; restaurant-schroder.no; +47 2260 5183) may be off the tourist trail but the eatery boasts a devoted local following drawn to its traditional Norwegian menu and, in particular, its cod and mackerel dishes (when in season). Grey lino, red-check tablecloths and huge sepia paintings of old Oslo give the place a stuck-in-the-1950s feel and, in Norwegian terms, the prices are, too – evening main courses are £11 to £15, with breakfast or lunch between £7 and £8. And if you're lucky, the diner at the next table might be crime writer Jo Nesbø, who likes the place so much he's also made it the favourite hangout of his fictional detective, Harry Hole.

Go on a culture binge

Sold in 24-, 48- and 72-hour form, the Oslo Pass not only offers free travel on all public transport (including ferries), it also gets you into more than 30 attractions, outdoor swimming pools and walking tours, and offers discounts at restaurants, shops and on concert tickets. Although not especially cheap (24-hour £29, 48-hour £42, 72-hour £53 – half price for four-15 years and 67+), it allows you, with clever planning, to gorge on a whole host of attractions and save yourself a wad of kroner. Highlights include the harbour-front Astrup Fearnley Museum of Modern Art (Strandpromenaden 2; ), the Holmenkollen ski jump (Kongeveien 5; holmenkollen.com) and the Popsenteret (Trondheimsveien 2; popsenteret.no), the surprisingly brilliant interactive museum of Norwegian pop (more than just A-ha, it turns out). The pass is available from tourist information centres, hotels, hostels, campsites, and some museums.

Get around cheaply

If you don't want to buy an Oslo Pass, grab a Zone 1 season ticket (24-hour adult £9, four-19 and 67+ £4.50; seven-day adult £23, four-19 and 67+ £11.50) valid on all buses, trams, trains, metro and ferries (except to Bygdøy), on all of which an ordinary single would set you back £3. They're available from kiosks and shops, ticket machines and bus terminals.

• Rail travel was provided by Voyages-SNCF (0844 8485848, voyages-sncf.com, which has returns from London to Oslo (via Eurostar) from £336, including couchette from Cologne to Copenhagen. The route 94 ferry sails between Vippetangen (Oslo) and Langøyene from 7.20am–9.20pm until 7 September, 92 and 93 ferries sail frequently between Vippetangen (Oslo) and the islands until 19 October. Fare and timetables from (ruter.no)