Imagine a verdant, landscape filled with rice paddies, tea plantations and olive groves. A land where you can hike up mountains in the thick mist of the morning and picnic by waterfalls on sun-weathered rocks in the afternoon. A land filled with golden apricots that taste like honey, peaches so succulent you barely notice the sweet juice that runs down your chin, and small black figs, firm and velvety to the touch, that erupt with jammy stickiness as you tear them open. I enjoyed all of these delights and more when I travelled through Iran in search of the secrets of the Persian kitchen.

On my journey, I cooked and feasted with Iranians of all walks of life who welcomed me into their homes to share their favourite recipes. In a country most commonly viewed through the narrow prism of its politics, food is a wonderful vehicle for discovery. A really good meal is something everyone can relate to.

Those unfamiliar with Iranian food often assume that it is fiery or spicy, perhaps befitting the country’s climate or politics. But it is, in fact, gentle and soothing, a poetic balance of subtle spices such as dried limes, saffron and rosewater. Slow-cooked stews, known as khoresh, and elaborate rice dishes layered with herbs, vegetables, nuts and dried fruit are the bedrocks of Persian cuisine, creating a dazzling mosaic of scents, textures and colours at the dining table. Regional and seasonal delicacies are plentiful, making the most of Iran’s bountiful produce.

Traditional dizi stew is made to an ancient recipe. Photograph: Kaveh Kazemi/Getty Images

My journey started in Tabriz, in north-west Iran, a place of culinary connection for centuries, a trading crossroads connecting the Caucasus, the Middle East and Europe. Tabriz was one of the capitals of the old Persian empire, famed for its bazaar, where spices from India and China were sold alongside delicate silks and intricately patterned carpets.

Today, the bazaar is a Unesco world heritage site and nearby is one of the best places in town to sample to city’s signature dish, kofte tabrizi. Shariar Traditional Restaurant (corner of Tarbiyat Street, +98 41 554 0057) is converted from one of the city’s old hammams, and the lamb meatballs are the size of your fist, stuffed with hard-boiled eggs, walnuts and dried plums. They are served in a tomato and saffron sauce that’s mopped up with warm flatbreads.

Tabriz also has some of Iran’s most comforting street food. I was shown around town by psychology student Yasamin Bahmani, who took me on a stroll around El Goli park with its famed Persian garden, insisting every few hundred metres that we stop at one of the street stalls that line the paths. We feasted on mashed potato and hard boiled eggs, smothered in thick slabs of melting butter, sprinkled with dried mint and wrapped in a warm flatbread, and tender steamed purple and yellow beetroot that we sprinkled generously with sumac.

A man roasting corn at a street stall in Darband. Photograph: Amos Chapple/Getty Images/Lonely Planet Images

Heading south, I hit the coast of the Caspian Sea and the rolling green hills of the Gilan province, famed for its river fish and caviar. The cuisine of Gilan is as green as its landscapes, making it the best place in Iran for vegetarians. Aubergines and garlic appeared at every meal, alongside the mounds of fresh coriander, parsley and dill that are used to create fragrant bases for stews and emerald-green kuku (a type of frittata).

I spent an afternoon with farmer Roya Baighi, who taught me how to cook torshi-tareh, an elegant green stew made from herbs we picked from her garden. It was bursting with flavour and virtuosity. Gilan is also home to one of Iran’s most famous dishes: fesenjoon, chicken poached in an earthy sweet-and-sour sauce of ground walnuts and pomegranates. I enjoyed it at Mahtab restaurant in Lahijan (Golestan Square, +98 141 422 2963), with white rice and crisp, buttery tahdig, the golden saffron-infused rice crust that Iranians prize so much.

This atmospheric restaurant celebrates Gilaki culture with a menu of regional dishes, and live folk music in the evenings. It is adjacent to one of Gilan’s most popular tourist attractions, Lahijan lake and promenade, which are a perfect spot to walk off any overindulgence.

The Koohpayeh restaurant in Darband.

No trip to the region would be complete without sampling koloocheh, small pastries stuffed with ground walnuts, cinnamon and cardamom which are the speciality of Fuman, a small town in the south-west of the province. Stalls all over town sell these baked treats and they were particularly welcome, washed down with dainty glasses of black tea, after a rigorous hike in the surrounding hills.

I visited saffron farms, rosewater festivals and pomegranate orchards – finding the history behind evocative ingredients

Tehran is filled with upscale restaurants serving dishes ranging from sushi and frozen yoghurt to dizi, a lamb, chickpea and potato stew made to a centuries-old recipe, cooked in a clay pot for several hours until the meat is so tender it can be mashed into a paste with a fork. The best local feasting, though, is in Darband, a neighbourhood in the north of the city at the foot of the Alborz mountains. It’s a district of narrow winding mountain paths lined with trees adorned with fairy lights. Koohpayeh restaurant is about a 10-minute walk up the Darband hill and provides a scenic backdrop for sampling some of the city’s finest juicy lamb kebabs. Finish the night by relaxing on faded Persian carpets in one of the many small wooden pavilions up and down the road and join the locals in smoking some apple-flavoured shishas.

In central Iran, I visited saffron farms, rosewater festivals and pomegranate orchards, discovering the history and horticulture behind Iran’s most evocative ingredients. The pomegranate is indigenous to Iran and, in ancient Persian mythology, the hero warrior Isfandiar is said to have eaten its seeds and become invincible.

Yasmin Khan choosing pomegranates, the nation’s favourite fruit, at an Iranian market. Photograph: Shahrzad Darafsheh

Today, pomegranates retain their near-mythical status and are revered as the nation’s favourite fruit. As well as being enjoyed on their own – their scarlet seeds sprinkled with a pinch of golpar, an earthy, citrussy spice – they are also salted, dried and pounded into fruit leathers or cooked into molasses to be added to savoury dishes.

Central Iran is also home to the country’s finest pistachios, which feature in both sweet and savoury dishes

The city of Shiraz is synonymous with poetry, and with the roses that flourish in the town’s famed garden, Bagh-e Eram. Roses are indigenous to Iran and it was here that the petals were first distilled into rosewater, over 2,500 years ago. Today, this is mainly used in desserts such as faloodeh, an aromatic and refreshing rosewater and lime sorbet with frozen vermicelli. The Hafez garden is one of the best places to sample this local speciality and I was taken there by Shahin Hojabrafkan, a handsome and charming secondhand car salesman. We sat overlooking Hafez’s shrine, squeezing wedges of lime into our fragrant rosewater-infused sorbets and watching a steady stream of Iranians pay reverence to their most cherished poet.

Central Iran is also home to the country’s finest pistachios, which feature in both sweet and savoury dishes. My favourite way to enjoy their creamy texture is at one of the many ice-cream parlours in the ancient city of Isfahan at night, such as Mahfal ice-cream on Makineh Khajoo. One of the most moreish is bastani akbar mashti, a saffron and rosewater custard ice-cream flecked with toasted pistachios.

Waiter serving lunch, Iranian style. Photograph: Jason Edwards/National Geographic/Getty Images

The final stop on my travels was the southern port town of Bandar Abbas, on the Persian Gulf. Bandar, as it is known, is a town of scorching sunshine, warm blue waters and towering palms, and was once an important post on the spice route from India to Europe. By contrast with the rest of Iranian cuisine, the food of this region is an assault on the senses – a thrilling mix of Persian, Indian and Arabian flavours. Tropical fruits, such as mangos, pineapples and guavas, are picked green and used for Indian-style pickles, and seafood from the warm Persian Gulf is stewed, grilled as kebabs, or fermented, dried and ground into powders and pastes.

The best place to sample the day’s catch is at the fish market, where burly men shout their deals of the day and women crouch on the floor next to them, deftly shelling prawns. Next to the market, a row of fish restaurants serve specialities including ghaleyeh maygoo – a prawn, fresh coriander and tamarind stew – and small spicy fishcakes called kuku-ye mahi, .

Travellers in Iran are always met with warmth and hospitality: it is not uncommon to be invited to an Iranian home for dinner after just exchanging a few pleasantries. For those wishing to expand their culinary knowledge, or simply enjoy one of the most sophisticated cuisines in the world, Iran offers a wealth of culinary delights. The only challenge for most visitors will be squeezing into their jeans at the end of the trip.

• Yasmin Khan is author of The Saffron Tales: Recipes from the Persian Kitchen (Bloomsbury £26). To buy a copy for £21.32 including UK p&p call 0330 333 6846 or visit bookshop.theguardian.com