It's not much bigger than a Wispa bar and it certainly doesn't glister. David Merry, who has been working with gold for more than 40 years, drops the ingot into my hand. It's deceptively heavy. It's also worth more than a Jaguar XF.

The Assay Office in the City of London is an intriguing blend of ancient and modern. At one station, workers sit engraving hallmarks into silver plate by hand; at another, they're scanning pieces of jewellery with an x-ray machine to work out their gold content. Nestled within Goldsmiths' Hall, the Assay Office's history dates back to 1327 when it was created to test the quality of the precious metals crafted by gold- and silversmiths.

Merry started out at the Assay Office as a 16-year-old apprentice. "We used to get through 18-to 20,000 items a day," he says, as we watch hallmarked items being packaged up for return to their owners. "But at the moment it's more like 10,000."

Since the gold price began its surge around 2005 – leaping from £253 an ounce to nearly £1,075 an ounce today – the Assay Office has been the best place to see the effect on the UK's jewellery market. The answer lies in a separate room, where heaps of gold items – earrings, necklaces, cigarette cases – lie in plastic trays, waiting for a man called Stefan to empty them into a very hot oven.

The Assay Office never used to smelt metal, but demand is high and they're now handling 150kg of scrap a week. Stefan dons a safety visor and empties the crucible into a mould, in one long, white-hot pour. "We get jewellers bringing in plastic bags full of unworn stock straight off their shelves," says Merry. "At least that way they can recoup some of their loss." He's been through recessions before, but he's never seen anything like this.

In the west, yellow gold has been falling steadily out of fashion since the Gerald Ratner years; now a combination of high inflation, low interest rates and economic downturn is making it more expensive than ever (gold prices reached an all-time high of £1,207 an ounce last year, and they're predicted to go even higher by the end of 2012). You can tell how dramatically our relationship with our jewellery boxes is changing from the fact that there's as much marketing persuading us to sell gold as there is to buy it.

The only yellow stuff in my portfolio is a trio of rings passed on from my grandma that neither she nor my mother liked or wore. I have kept up the family tradition, and they have sat inert in their boxes ever since, so I make my debut trip to a pawnbroker to have it valued. At the first one I visit, the young man behind the counter seems desperate to take my unloved fingerwear. "You've got a mix of carats here!" he exclaims, as if I've just presented a piece of Ming china at the Antiques Roadshow. "You don't normally see this!"

He promises to match any price I find on the high street, although subsequent research trips to the H&T Gold Bar and the Money Shop down the road prove that my rings are worth a total of £160 whatever scales they were placed on. But what I really want to know is – what's to become of my rings after they've left my venal little hand? The latest global gold figures show that 50% of all the gold in the world exists in jewellery. But if it is so expensive, who exactly is wearing it?

Andrew Hinds, director of F Hinds, is the sixth generation to run the family business. He never imagined a day when customers would come into his high-street shops and ask to sell their gold as they could at pawnbrokers. "We didn't do it for a long time," says Hinds, whose company has more than 100 branches in England and Wales. "That sort of thing was all a bit backstreet, and people didn't want to be associated with going into pawnbrokers'."

Heavy metal: top necklace by Kenneth Jay Lane (kabiri.co.uk). Right arm: all Amanda Wakeley for Ernest Jones (ernestjones.co.uk). Earrings, lower necklace and left arm, all stylist’s own. Photograph: David Coates

Now gold-buying is a business the high street has good reason to be grateful for. The World Gold Council reports that in the US, whose market is considered a little ahead of ours, there is now as much gold being recycled as there is being sold – and that the UK is not very far behind. If it wasn't for the profits they can make on scrap, many independent jewellers would struggle to be in business at all.

It's a far cry from the bling of the 80s, when yellow metal adorned the arms of Sloanes and street rappers alike. Gold has, of course, been a status symbol and a signifier of wealth since it was minted for coinage in the 7th century BC, and as a commodity its value has always fluctuated violently, depending on the economic environment. But its accessibility as an adornment increased dramatically in the mid 19th century when new gold standards were introduced by the Assay Office, allowing goldsmiths to work in caratage lower than 18. During the Second World War, the British government allowed caratage to be dropped to nine, to maintain production of wedding rings, considered an "essential wartime item"; by the late 20th century the high-street jewellers was awash with nine- and 14-carat jewellery, made at low manufacturing costs and sold relatively cheaply.

Today, those simply made items – like chains and signet rings – can be some of the most difficult to shift. "Where the cost of the item is mostly in the weight of the gold, we're selling fewer," says Hinds, who has seen more significant changes in the market in the past few years than in the previous 20. "People are becoming much more flexible about what they'll wear. Customers used to come in knowing what kind of metal they wanted. Now, if they can get the look they want in silver, why pay the price of gold?"

It's only 10 years since major supermarkets were flogging impulse purchases to mums doing the weekly shop; now that it's physically difficult to make something in gold that meets consumer expectations for even a couple of hundred pounds, mass-market appeal is dwindling. But it's not only the lower-carat market that has been affected: the same concerns are reflected at the luxury end, too.

Hannah Martin founded her eponymous brand in 2005; she is now one of Britain's most influential contemporary jewellery designers. "My first two collections were all 18 carat – I refused to do anything but gold," she says. "But a piece that was £2,000 when I started out is now £8,000 – and that's with us cutting our margins, doing as much as we can to keep the price down."

"It's a really difficult situation," she sighs. "There are certain designs I can't produce any more in gold – we've had to move down to silver." As her client base has moved upward, Martin's response has been to launch a new, lower-cost line, called Hannah Martin Show. It remakes some of her most famous designs – such as her sleek "eagle icon" and "spur" rings – in brass, powdercoated in vibrant colours using an industrial technique commonly used on bike frames.

To find out where my rings might end up next, I head to Birmingham's jewellery quarter. In the centre of the district – whose warehouses have, since the decline of the manufacturing industry, been largely employed for trendy loft living – are the offices of Cookson Precious Metals, one of the leading providers of jewellery supplies in the UK and Europe.

In the front office, there's a trade counter where scrap is assessed and bought. But the real business happens out the back, where cheerful men oversee extraordinarily hot furnaces and melt the gold down into bars. That's then shipped to Europe, where it's refined, and returned to the supply chain as pure gold. If not to jewellers, Cookson might sell it on for industrial use, or to technology companies (gold is useful in smartphones, in catalytic convertors and even in cancer treatments).

Asa Harrison, its UK sales director, points out that jewellery was already being overtaken in the marketplace as a "disposable spend". "If you're going to buy a present for your girlfriend or your wife, you might buy her an iPod or an iPad now. But without doubt the gold prices have made the market more difficult." The trade press regularly advertises retirement sales as independent jewellers cash in their old stock for scrap.

The evidence is there in the windows of the remaining Birmingham jewellers. Where gold necklaces and bracelets were once piled high on black felt, the displays are now more dominated by branding boards and marketing images – tennis players sparkling in Swarowski crystals, James Bond glowering over his watch – than they are with product. There's also the ubiquitous Pandora, a brand that can justifiably claim to be the saving grace of many an independent jeweller through the recession. With their concept of "individual" jewellery – customers create their own necklaces and bracelets using beads and charms – Pandora introduced a seasonal fashion element that allows a lower spend and encourages repeat purchases. "Because the beads are silver, you're buying real jewellery, but because they start at £40 they're an affordable payday treat," says Laura McCreddie, managing editor of Retail Jeweller magazine, who points out that it couldn't have come along at a more opportune time for the industry. "Where people used to come into your shop for considered, high-value purchases, now you have women coming in month after month to fill their bracelet."

Across the road from Cookson is Birmingham City University's School of Jewellery. Gay Penfold, who manages its Jewellery Industry Innovation Centre, points out that white metal remains far more popular than yellow. "What we are finding is that people are combining metals more," she says, "using titanium with silver, or silver with gold." Meanwhile many designers are experimenting with so-called "contemporary" metals such as tungsten and cobalt, or palladium, a precious metal which looks like platinum but comes in at a fraction of the price.

"It gets harder and harder every year for people starting out," says Hannah Martin. "The jump in prices has been so rapid that unless you have outside investment it must be impossible to make a full collection in 18 carat."

If my rings do make it back into a piece of jewellery, it sounds unlikely they're going to be launching a new young designer's collection, or swinging round the necks of two teenage BFFs with matching heart lockets. So who is wearing gold?

I ask David Lamb, managing director at the World Gold Council (WGC), the body responsible for promoting the industry's interests around the world. Lamb tells me that the world's gold jewellery market is heading in two directions: "upwards and eastwards".

The latest WGC figures show that more than half of all the world's gold jewellery is now bought, sold and consumed in two markets: India and China, where it's traditionally sold in 22 and 24 carat respectively, and considered an investment purchase as much as a fashion one. "If you ask anyone in those countries the gold price, they can tell you within 5%. They're constantly aware of it – they know it the way you know the price of a pint of milk."

But the high price of gold has also prompted something of a renaissance in the west at the luxury end of the market, where a purchase is seen as both proof of wealth and as an investment. A new bling, if you like. "There's a lot more yellow metal in Tiffany than ever there was," says Lamb. There's also a trend in statement cuffs – a sort of man-bangle you wear opposite your designer watch – while a series of gold lockets created by New York designer Kim Kaufman, that start at $14,000 and go up into the hundreds of thousands, have become so popular that she can't keep up with demand.

Perhaps the most significant effect of the new gold rush has been the creativity it has inspired in recent years as designers are forced to work with the metal in innovative ways, and to find ways of creating visual impact using considerably less of their raw material. "Designers are creating lighter and more airy jewellery," says Carmen Borgonovo, senior style editor at Harper's Bazaar. She cites Gucci's new Diamantissima ring, launched at the Basel jewellery fair last month, as the perfect example – an intricate band whose woven design limits the weight of the gold and allows it to be sold at £648. Other trends on show in Basel included gold lace that mirrored the spring/summer catwalk trend, and a boom in smaller, lighter accessories for everyday wear.

Australian designer Jordan Askill has been leading the way in this new style of "personal jewellery" with his sentimental heart rings. "It's been kind of annoying but good at the same time," says Askill. "We've had to be resourceful, and because these designs are so fine and delicate and the wirework's quite open I don't use that much gold. It's been able to be quite wearable and accessible."

Even some of jewellery's most intractable "rules" have been jettisoned: it's no longer considered infra dig to set diamonds in silver, or to mix silver and gold. In fact, gold's value is so high that it's started to be incorporated in designs as a jewel in its own right.

"Whether you're designing at a low or a high level, you have to think your way around the price issue," says Martin, "either mixing metals or finding new metals or a new way to construct things. You've got to think creatively just to stay in the game."

Harrison, at Cookson, believes that the high street, too, has enjoyed a makeover thanks to the high price of scrap, which makes it easier for retailers to refresh tired collections and moribund windows. "The average independent retail jeweller used to turn over their stock every 12 months," he says, "but now they'll update their stock several times a year."

From an environmental and ethical point of view, gold's repositioning as a premium purchase couldn't be better. Gold mining is a damaging, invasive and costly process, so treating it as a rare resource – and recycling it until the market reaches self-sustainability – can only be a good thing.

"If you believe in a world where we're going to have relationships with fewer but better things, gold should be well placed," says Lamb. "Will gold return to its high-street penetration through mass-market jewellers? I doubt it very much. Will it increasingly be centre stage at Tiffany and in Bond Street? Absolutely."