A little-known art and heritage tax exemption scheme designed to allow the British public access to a treasure trove of privately owned works has been described as a racket in urgent need of reform.

More than 115,000 works, from Roman busts to Titians and important Pre-Raphaelites, are listed on a rarely publicised database operated by Her Majesty's Revenue and Customs (HMRC). Their owners are exempted from paying inheritance tax or capital gains tax on them as long as they allow public access.

But a Guardian survey of the scheme reveals that it is patchy at best. While no owner refused access to the works, it was far from easy to make arrangements: out of 30 cases that were contacted, only five appointments to view were made.

Labour's shadow culture minister Helen Goodman said she was not surprised by the results and had been discussing the scheme with museum leaders. "There is a consensus that we do need to do more to extend access to these objects. It's a bit of a racket, really. We need to strengthen the obligations on people."

Under the scheme, inheritance tax can be deferred if an owner commits to keeping significant works in the country and makes them available for the public to view.

Goodman said the last condition "is an incredibly small obligation", with owners having to make an item available for no more than 28 days a year.

"You could open your house and make it genuinely available for 28 days, but people don't do that," said Goodman. "It is not done on a systematic basis, so that's why I think – and I know – that what people do is they either respond to one-off requests or they say: 'Oh yes, you can come and look at it on Tuesdays and Wednesdays from November to February'.

"They make the times really awkward to keep people out. The scheme is not working, the obligations are not strong enough and we need to strengthen the obligations."

The scheme has cost the public purse more than £1bn in lost revenue in the last few decades. In a series of parliamentary questions, Goodman has discovered the scheme is costing £30m a year in lost inheritance tax, or around £900m over the 30 years it has been running. Prior to 1975, nearly 45,000 items were exempted from what was then estate duty; the Treasury estimates that cost at £250m. The total figure is £1.15bn.

"We have all paid for it because it's tax foregone," said Goodman. "We're talking abut 115,000 items, worth £1bn, which are not being seen by the public. That's a hell of a lot. I was completely taken aback by it."

The database can be found on the HMRC website in the tax-exempt heritage assets section. The search facility allows interested would-be viewers to type in, for example, Van Dyck, and see 203 items – mostly "school of" and "after Van Dyck", but also nearly 50 works by or attributed to him.

Type in Rossetti and eight works come up, although the listings contain inaccuracies. They list an 1867 work A Christmas Carol as being: "On loan to the Lady Leverhulme [sic] Gallery, Port Sunlight, Merseyside for 150 days once every 5 years. Currently on loan there."

In fact the painting was on long-term loan at Lady Lever – its correct name – gallery from 2001 until September this year. It was sold at Sotheby's for £4.5m earlier this month.

The accessibility survey reveals more holes in the scheme. On 21 occasions reporters from the Guardian left details for owners or their representatives to respond, but they did not.

Others who did respond made it difficult for the works of art to be viewed. "You will need to send me a written request to the address below and provide me with a certified copy of either your passport's photo page or your photographic driving licence," said the representative of one owner.

Many items were in storage at addresses other than those listed on the HMRC website, and some had even been misplaced altogether.

One man listed as a contact on the HMRC website for a painting said he did not know which of his clients owned it and told the Guardian reporters to look for the owner themselves.

Another was completely unfamiliar with the piece of furniture being enquired about and said the business listed as the owner on the HMRC website had been taken over.

Another agent acting for a family that owns an engraving by Benjamin Smith had not realised he was listed on the website as a contact.

No one contacted by the Guardian refused access, although there were awkward obstacles. The owner of a 19th-century harvest scene suggested the request to view the work could be "fronting up a burglary parade", although he did give an email address and said a viewing might be possible in the new year.

It was not all negative. The owner of three 18th-century aristocratic portraits, including one of Lady Newport and one of Anne Catherine Ward, was very helpful, although she asked to remain anonymous.

Portrait of Anne Catherine Ward, whose owner said it was a joy to share the work with the public. Photograph: Private owner

"I get several calls a year, but it's variable," she said. "It depends on who's doing what kind of research. I hear mostly from graduate researchers and museums; people who are doing their own family histories, as well.

"There isn't museum space in the UK for all these things. They're cared for. I give all the information I've got and they give me information back. For me, it's a joy – a great voyage through a particular period of history."

Goodman said she would like the scheme to function better and is proposing that the number of days on which owners are obliged to allow access should be doubled to 56. She also believes there should be a requirement to lend works to public museums and galleries, should they be wanted. Photographs of all the works ought to be put online through the British Library's digital space project, she added.

"The scope and the possibilities, when I talk to the professionals, seems to grow, really. This is hidden art and we're trying to open it up to the public."