Updated The United States' tax-filing software industry actively prevents search engines from discovering their free-filing versions, it has been discovered, adding further criticism to an industry that drives Americans toward unnecessary paid-for products.

Internet users, incensed at efforts by the tax filing software market to legally lock-in their business model, have been digging into the actions of Intuit and competitors including H&R Block and discovered that the webpages for their free tax filing software has code to stop search engines like Google from linking to or indexing to it.

And by that we mean, of course, nofollow and noindex HTML metatags, as well as a comprehensive robots.txt file, which are used by webmasters to indicate where they doesn't want search engine robots to look. Typically these mechanisms are used to stop search engines from accidentally gathering confidential or dynamically generated information. It is a sort of honor system that has been in place since the early days of the internet.

Except in this case, it is used to pull a veil across free filing websites in their entirety, meaning that it is harder for people to find the free service and so drive people toward paid-for software.

The source code on the TurboTax Free File site adds "noindex, nofollow" metatags to its HTML, and so erects a great big STOP sign to search engines. Whereas the website for the paid-for version of its filing software takes the opposite tack: "index, follow." H&R Block does the exact same thing: its free file software is hidden from search engines while its paid-for product is made as Google-friendly as possible.

The reason this is important is because the free filing software offered by the for-profit companies is a result of a formal agreement between the tax software industry and the Internal Revenue Service (IRS).

The IRS has agreed not to develop free software for its own systems so long as the software industry offers free versions for anyone earning under $66,000 a year or anyone receiving an income tax credit.

Tail wagging dog

Over time however, the power dynamic between the IRS – which has increasingly been starved of funds – and the software industry – which has grown rich from charging tens of millions of Americans every year to navigate the overly complex US tax system - has flipped.

That dynamic blew up earlier this year when it was revealed that a tax reform bill due to become law made it illegal for the IRS to develop software for its own systems. Previously it had been a voluntary agreement that the IRS was in charge of.

Following a public outcry, the IRS's general counsel said that his understanding of the new law is that the IRS can terminate the entire Free-File system and design its own direct-file product if its provides 12 months’ notice. But that assurance has failed to mollify critics who say the software tail has started wagging the tax dog.

The issue put a spotlight on the software companies' effort to direct people toward their paid-for products and it emerged that despite around 70 per cent of Americans being eligible to use the free products, just three per cent of them actually do.

That is entirely deliberate and the software companies go to extraordinary lengths to push people toward their paid-for products. One extensive report by ProPublica noted how TurboTax used deceptive design and practices to drive people toward paid-for products.

As just one example, how many of the following "editions" do you understand will offer free tax filing?

TurboTax Deluxe

TurboTax Premier

TurboTax Self-Employed

TurboTax Freedom

TurboTax Free

The answer is none of them. What you actually need to do it find the specific website for "TurboTax Free File" – the one that the company tells Google not to index.

Freedom isn't free. Nor is free

Intuit uses the name "free" and also loads its non-free product websites with SEO terms that someone looking to file for free will type in, in order to direct people to paid-for editions. Unless you land on the right webpage – the one for Free File – there is literally no way to find the TurboTax free edition; it will always loop back to a paid-for version.

Which is why Intuit and TurboTax go to such lengths to stop people from landing on the Free File versions of their software.

Free online tax filing? Yeah, that'll soon be illegal thanks to rare US Congressional unity READ MORE

The costs are not insignificant either: the cheapest option is typically $60 and goes up to $200. On top of that, the companies introduce all kind of additional charged-for services that are rarely needed by taxpayers, which range from $50 to $200, but are pushed using language intended to prey on people's fears.

And if you thought that was the extent of the software companies malfeasance, you'd be wrong: there are numerous examples of where the industry has actively lobbied to prevent lawmakers from simplifying the tax code out of concern that it would stop people paying for software to do the number crunching for them.

And, as one man who developed a free filing system for California came to understand, they are both relentless and unscrupulous when it comes to protecting their bottomline.

So what is the solution? For one, telling your elected representatives that you are aware of their efforts to turn what was a voluntary agreement into law, and are opposed to it, may strengthen some lawmakers' backs. But more immediately practical is to make sure that links to the free filing software are widely available on the internet.

That way, when people look for actual free tax filing software in future and type relevant terms into search engines, they are more likely to land on the actual Free File software, as opposed to the "Free" or "Freedom" editions. ®

Updated to add

The Reg has received the following statement from Intuit:

TurboTax offers two free tax filing products: TurboTax Free Edition and TurboTax Free File Program, which is part of IRS Free File. Our intent in implementing our search practices was to make clear the distinction between these products by educating customers so they could find the product they were looking for. We did this by generating original content and publishing answers to frequently asked questions about the IRS Free File Program and the TurboTax Free File product, and optimizing that content to rank highly in organic search. While we believe amplifying this content meaningfully informed taxpayers and contributed to IRS Free File growth, we recognize that our overall search approach may have made it harder for some customers to find a TurboTax Free File Program landing page. So we are undertaking a thorough review of our search practices to ensure we are achieving our goal of increasing eligible taxpayers’ awareness of the IRS Free File Program and its availability. We are pleased to see that the IRS Free File program grew 6% this year, and 11% over the last two years. We are also encouraged that during this tax season our TurboTax Free File Program donated nearly 1.2 million tax returns free of charge to taxpayers. Additionally, another 50 million taxpayers are eligible to file for no cost through TurboTax Free Edition.