Solar farms must not spread unrestricted across the British countryside and become as controversial as onshore wind turbines, a minister warned on Friday.

Instead, solar panels will be rolled out on millions of homes, businesses, schools and government buildings, said energy and climate change minister Greg Barker.

Barker, one the greenest Conservative ministers, launched the government's first solar power strategy at the end of a week in which senior Tory sources revealed their plans to heavily curb or even dismantle windfarms after the next election.

The number of large solar farms, often housing more than 100,000 panels, has doubled in the first three months of 2014 but some have attracted local protests.

At the opening of a new Sunsolar panel factory in Birmingham, Barker told the Guardian: "I do not want solar farms to become the new onshore wind. Solar power enjoys huge popularity, so we have to be careful. I do not want to see unrestricted growth of solar farms in the British countryside."

Barker has previously called large farms "monsters" and, due the subsidies they receive, "gold-diggers".

But he said there was a great opportunity elsewhere for solar power, which has fallen in price by two-thirds in the last four years. Barker said he expected the 500,000 homes with solar panels to double by the end of 2015.

He also announced streamlined planning rules to make it simpler to put panels on large industrial and commercial roofs. Putting panels on just one in six roofs would generate electricity equivalent to two nuclear power stations. On Thursday, carmaker Jaguar announced the completion of the UK's largest rooftop solar array, with 21,000 panels on the roof of their engine factory in Staffordshire providing a capacity of 5.8MW.

Barker also announced a programme to put up to 4m solar panels (1GW), on the roofs of government-owned buildings across the UK before the end of the next parliament in 2020. A new unit in the Cabinet Office, under Francis Maude, is working on this programme, which will be funded by the private sector.

Another new programme aims to put solar panels on the roofs of all of England's 22,000 schools, which currently spend £500m on energy every year.

Local authority housing fitted with solar panels in Newport, south Wales. Photograph: Alamy

Barker said he wanted to see 20GW of solar power in the UK by 2020, although the current government forecast is for 12GW. "My 20GW is predicated on the ability of the industry to drive costs down to give grid parity [the same price as electricity generated from conventional sources] by 2020. That is eminently possible," he said. "But we will not get there with high subsidies."

Solar power capacity has doubled in the UK in the last two years to about 4GW, although it remains far behind Germany's 35GW.

"It's encouraging to see the prime minister's Conservative colleagues promoting a positive solar vision for the UK and as a solution to climate change, particularly when it comes just days after Cameron took a swipe at onshore windfarms," said Friends of the Earth executive director, Andy Atkins.

Paul Barwell, chief executive of the Solar Trade Association, welcomed the strategy: "With the Royal Society, the UN Intergovernmental Panel on Climate Change and even Shell anticipating solar could be the world's biggest energy source, the UK needs to make the most of its R&D, product design and manufacturing skills to steal a march in the global clean energy race."

Ray Noble, co-chair of the solar strategy board that advises ministers, said solar power could play an important role in the UK's energy security as old power stations are closed in the next few years. "If we have built the solar equivalent of a big power station in the two-three years by 2015, and can do so again, politicians are going to have to pay attention."

Other industry figures tried to play down the negative impact of large solar farms on the popularity of the technology with the public. Reza Shaybani, chairman of the British Photovoltaic Association, said: "There are a very, very few projects sited in the wrong place, that spoil it for the rest."

Farm buildings and land host about half of the UK's solar panels at the moment, and National Farmers Union renewable energy adviser, Jonathan Scurlock, said: "There is a substantial difference between wind turbines, which are very visible – and like Marmite, you love them or hate them – and solar farms which are very subtle when well-sited."