Controversial landlord spells out why his housing benefit tenants had to go – and northerners must get on their bikes

Fergus Wilson, the buy-to-let baron with nearly 1,000 homes who has become the most controversial landlord in Britain in a generation, reckons he is out of pocket by £800,000 because housing benefit claimants have failed to pay the rent.

Wilson and his wife, Judith, have provoked fury since he told the Guardian earlier this week that he had sent eviction notices to 200 tenants who were on welfare. In the future, he said, he would not accept applications from people claiming housing benefit (HB), preferring migrants from eastern Europe instead.Wilson further inflamed public opinion after telling Channel 4 News "I feel sorry for battered wives who have come to us because we are very much consigning them to go back to their husbands to be beaten up again – but the situation is it cannot go on."

In a long and provocative statement sent to the Guardian, Wilson remains unrepentant. He repeatedly tells benefit claimants to "get a job"; suggests northerners should move south; says that eastern Europeans "have to work, so why not the English people"; and details how he evicted one tenant he suspected of benefit fraud after spotting two new cars on her drive.

Wilson also makes a direct appeal to David Cameron, urging the prime minister to stop paying HB to tenants rather than directly to their landlords, saying: "Calling Dave, Calling Dave, Fergus to Dave. Take a tip Dave! Stop this nonsense of paying housing benefit direct to the tenant!" He adds: "We have £800,000 arrears for those on HB. Has the £800,000 caused financial difficulty … to be honest not an awful lot, but it has some!"

Wilson also revealed that the value of his properties has risen by 8% over the past year, netting him paper gains of tens of millions of pounds, and that he has pushed up the rent on his average property from £850 to £900 over the year, a rise of 5% compared to wage increases of 1-2%.

He cares little about the outrage he has generated. "Everyone this week on TV and radio has been saying what a rat this man is. But the issue is that there simply isn't enough housing to go round. Either working eastern Europeans or single mums on benefits have got to go without housing. None of the big political parties has an answer."

The Wilsons first came to the public's attention in 2006 at the height of the buy-to-let bonanza, when pre-credit crunch lenders were issuing loans almost indiscriminately. At times, the married couple, both former maths teachers, were buying a house every day on the estates going up around the Eurostar terminal in Ashford.

Judith Wilson said in 2006 that they built their property empire almost by accident, bidding £98,000 for a house even though they had just £10,000 in the bank. "We only found the mortgage afterwards. At the time, it was the scariest thing I'd ever done. Today I don't bat an eyelid when I buy a house."

Their first purchase became the base for a pyramid of properties bought by constantly remortgaging and using equity from rising values to put down a deposit on yet another house.

Their wealth ballooned to more than £100m but apart from a £2m-plus home outside Maidstone and eccentric Burberry suits, their main extravagance was a much derided stable of racehorses. Maidstone Mixture, owned by Fergus Wilson, was dubbed "the worst racehorse in history" after the 1000-1 outsider came last in the 2008 Epsom Derby. One pundit predicted there was "more chance of Lord Lucan riding in the Derby than Maidstone Mixture winning".

Pundits also forecast that the Wilsons' buy-to-let empire would collapse during the financial crisis, and as house prices in the Ashford area slumped 20% and arrears worsened, the couple came close. "People were wondering if we would survive. But then mortgage rates dropped rapidly. We would not have drowned anyway, but it did keep us dry."

For such a large property empire, publicly available data is thin on the ground, as the homes are not held in a traditional corporate structure. At Companies House, Fergus Wilson is named as director of only one company, JWIPB Ltd, now defunct, while Judith is not named at all. All the properties on the Land Registry which the Guardian was able to establish were owned by the Wilsons had loans from Mortgage Express, the subsidiary of Bradford & Bingley that came close to collapse in 2008 and was rescued by the taxpayer.

Wilson readily admits that his property empire is heavily mortgaged, but says the loans work out at 61% of the value of the properties. He says he pays interest on most of the loans at around 2.25%, with some costing as little as 1.65%, compared to the typical 5%-plus paid by first time buyers under the government's Help to Buy scheme.

The 200 eviction notices to his housing benefit tenants were sent out across the course of 2013, with the last of the bailiffs going in this week.

"We were never going to send out the notices in one go. That would leave us with too many to let out. Only the bailiff ones are left this week. We let some of the mothers and children stay because they didn't want to be out over Christmas and New Year. We have tried to do it as humanely as possible."

He talks of one tenant who fell into arrears, where the Wilsons obtained a court order to have his car seized and sold at public auction. "I'll tell you something he doesn't know. I went to the auction and bought it myself. It was a Land Rover Freelander. Nice car, and I got it at a good price."Ashford Council housing director Tracey Kerley talks of dealing with "the Fergus factor" in the local housing market. "Fergus writes to us on a regular basis, saying he doesn't want people on benefits. A year ago we did sign a deal with him on 10 of his properties where he would take people on benefits who needed accommodation but he didn't like it and terminated the arrangement." Wilson says that most of them never paid the rent. "75% of the tenants in those houses defaulted, which was even worse than I predicted," he says.

Kerley suggests that the Wilsons' desire to remove housing benefit tenants is more to do with obtaining higher rents rather than problems with arrears. "Local housing allowance [the maximum level of housing benefit a council is permitted to pay] is getting smaller than the market rents around here. It just about meets the cost of a three bed, but not a four bed home. I think what's happening is that landlords can see they can obtain a higher market rent than they are getting under LHA. They are saying 'if I can get them to leave now, I can charge a higher rate'."

But Wilson regards himself as a crusader against benefit fraud. He says one tenant managed to obtain one of his homes after "the young girl in the letting agency felt sorry for her". She was paying £850 a month in rent even though she was only in receipt of LHA of £600 in local housing allowance – the maximum level of housing benefit a council is permitted to pay.

"I drove by and saw she had a couple of brand new 4x4s on the drive. It told me she was on the fiddle. I gave her notice, and personally went to the courts for the possession order. Why are working people in Maidstone and Ashford paying full rent on a two-bed flat while someone is on the fiddle in a three-bed house?" Tenants evicted from the Wilsons' properties face a struggle to find local affordable housing. "Like any Kent authority, we have got growing waiting lists and a high number of homeless," says Kerley. Local councillor Tina Heyes, who represents the Park Farm area, adds: "I don't agree with what he's doing at all. It will just push people further into poverty."

In 2009 the Wilsons, named that year as the 34th richest couple in the UK by the Sunday Times, took two tenants to court demanding £3,000 to pay for a new bathroom suite after they accidentally broke a toilet lid, but the judge threw out the claim, describing it as "exaggerated".

But neither the local housing department nor councillors receive many complaints about the standard of the Wilsons' homes. "The standard of our properties is top of the range," claims Wilson.

Local councillor Tina Heyes said: "To be honest, I would have heard rumblings if his properties were that bad, but I have heard nothing." Wilson says he employs a team of eight workers to maintain his homes – and all are from Eastern Europe.

Since abandoning hopes of a big racing title, Fergus Wilson has found a new hobby – as a prolific author of children's books and "alternative history" books that often imagine a Britain taken over by the Nazis.

He has self-published and co-authored 135 books available as downloads on Amazon. The cover of Hilda's War features a young blond female in SS uniform, while Rebecca's War is the tale of a teenage girl "forced into duties at a Lebensborn home to generate new German babies under a strict regime".

Meanwhile, his "Larry the Liger" children's series features the story of a lion-tiger left homeless and abandoned. "And no, I'm not the big bad wolf," says Wilson. It is ranked 444,092 in the Amazon bestsellers table.