Buy-to-let landlords could turn potential tenants away or even evict them if they are on housing benefit, the government has been warned, amid confusion over whether the benefit will be paid to tenants rather than directly to landlords under the new universal credit system.

Kevin Green, one of the country's biggest private landlords, who rents out more than 700 houses, mainly in Wales, described the universal credit benefit system as a timebomb for private landlords.

He said he hoped to stick with benefit tenants but believed he was in a small minority. "Most other landlords are changing now. At least 90% of landlords are considering it. I know a lot of landlords all across the UK and most are saying they're not going to support housing benefit. It's a major problem for government."

Private landlords accommodate just over 1 million people who have part or all of their rent paid by the state. "My worry is for the customers. People are going to be homeless as a result of it. There's absolutely no doubt about it," Green said.

Fears that housing benefit will go directly to tenants under universal credit, bringing an increase in rent defaults, is prompting many landlords to consider following Fergus and Judith Wilson, who own almost 1,000 properties in Kent, who announced a week ago that they would no longer take tenants on benefits.

Fergus Wilson was unrepentant, sending an outspoken and at times bizarre statement to the Guardian urging benefit claimants to "get a job" and calling on David Cameron to make it clear that landlords can receive benefits directly. He says: "Calling Dave, Calling Dave, Fergus to Dave. Take a tip Dave! Stop this nonsense of paying housing benefit direct to the tenant!

"We have £800,000 arrears for those on HB [housing benefit]. Has the £800,000 caused financial difficulty … to be honest not an awful lot, but it has some!"

Marion Money, who rents out more than a dozen flats and houses in north Kent, said three-quarters of private landlords were considering turning away would-be renters on benefits. She said: "Landlords are running scared. They are saying 'if I've got a choice I'll choose somebody who's working'. The government isn't giving the right message about universal credit. They seem to be in a complete mix-up. The fact it has been delayed is compounding the problem."

Currently around a quarter of private tenants on housing benefits have the money paid directly to their landlords. In some cases this is because they are viewed as vulnerable, and it happens automatically if a tenant misses two rent payments.

Universal credit will mean money being paid monthly rather than weekly, with an emphasis on recipients learning to budget properly. The paradox is that after trials in which some social tenants received rent money themselves for the first time, bringing a rise in defaults, the protections for private landlords are now arguably stronger.

The latest Department for Work and Pensions plan would keep the system under which rent goes directly to a landlord after two missed payments, while adding a review after a first month's default. But some warn that the initial message about recipients managing their own money has stuck, while the DWP has yet to confirm whether tenants who currently have rent paid straight to landlords will keep this under universal credit.

Kate Webb of Shelter argues that an already difficult climate for tenants on benefits has been made worse by the DWP. "The rules will be broadly the same, and the arrears trigger will actually be stronger, but landlords are very worried. The DWP has bungled the communications. They need to be doing much more to reassure landlords about the rules," she said.

A DWP spokesperson said: "This is needless scaremongering. The majority of Housing Benefit claimants already receive payments directly and there is no change under Universal Credit. Our reforms ensure our strong housing safety net remains in place and there has been no fall in the number of claimants in the private rented sector. Ignoring over a million private rented sector claimants would be foolhardy for landlords' business plans."

What seems certain is that publicity about universal credit, and the Wilsons' decision, is making it even harder for tenants on benefits to find a home.

Benjamin Matthews, a letting agent in south-east London, said he was called this week by a client in Singapore: "He has quite a portfolio and the majority are rented out to people who are on benefits. He wanted to know: 'Has something happened? Has there been a big change?'"

A poll of National Landlords' Association members has found that while three years ago almost half were aiming properties at tenants on benefits this has already dropped to 22%.Chris Norris, the NLA's head of policy, said universal credit was a major issue: "The difficulty is the government has been very cagey in releasing information and timelines, and so many deadlines have been missed. They have been a little bit secretive and landlords have jumped to conclusions, or filled in the gaps themselves."

Housing charities and landlords' associations, while stressing that the extent of the chaos may yet not be quite so significant, have called on ministers to reassure landlords about the planned changes under universal credit, particularly those connected to housing benefit being paid to tenants rather than directly to property owners. Even landlords with a significant commitment to housing benefit are considering their options. Almost three-quarters of Green's homes are leased to such people, some with the help of charities working with young homeless people or victims of domestic violence.

Green says he is something of an "oddball" among landlords for such efforts, but says this might have to end. "It could be before the end of this year that we follow the Wilsons and say OK, we're not taking people on benefits any longer. It has to be an option for us. It's a commercial business. A lot of properties still have mortgages on them and they need to be paid."