By 2021, robots will have eliminated 6% of all jobs in the US, starting with customer service representatives and eventually truck and taxi drivers. That’s just one cheery takeaway from a report released by market research company Forrester this week.

These robots, or intelligent agents, represent a set of AI-powered systems that can understand human behavior and make decisions on our behalf. Current technologies in this field include virtual assistants like Alexa, Cortana, Siri and Google Now as well as chatbots and automated robotic systems. For now, they are quite simple, but over the next five years they will become much better at making decisions on our behalf in more complex scenarios, which will enable mass adoption of breakthroughs like self-driving cars.

These robots can be helpful for companies looking to cut costs, but not so good if you’re an employee working in a simple-to-automate field.

“By 2021 a disruptive tidal wave will begin. Solutions powered by AI/cognitive technology will displace jobs, with the biggest impact felt in transportation, logistics, customer service and consumer services,” said Forrester’s Brian Hopkins in the report.

The Inevitable Robot Uprising has already started, with at least 45% of US online adults saying they use at least one of the aforementioned digital concierges. Intelligent agents can access calendars, email accounts, browsing history, playlists, purchases and media viewing history to create a detailed view of any given individual. With this knowledge, virtual agents can provide highly customized assistance, which is valuable to shops or banks trying to deliver better customer service.

Forrester paints a picture of the not-too-distant future.

“The doorbell rings, and it’s the delivery of a new pair of running shoes, in the right style, color and size, just as you needed to replace your old ones. And here’s the kicker: you didn’t order them. Your intelligent agent did.”

In the transportation industry, Uber, Google and Tesla are working on driverless cars, while similar technology is creeping its way into trucking to replace expensive human drivers.

It’s easy to get dazzled by such innovations, but what happens to the 6%? The call center staff, the taxi drivers and the truckers. There may be new jobs created to oversee and maintain these automated systems, but they will require an entirely different skillset.

“Six percent is huge. In an economy that’s really not creating regular full-time jobs, the ability of people to easily find new employment is going to diminish. So we will have people wanting to work and struggling to find jobs because the same trends are beginning to occur in other historically richer job creation areas like banking, retail and healthcare,” said Andy Stern, the former president of the Service Employees International Union.

“It’s an early warning sign and I think it just portends a massive wind of change in the future.”

Studies have shown that higher rates of unemployment are linked to less volunteerism and higher crime. Taxi drivers around the world have already reacted with violent protest to the arrival of ride-hailing app Uber. Imagine how people react when Uber eliminates drivers from its fleet.

“There is a lot of correlation between unemployment and drug use,” said Stern. “Clearly over time, particularly in urban settings, the lack of employment is tinder for lighting a fire of social unrest.”

The challenge posed by automation is not being taken seriously enough at a policy level, Stern added. “Politicians would rather talk about getting a college degree and technical skill training, things that are probably five to 10 years too late. We don’t really have a plan and we don’t appreciate how quickly the future is arriving.”

Does this mean we’re all doomed? “No. But what level of pain do people have to experience and what level of social unrest has to be created before the government acts?

Workers, many of whom don’t have technical skills, are competing for less and less jobs. If the market works without intervention we’re going to have no way to mediate the displacement.”