Though unauthorized workers represent a relatively small share of the overall American workforce, they are the fastest-growing segment, according to a new Pew Research Center study.

Over the next two decades, as baby boomers leave the workforce they entered in droves in the 1960s, the growth of the labor force will slow to levels not seen in five decades. Among those born in the U.S., whose parents were also born in the U.S., the working-age population is expected to shrink by 8.2 million over the next two decades, according to Pew.

That drop with will be partly offset by an increase of about 13.5 million working-age adults born in the U.S. to immigrant parents.