Without new immigrants, by the middle of the century the nation’s population would begin to decline, the elderly would account for nearly one in four Americans and non-Hispanic whites would remain a majority, according to new projections by the Census Bureau.

But if immigration were to merely slow, rather than stop, non-Hispanic whites, who now account for nearly two-thirds of the population, would become a minority by 2050, according to the projections released Wednesday. If the pace of immigration increases, that benchmark could be reached as early as 2040.

Depending on the pace of international migration, the nation’s population, currently at 308 million, could grow to as much as 458 million by midcentury, with immigrants accounting for up to 136 million of the increase.

Since 2000, the population has been growing by just under one million immigrants annually. The bureau’s lower estimates assume a range of 1.1 million to 1.8 million; the higher estimates range from 1.5 million to 2.4 million.