Despite a historic level of low unemployment, Goldman Sachs believes the labor market is divided, with a "pool of struggling workers" on the fringe.

In a research note by chief economist Jan Hatzius, Goldman notes that while short-term unemployment has improved to historic lows, a higher rate of long-term unemployment and lower rate of prime-age participation betrays another side of the workforce.

"The U.S. labor market appears divided in two. At one end, the rate of short-term unemployment, defined here as those unemployed fewer than 15 weeks, is lower than at any point since the Korean War," wrote Hatzius on Monday. "At the other end, the pool of struggling workers at the margins of the labor market remains somewhat larger than in past expansions. In particular, the rate of medium- to longer-term unemployment, … remains 0.75 percentage points higher than the low reach in the late 90s boom, and almost none of that gap is attributable to demographic change."

Source: U.S. Department of Labor, Goldman Sachs

According to the latest data from the U.S. Department of Labor, the unemployment rate in the United States declined 0.2 percentage points in September to a low of 4.2 percent, with expectations of further decreases in 2018. But digging deeper into the data, economists point out not only that the long-term unemployment rate remains elevated, but that the participation rate for prime-age men also remains well below pre-crisis levels.

The next report on the employment situation is published by the Labor Department on Friday.

"While we have long seen some room for cyclical recovery in the participation rate, we have been skeptical about the scope for a full recovery," added Hatzius. "The main reason is that male prime-age participation has been declining for more half a century and appears to partly reflect deeper social problems such as incarceration, drug abuse, and deteriorating health."

Source: U.S. Department of Labor, Goldman Sachs

To be sure, Hatizus recognizes that the employment situation has certainly improved over the past several years and may continue to improve for these marginal groups in the coming months as the labor market continues to tighten, something members of the Federal Reserve are keeping a close eye on.

The policymaking Federal Open Market Committee has been debating over the health of the current economic situation for months, with hawks pointing to decreases in unemployment and strong markets data. Doves caution against disruptive moves in the form of frequent rate hikes that could shock unsuspecting investors.

But despite occasional disagreement, the central bank has signaled that a December rate hike is all but certain and that barriers to inflation (and wage growth) are soon to be dissolved entirely.

Weighing in on the debate, Hatizus suggested that it may be worth waiting a while longer for these marginal groups to finally rejoin the labor force. But on the other hand, he notes, the Fed may not want to wait to find out if the fervor in short-term employment causes the economy to overheat.

"This trade-off offered by a high-pressure economy did not appeal to Fed officials last year, and despite the still-restrained rates of wage growth and inflation, we doubt that they find it much more appealing today. We expect that preventing the labor market from moving into almost unprecedented territory will continue to provide a very strong motivation for a steady pace of tightening in coming years."