"Only about one-fifth of Americans said they've seen any financial benefit from the tax cuts – a not-unexpected result given that the tax benefits were aimed at corporations and wealthier Americans."

More recently, Barack Obama won re-election in 2012 when the country was still climbing out of a recession. Four years later, Hillary Clinton couldn't leverage a fully-recovered economy into an electoral victory.

For the Trump Administration, the centerpiece of its economic record is strong jobs numbers, with the unemployment rate now at the lowest level since 2000. Last year's tax cut also allows the White House and congressional Republicans to argue they're putting more money in people's pockets.

Not surprisingly, Trump has used every occasion to tout his economic record. He even marked Memorial Day by boasting that fallen soldiers would be "happy and proud" about low unemployment numbers.

The problem for Trump, however, is that strong jobs numbers don't necessarily tell the full economic story for most people in this country.

According to a recent Monmouth University poll, only 12 percent of Americans said their family has benefited a great deal from recent economic growth. A whopping 53 percent said they've been helped "not much" or "not at all" by economic growth. These figures are virtually the same as when Trump took office in January 2017.

Despite the strong unemployment numbers, the percentage of working-age Americans who are employed today is actually lower than in 2008. College graduates hitting the job market this year will find 1.3 percent fewer jobs waiting for them, compared with last year. And wages are growing at a meager 2.6 percent rate, which won't make up for a long period of wage stagnation.

As for the much-touted tax cuts, the law remains unpopular. In a troubling sign for Republican candidates, only about one-fifth of Americans said they've seen any financial benefit from the tax cuts – a not-unexpected result given that the tax benefits were aimed at corporations and wealthier Americans.

At the same time, the price for everything else is going up. Gas prices have risen over 50 cents over the past year and are now at the highest level since 2014. An average household is expected to spend $248 more on gas this summer compared to last year. For the bottom one-fifth of households, the rise in gas prices has already offset whatever windfall they received from tax changes.

The cost of other necessities is rising as well. Rent prices are soaring in cities like Orlando, Las Vegas, and Sacramento. Health care premiums for Americans on Obamacare will increase an average of 15 percent this year, largely because of Trump's actions to undermine the law. In individual states, the premium increases are much higher. One health plan has asked to boost rates by 64 percent in Virginia and 91 percent in Maryland.

For the least fortunate in this country, the Administration is engaged in a systematic effort to make Medicaid, food stamps, and federal housing assistance more difficult to receive. Overall, four in ten Americans can't cover an unexpected $400 expense without borrowing money or selling something.

When you consider all aspects of a typical household budget, it's no wonder that, despite low unemployment, only about one-third of Americans say the country is headed in the right direction, compared with 52 percent who say it's on the wrong track.

Certainly, low unemployment numbers can provide Trump with an important rhetorical weapon against a potential Democratic wave. However, unless the rosy picture presented by Trump and his allies matches the reality for most Americans, a strong overall economy won't be a decisive issue for midterm voters, especially given the constant swirl of controversy, chaos, and divisiveness surrounding Trump.

Much can change over the next five months. But if Donald Trump thinks that low unemployment alone will ensure Republican control of Congress, he'll be in for a rude awakening the morning of November 7. This year, understanding voter sentiment won't be as simple as reading a few pieces of economic data.

Commentary by Chris Lu, who served in the Obama Administration as White House Cabinet Secretary and Deputy Secretary of Labor. He is now a senior fellow at the University of Virginia Miller Center. Follow him on Twitter at @ChrisLu44.

For more insight from CNBC contributors, follow @CNBCopinion on Twitter.