We are generally trusting eaters. A server brings a dish: It must be what the menu says. But labels can lie, and when it comes down to it, can you really tell the difference between one whity flaky fish and the next?

Whether you like it or not, food fraud is real, and it’s probably happened to you.

The food fraud du jour is olive oil, thanks to an infographic in the New York Times last week that portrayed the problem as a clever, if vaguely sensationalist, cartoon.

Food fraud can be a tough issue to garner sympathy for—along with wine and caviar, fancy olive oil is a luxury product; if you’re willing to shell out that much money for something you can’t even identify, aren’t you just asking for it? Decidedly not, is our opinion. But you do need to do your homework.

So if you don’t have a chromatography set in your pocket, can you ever eat without paralyzing uncertainty again? There are a few red flags to watch out for across the board.

Number one: Read the ingredients. Words like aroma or essence should make you turn and run. Added sugar and sneakily suffixed ingredients (vanillin for vanilla, for example) are frequent offenders. Above all, buy whole foods whenever possible—because it’s easy to add paprika extract to orange juice, but what could they possibly do to this orange? Well, actually… but that’s another slideshow.

1. Olive Oil

When it comes to olive oil, reading the label can only get you so far. Much olive oil that claims to be "From Italy" is actually brought into Italy and re-exported from there—because they're not technically lying if it passed through one of the ports.

Olive oil may be diluted with soybean or vegetable oil, or even fabricated entirely from a cheaper oil that's been doctored up with beta carotene and chlorophyll. You may look for the "extra-virgin" label, though one study claims that 69 percent of bottles labeled as such did not pass the test. Take advantage of tasting bars—free samples and quality control—and look for bottles that designate specific points of origin.

2. Honey

At first glance, the teddy bears look cute. But take a closer look. See the beady little eyes. Those smug upturned noses. They're hiding something. In fact, reported cases have unmasked so-called "honey" as sucrose, beet sugar, or even high-fructose corn syrup. The Al Capone of honey laundering was busted in 2008 for allegedly smuggling cheap honey from China and relabeling it as White Korean Honey.

To avoid honey fraud, choose small-batch honey. Be sure to check the ingredients to avoid any products bolstered with added sugar or syrup.

3. Fish

The main form of seafood fraud is mislabeling: a cheaper fish passed off as a more expensive one. The major offenders involve fish with ugly names masquerading as their prettier-sounding, pricier counterparts (toothfish for Chilean sea bass; escolar as white tuna; threadfin slickhead as Alaskan cod), and in a study conducted between 2010 and 2012, ocean conservation organization Oceana found that every single sushi restaurant tested in New York carried mislabeled fish. Not only does this mean the customer overpays, but the fish that end up on the plate are often ones you wouldn't want to eat in the first place.

Escolar, which often shows up instead of tuna, contains histamines that can cause very disturbing side effects—think diarrhea, but oilier. The FDA currently advises against "importation and interstate marketing" of escolar, and the fish is flat-out banned in Japan and Italy. To guard against seafood fraud, watch out for snapper and tuna, the most defrauded varieties. Oceana's findings may make you want to stop eating sushi altogether, but resources like Monterey Bay Aquarium's Seafood Watch and Trace and Trust are on your side.

4. Scallops

A quick conspiracy theory interlude: Seafood fraud is rampant, as we've seen, but a special case exists for so-called scallops that might actually be rays, skate, or shark that have been sectioned up by a cookie cutter. Instances of scallop fraud have largely been relegated to the dark realm of Internet message boards, so it's unclear how often this substitution takes place, if at all. While it is decidedly less gross than the dreaded pork-bung calamari, the prospect of cookie-cuttered scallops is still capable of inducing major cringes. This American Life, care to take a look?

5. Balsamic Vinegar

Traditional balsamic vinegar is aged in oak casks for years and, like wine, must adhere to strict codes. But after drizzles of balsamic vinegar reduction found their way to every restaurant table, the market became, very literally, diluted. Don't be fooled if the bottle says "Modena" (as in, the Italian city known for balsamic). For the real high-end stuff, look for "grape must" in the ingredient list or the designation Aceto Balsamico Tradizionale, which will guarantee the vinegar has been aged at least 12 years. For an ingredient in salad dressings, no need to raid the ancient Italian cellars, but avoid added sugar or caramel coloring at all costs.

6. Saffron

There was no chance the world's most expensive spice was staying off this list. Real saffron is made from the stigma of the saffron crocus and costs upwards of $2,000 a pound. Reports have found all sorts of dried flowers masquerading as the spice, even dyed onion.

How to keep from getting conned? Buy saffron in whole threads, which are much more difficult to fake. And as tempting as the deals may be, do not bargain hunt. Saffron is expensive for a reason: It takes 150 flowers to make a single gram of saffron, but it only takes an onion and some orange dye to make the powder sitting on clearance.

7. Vanilla

The second most expensive spice is also ripe for the faking. Vanilla extract is an aged solution of vanilla beans macerated in alcohol and water, and is the most common form of vanilla over the pricier whole beans. Much imitation vanilla is made from vanillin, an organic compound that does occur naturally in vanilla beans but more often is synthesized in a lab, and makes no bones about being a cheaper, if inauthentic, alternative. But synthetic vanillin often sneaks into extracts claiming to be pure.

Real vanilla contains 4-hydroxybenzaldehyde, not found in the artificial stuff, but when you don't have your science kit handy, just glance at the ingredient list. If vanillin is listed, it's likely synthetic. Added sugar? High-fructose corn syrup? Stay away.

See the full list of foods that might be fake.

More from Bon Appétit

8 Foods That Could Kill You

10 Snacks You Thought Were Healthy But Aren't

How to Reheat Leftovers, from Pizza to Pasta to Eggs

22 Recipes Everyone Should Know How to Cook