If you’re guilty of Googling your symptoms at the first sign of pain in your side or a scratch in your throat, take heed. A new study published this week found that online symptom checkers often give inaccurate diagnoses.

In the first large-scale study of its kind, researchers at Harvard Medical School discovered that symptom checkers like Web M.D., the Mayo Clinic, DocResponse and others that prompt users to type in their symptoms and provide a diagnosis using a computer algorithm, were only effective about one-third of the time.

In the study, researchers created a list of symptoms based on 45 illnesses and input them into 23 commonly used websites hosted by medical schools, hospitals, insurance companies and government agencies based in the United States, the United Kingdom, the Netherlands and Poland. In many cases, users should be cautious and not take the information they receive from online symptom checkers as gospel.

The results? The online checkers provided the correct diagnosis 34 percent of the time, listed the correct diagnosis within the top 20 results 58 percent of the time and provided the appropriate triage advice 57 percent of the time.

The bottom line: “It is not nearly as important for a patient with fever, headache, stiff neck and confusion to know whether they have meningitis or encephalitis as it is for them to know that they should get to a doctor quickly,” senior study author Ateev Mehrotra, associate professor of health care policy and medicine at Harvard Medical School, said in a statement.

“In many cases, users should be cautious and not take the information they receive from online symptom checkers as gospel.”

The study acknowledged that online symptom checkers are part of a growing trend in our digital world of both patients and physicians turning to the Internet for many health-related questions. Patients increasingly email and chat online with physicians for medical advice, and more doctors diagnose illnesses and provide patient care through e-visits and smartphone apps.

“With symptom trackers, we are looking at the first generation of a new technology,” first study author and research assistant Hannah Semigram said. “It is important to continue to track their performance to see if they can reach their full potential in helping patients get the right care.”