Conclusions Recommendations made on medical talk shows often lack adequate information on specific benefits or the magnitude of the effects of these benefits. Approximately half of the recommendations have either no evidence or are contradicted by the best available evidence. Potential conflicts of interest are rarely addressed. The public should be skeptical about recommendations made on medical talk shows.

Results We could find at least a case study or better evidence to support 54% (95% confidence interval 47% to 62%) of the 160 recommendations (80 from each show). For recommendations in The Dr Oz Show, evidence supported 46%, contradicted 15%, and was not found for 39%. For recommendations in The Doctors, evidence supported 63%, contradicted 14%, and was not found for 24%. Believable or somewhat believable evidence supported 33% of the recommendations on The Dr Oz Show and 53% on The Doctors. On average, The Dr Oz Show had 12 recommendations per episode and The Doctors 11. The most common recommendation category on The Dr Oz Show was dietary advice (39%) and on The Doctors was to consult a healthcare provider (18%). A specific benefit was described for 43% and 41% of the recommendations made on the shows respectively. The magnitude of benefit was described for 17% of the recommendations on The Dr Oz Show and 11% on The Doctors. Disclosure of potential conflicts of interest accompanied 0.4% of recommendations.

Main outcomes measures Percentage of recommendations that are supported by evidence as determined by a team of experienced evidence reviewers. Secondary outcomes included topics discussed, the number of recommendations made on the shows, and the types and details of recommendations that were made.

Interventions Investigators randomly selected 40 episodes of each of The Dr Oz Show and The Doctors from early 2013 and identified and evaluated all recommendations made on each program. A group of experienced evidence reviewers independently searched for, and evaluated as a team, evidence to support 80 randomly selected recommendations from each show.

Popular television talk shows such as The Dr Oz Show often engender skepticism and criticism from medical professionals. 13 14 15 However, no research has systematically examined the content of the medical information provided on these talk shows. Our objective was to review the most popular medical talk shows on television, to (1) determine the type of recommendations and claims given and the details provided, and (2) search for and evaluate the evidence behind these recommendations.

According to Nielsen’s report, American citizens spend an average of over five hours a day watching television. 9 International health information programs, such as The Dr Oz Show and The Doctors have become a regular part of television broadcasting. In the 2012-13 season, The Dr Oz Show was consistently ranked in the top five talk shows in America with an average of 2.9 million viewers per day, while The Doctors had a high of 2.3 million viewers. 10 11 In the 2012 Greatist report, Dr Mehmet Oz and Dr Travis Stork (one of the hosts of The Doctors) were both included in the top 100 health and fitness influencers. 12

Mass media in the form of television, radio and printed material are frequently used to deliver medical information to the public. Research suggests that mass media can improve public knowledge 1 and potentially improve health behaviors. 2 Television is one of the most important mass media sources of health information. 3 4 However, concerns have been raised about the quality, completeness and accuracy of medical information covered in the news media, 5 6 7 8 and television news media is no exception. 7 8 The quality of information outside of the news media has not been examined.

Methods

Programs We searched for internationally syndicated medical or health television talk shows that aired daily (weekdays). Two internationally syndicated medical or health television talk shows (The Dr Oz Show and The Doctors) were identified by internet search. We prospectively recorded all episodes of The Doctors from 11 January to 1 May 2013 (79 episodes) and The Dr Oz Show from 7 January to 1 May 2013 (78 episodes). Format of a typical medical talk show Shows are typically divided into 3-4 topics (range about 1-5), each topic runs about 12-15 minutes, often divided by commercial breaks

Each topic may have 4-5 recommendations within it (range about 0-10)

Recommendations may come from the host, guest, or rarely audience members

Data collection and outcome measures Three members of the research group (GMA, MRK, CK) reviewed two episodes of each show from the previous year (2012) to help inform the development of a data collection spreadsheet. The primary outcome was the percentage of recommendations and claims (henceforth referred to as recommendations) supported by evidence as determined by a team of experienced evidence reviewers. Secondary outcomes included topics discussed, the number of recommendations made, the types and details of the recommendations, and general episode characteristics. We had no previous information on which to base an estimation of the needed sample size to make a reasonable estimate of the number of recommendations that were evidence based. Based on direct observation of two of each of the shows, we hypothesized that 50% of the recommendations would be supported by evidence, then calculated that 158 recommendations would give an 80% chance that the 95% confidence interval would have a precision within ±10%. We estimated there would be at least two definitive or stronger recommendations per episode. Therefore, to attain 158 recommendations, we needed to review 80 episodes. Using a random number generator, 40 of the recorded episodes for each of The Dr Oz Show and The Doctors were randomly selected for analysis. Being unfamiliar with broadcast health information, we were unaware of the non-specific nature of many statements and recommendations given on medical television talk shows. In addition, the content of the medical information provided on talk shows has not previously been assessed systematically. This required us to use an iterative approach to the study design, ultimately resulting in three small changes to the protocol and one larger change (see data supplement for full details). A flow chart of the final study is presented in figure 1⇓. Fig 1 Flow chart of study design

Classification and content of recommendations Two reviewers (VL, KO) independently watched each randomly selected episode to document the topics discussed and the specific details surrounding recommendations. Further information on data extraction is included in the data supplement. To focus on stronger or clearer recommendations for the evidence review portion of the analysis, the reviewers attempted to delineate the more definitive recommendations. This was based on both the strength of the wording (for example, “Get your kids vaccinated” was considered a strong recommendation, whereas “Prescription retin-A helps with ear and other types of acne” was not) and the context in which the recommendation was made (for example, a recommendation may have been classified as strong if it was repeated multiple times). After independent review, data extraction was compared, disagreement of coding and classification was resolved by consensus or third party adjudication (CK, MRK, or GMA), and overall agreement was calculated. We found some of the initial classification of topics and recommendations included categories that were too broad. Therefore, two investigators (CK, JMcC) reclassified all topics and recommendations, again with a third investigator (GMA) resolving any uncertainties. After preliminary review of the data, it was determined that more specific detail was needed to better quantify the type of information provided for all recommendations. Two new reviewers (CC, CF) independently watched the same randomly selected episodes and focused on the recommendations to identify if a benefit was described (such as “heart healthy”), if the benefit was specific (such as “reduces heart attacks”), if a magnitude of benefit was given (such as “reduced 5%”), if costs were mentioned (such as “this supplement costs $5 a month”), and if potential risks or harms were mentioned. In addition, the observers recorded any mention of potential conflict of interest with each recommendation or acknowledgement of potential conflicts of interest at any point during the show. After independent review, data extraction was compared, disagreement was resolved by consensus or third party adjudication (CK, MRK, or GMA), and overall agreement was calculated.