Evaluating Medical Research

Hardly a day goes by without a story on television, in the newspaper, or on the Internet about new medical research findings. You might hear about a new drug to treat Alzheimer’s, a promising “cure” for cancer, or a breakthrough discovery in Parkinson’s disease. Or you might see articles about particular foods or dietary supplements that are said to promote health or prevent or slow the course of an illness. Should you try to get these drugs for a family member who is sick? Should the person change his diet? Take more vitamins?

It’s confusing when research findings are contradictory. Conflicting health news stories—for example, drinking a glass or two of wine is good or bad for you, taking vitamin A may prevent one form of cancer but cause another to worsen—leave everybody, caregivers and health professionals alike, wondering what to do next.

It is possible to find your way through the massive amounts of information and misinformation and make informed decisions about your health, and the health of your loved ones. Reaching that point takes effort, awareness, and trust in your own powers of perception.

Before deciding whether to investigate news reports further, consider the following:

Is the headline or story presentation sensationalized—a tactic used mainly to grab attention?Is there “context” for the story—for example, background information that can help you evaluate the importance of a new finding?

Is there “fair balance” in the reporting—that is, presentation of varying points of view?If the story is on television or radio, does it somehow tie in to one of the show’s advertisers? If on the web or in print, is the story positioned near a related advertisement? On close inspection, is the story identified as an advertisement? Does the story’s originator also sell the product?

If, after considering these factors, the new finding or treatment seems possibly helpful to a loved one, then it’s time to get more details. The following guidelines can help you evaluate the research, separate fact from hype, and identify stories that may be misleading, inaccurate or incomplete.

General Guidelines

First, try to find the original research article by contacting the website, newspaper, radio or television station responsible for the news story. This may be easier to do online or at a library or by calling the institution that sponsored the research; a journalist may or may not get back to you. Once you locate the article, read it thoroughly. Keep in mind that understanding articles written by physicians or scientists for their peers (other members of the scientific community) can be a challenge. The writing style is often technical. Many of the words will be unfamiliar, and you should be prepared to look them up in a medical dictionary or glossary.

Here are some questions to help you assess whether an article might be of interest.

How was the study conducted? In the laboratory, with animals, or with people? Laboratory research, or research with animals, is usually years away from use in a clinic. The results of research with people are more likely to be meaningful to you and your loved one.

Where was the article published? If it appeared in a peer-reviewed medical journal, it has more credibility than if it is posted only on the investigator’s website or in a company’s brochure. If the study is reported at a medical meeting, but has not been published in a journal, then the treatment probably needs to undergo further study.

Who conducted the study? A group of researchers based at a reputable university or research institution, or an individual without clear affiliations? Be particularly cautious about accepting claims about treatment effectiveness in articles by people who intend to sell the treatment.

Who funded the study? If a researcher received funding from an independent granting agency that uses a peer-review process for evaluating and awarding funds, the findings may be more credible and less subject to bias than if the funding agency has a vested interest in the results. If a study is sponsored by a commercial entity, this should be disclosed. But although sponsorship may influence the authors’ conclusions in some instances, this is not always the case. There are many research partnerships among industry, university-based researchers, and institutes, so this can be a difficult issue to assess.

How many people were tested? How many completed the study? A small number of participants does not mean the study is invalid, but the small number could lead to questionable results. In most cases, particularly when investigating causes and treatments of chronic illnesses, researchers need to study a large population over a long period of time to ensure that any observed differences are not simply the result of chance.

What were the characteristics of the people who participated in the study? Sometimes, treatments are effective only for certain groups of people—for example, a particular age group, or those whose disease is at a particular stage. Did the researchers study people like your family member? Studies of groups of people who differ from your age, gender, health status, and ethno-cultural background may or may not apply to you.

Have the results of the study been repeated by other investigators? If this is the only study of its kind, then further research is needed to validate and/or repeat the results.

If a drug was tested, were there any side effects? If so, do they outweigh the potential benefits of using the drug? Has the drug been approved by the Food and Drug Administration? If not, are clinical trials of the drug underway? When does the manufacturer estimate that the drug will be available? It often takes many years of testing before drugs are deemed safe enough for use by the public.

Health Information Resources

Consumer’s Guide to Taking Charge of Health Information
https://cdn1.sph.harvard.edu/wp-content/uploads/sites/1273/2013/06/RISK_IN_PERSP_OCTOBER1999.pdf

How to Evaluate Health Information on the Internet
https://ods.od.nih.gov/HealthInformation/How_To_Evaluate_Health_Information_on_the_Internet_Questions_and_Answers.aspx

Medline Plus
www.medlineplus.gov

Quackwatch
www.quackwatch.com

Excerpted from FCA Fact Sheet, Evaluating Medical Research Findings and Clinical Trials. To read full text of this Fact Sheet, visit caregiver.org/caregiver/jsp/content_node.jsp?nodeid=402.