Reading a scientific study can feel like reading something in a foreign language. Imagine this common scenario: you’ve just come across a new claim about your favorite herb or essential oil. There’s even a link to a study to support this claim. You know that you need to click on the study to confirm the claims…. and then what? These four keys are must-haves when analyzing at any published scientific research.
The Abstract Will Not Cut It
The first step to reading a scientific study is to gain access to the actual study. It may be tempting to just stick with the indexed abstract on PubMed. After all, the gist of the purpose and conclusions are clearly explained. And you may have even been told at an herb or essential oil conference that the abstract of a research study is all you really need. It’s not. Always get access to the whole research study. This will frequently mean that you have to pay a fee-often in the $30-$35 range. But if you want to really know what the study says, you have to actually gain access to the whole thing.
Otherwise, you’re trying to draw conclusions about the complex writings of a book based exclusively on the back cover. Yes, back covers should provide the big picture of its contents, but the complex intricacies of the book itself can only be understood by reading the actual thing, not merely a brief overview. In many situations, the abstract fails to mention crucial points of the study as it may omit key analyses, moderating factors, or caveats based on the research. If reading the entire study seems like a lot of work, keep in mind that the published version is already a fraction of the entire research process itself. Most researchers could easily publish 100+ pages on a single research study–if anyone were around to actually read it. The condensed version which makes its way to a journal is already shortened to make the evaluation process much easier on the reader.
Get to the Point
A scientific study exists to answer a key research question. Missing that point is one of the biggest errors in study interpretation, especially for the neophyte. As you are reading a scientific study, look for clues to answer the question: What is the main research question? Be careful not to look at key phrases or familiar words and jump to conclusions; this almost always leads to a misinterpretation and frequently forms the basis for pseudoscientific interpretations of the literature.
Similarly, remember that interventional studies (clinical trials) only comprise a small portion of the studies we use in research. So the main research question may not be related to the establishment of causation–in many cases, studies exist to look for trends or patterns to establish a relationship that can later be evaluated for causation.
One of the first ways to know the inherent strengths and limitations of a study is to identify the type of study you are reading to develop some context for its findings. In other words, you will need to answer two questions: What type of research study is this? How are those studies interpreted? (Or perhaps more importantly, is this even a scientific study? Or is this a letter to the editor or opinion piece that was published in a scientific journal?)
There are many different types of research methods that are used to study herbs and essential oils, ranging from the case report to a meta-analysis. Each specific research method used for the study has its own criteria for quality and for interpretation. For example, some studies are designed to identify relationships but do not establish causation. Others focus on comparing interventions. Still others look for patterns related to rare health outcomes. Knowing which type of study you are reading will empower you to understand the context required to integrate the study’s findings into your knowledge base.
Bigger is Not Always Better
Another common practice when reading a scientific study is to identify the sample size and use it to determine the strength or accuracy of the study’s results. On the surface, this makes sense; if these findings are produced by a large sample size, they must be accurate. But while sample size matters, it’s not what tells us how useful the study is or is not.
More important metrics, aside from the suitability of the research design to answer the research question, include the effect size and practical significance. Not all statistically significant findings are practically relevant; some studies find that a relationship can be established–sometimes even a causal relationship–yet the real-life meaning of that outcome is nonexistent. For example, it might be accurate that an essential oil reduces blood pressure by a half a point in high quality studies, but for someone with hypertension, that half a point–as predictable as it may be–is simply not going to cut it. Finding an outcome that has statistical, but not practical significance is common. The effect size of the intervention tells us, as the name implies, the size of the effect we are measuring.
So how do you know if the sample size is large enough in a clinical trial? Look for mention of a power analysis. This indicates that the researchers have determined just how many patients should be in an interventional study to properly evaluate the data, based on the known effect size of the intervention. Interventions which have small to moderate effects on the outcome may require larger samples, but interventions which have dramatic effects can be well established with fewer participants.
Integrating the scientific literature into an approach or protocol requires more than entering some search terms into PubMed and finding research studies that appear to support a theory. Scientific evidence in the form of health research comes with context and it must be analyzed, interpreted, and utilized within that context–and within the context of the entire body of evidence about a subject as a whole–for a fully evidence-based approach.
Therefore, understanding research methods is a critical tool for today’s herbalists, aromatherapists, and health promotion professionals. Get the education you need to excel in your field through the Franklin Institute of Wellness’ certification and continuing education programs.