Systematic Reviews and Meta-Analyses

The second level of the pyramid and the second source of filtered information is systematic reviews and meta-analyses. A systematic review collects the results from all available studies of a particular health topic and integrates the findings into a single, cohesive set of findings. Meta-analyses also collect information from all available research studies on a topic but go beyond systematic reviews to incorporate statistical analysis of the combined findings. Both of these types of research articles answer specific research questions by collecting and evaluating all research evidence that fits the reviewer’s selection criteria (Turner, 2014). Systematic reviews and meta-analyses are considered a lower level of evidence because they do not include a rating of the evidential support for the findings.

Unfiltered or Primary Information

The unfiltered information section of the evidence pyramid includes the primary research studies that form the basis of the systematic reviews and meta-analyses found in the filtered information documents. All the studies found in this section are reports of original research findings published in peer reviewed journals. These studies are considered primary because their findings are based on unique, documented observations by the authors. in our model, the unfiltered information is divided into 3 levels: experimental studies, observational studies of groups and observational studies of individuals. These levels are ordered by their applicability to larger populations of individuals. Randomized controlled trials and grounded theory research are most generalizable because they compare outcomes in the intervention group to a control group. Observations involving larger groups of people make those findings more applicable across the population than observations on individuals. As with the systematic reviews and meta-analyses, articles assigned to these levels of evidence are expected to be peer reviewed. However, these studies have not been subjected to initial analysis and review beyond that of the peer reviewers for each study. In most cases, unfiltered levels of evidence are difficult to apply in clinical decision making (Kendall, 2008). However, higher levels of evidence may not be available to answer every focused clinical question. In these instances, the healthcare professional will need to evaluate the primary sources carefully to determine the trustworthiness and relevance of the study to the clinical situation.

Background Information

Background information and expert opinions are not necessarily backed by research studies. Background information and expert opinion can be found in textbooks or medical books that provide basic information on a topic. They can be helpful to make sure you understand a topic and are familiar with terms associated with it.

While some textbooks can provide synthesized content backed by peer-reviewed research, it is not easy to tell which books are evidence-based and which are not without looking at the content of the book. As EBP beginners, you should assume that textbooks and other eBooks provide evidence at the background information/ expert opinion level.

Identifying the Target Database(s)

Much of locating relevant evidence depends upon the evidence-based sources available to you. You are familiar with the process of purchasing textbooks. When libraries provide resources to people, they undergo a similar process. They either have to purchase the books that you are using or subscribe to that access through the publisher. As with everything else related to health care, the purchase price and subscriptions for health related resources tend to be much higher than the cost for similar resources in other disciplines. So while academic medical centers may provide access to a wide range of resources smaller facilities may only offer one or two options. We will look at some of the most common paid resources as well as some freely available resources that you can access without a subscription.

Deciding on the level of evidence that is most appropriate will help you narrow down the array of possible information sources to the databases that are best suited to provide the level of information you need.

Resources for Finding Filtered Information

The highest level of filtered information – Guidelines with Ratings – converts the research findings of numerous studies into guides that focus on how the findings should be applied to clinical practice along with an indicator that communicates the strength of the research supporting each particular recommendation. The next level down – Systematic Reviews and Meta-Analyses – combines and summarizes the research findings of numerous studies that focus on answering the same research question, but do not translate those findings into actionable recommendations for clinical practice.

Information Resources to AVOID

In a world where we now carry more computing power in our pockets than was used to fly the first space shuttle, we are comfortable opening our devices and asking google any question that we can think of. This can be problematic because google searches the internet and A LOT of stuff on the internet can be questionable. Don’t believe me? Try searching for thyroid shielding during mammography (and the guy who started this misinformation actually IS a doctor…)… or benefits of colon hydrotherapy…. sometimes you can’t make this stuff up! Will google turn up some good information? Sure! But you have to be a lot more careful that the information being presented isn’t being biased by advertisers and is accurate. Even the information that is pretty solid is frequently from hospital websites that are providing information for patients and the general public. When we are making care decisions, we need the details, so this information is not at the appropriate level for our use. It also is generally not peer reviewed, so errors may inadvertently slip into the information. Resources that look pretty good, but are geared for patients include WebMD, MayoClinic.org, Healthline.com, and MedicalNewsToday.com. This information is typically written by health reporters and many of these sites have paid advertisers who may influence the perspective of the information presented.

Formulating a search strategy

It’s easier to find appropriate sources when you start with a plan.

Now that we have decided on one or more resources that we want to search, it is time to think about how we want to structure our searches to find the information we need quickly and easily!

Determining Search Terms

First, let’s go back to your PPAARE grid and question. For each PPAARE component, we want to think of alternate words or synonyms that can be included in the search. We also want to think of terms that should be excluded. Let’s look at an example.

Once we have brainstormed synonyms for our search terms and determined categories of studies that should be excluded from the search, it is time to structure our search.

Structuring the Search Query

When we are searching databases, we are basically programming a set of instructions for the computer to follow. While we can enter single words or entire sentences into the search bar like we do in google, our searches are more effective if we follow some basic principles and are aware of some common tools.

One Concept at a Time

In our searchable terms grid above, we have identified 8 different concepts that describe a different part of the clinical problem we are working on. If we put all of those terms in the search box at the same time, we probably aren’t going to get any results, because the exact scenario we are facing may not have been written about yet. So, it is better to start with one term and then add others one at a time. While you may not be able to get quality results with all of the search terms, you can mix and match the information in different combinations to arrive at an answer to your clinical question.

The best way to start your search is by using your problem (disease process) term. From our example above, we would start by searching for “prostate cancer.” If you don’t know the exact disease process, try searching “differential diagnosis” AND the symptoms the patient is experiencing.

Quotation Marks

If you are searching for a term that has more than one word or a phrase, you will want to enclose the search term in quotation marks. This tells the computer that these words must appear in this exact order with no words appearing between the terms. If you don’t use the quotation marks, the computer will treat your search as an AND search (more on this in Boolean Operators, below) and return all articles that include those terms in any order, anywhere in the article, which is much less likely to meet your needs. For example, if you are wanting information on the stages of grief, just typing stages of grief in the search box will return all articles with the word stages and the word of, and the word grief anywhere in it. If you enter “stages of grief” in the search box, the computer will only return articles with the exact phrase “stages of grief” in it. You can see how the second approach will give you much more relevant information.

Boolean Operators

Once we are ready to start combining terms into our search, we want to use Boolean Operators. We know that everything a computer does is boiled down to a math problem. One of the ways we can structure these math problems is with Boolean operators. Boolean Operators are simple words (OR, AND, or NOT) used as conjunctions to combine or exclude keywords in a search, resulting in more focused and productive results. This should save time and effort by eliminating inappropriate hits that must be scanned before discarding. Boolean Logic is named for George Boole, a mathematician who had the happy thought that you can combine sets with OR, AND, or NOT. Each search engine or database collection uses Boolean operators in a slightly different way or may require the operator be typed in capitals or have special punctuation. The specific phrasing will be found in either the guide to the specific database found in Research Resources or the search engine’s help screens.

OR

The OR operator is used to combine terms to get more results. Using OR broadens the search. If there isn’t much on your topic, combine terms (e.g.  subject terms & the keyword terms) with OR to find the most resources. Using our example above, we could search for “older adult” OR “above 65” OR geriatric. This search will return more resources than if we just searched for “older adult” by itself. This is one of the ways you can use the synonyms you came up with when determining your search terms. You can also try each synonym individually to determine if that term retrieves any useful information.

AND

The AND operator is used to combine topics and get fewer results. Using AND narrows the search. Searches with AND require both terms to be in each item returned. If one term is contained in the document and the other is not, the item is not included in the resulting list. Using our example above we can combine a search for the problem (prostate cancer) with the action we are undertaking (diagnostic testing or screening). The search entry would look like:

“prostate cancer” AND “diagnostic testing”

Both terms are encased in quotation marks because they are 2-word terms. If we wanted to include the alternate action term “screening” in addition to “diagnostic testing,” we will include the second half of our search in parenthesis like this:

“prostate cancer” AND (“diagnostic testing” OR screening)

NOT

The NOT operator (sometimes AND NOT depending on the coding of the database’s search engine) is used to exclude words from your search. Using NOT narrows the search. Using NOT provides results that contain the first keyword but not the second. When you use NOT you are telling the computer to ignore any resources that include that term, even if it included the original term. Make sure to put your keywords in the correct order when using NOT, as the search results provided will exclude the latter keyword. An example would be if we were wanting to know more about cloning, but didn’t want documents that deal with cloning sheep. That search entry would look like:

cloning NOT sheep

In the image above, the blue shading represents the resources that will be returned with each type of search.

Collecting the Relevant Resources

Citation Engines

Zotero

There are many citation managers to chose from, but Zotero is recommended for the following reasons:

• Zotero is free (you can pay for more storage space)
• Zotero is open-source and run by a non-profit
  • Many popular citation managers are run by publishers that use your data for profit
• Zotero has a great support
• Zotero is easy to use

A Video to Get You Started

Downloading Zotero

Download the two components of Zotero at https://www.zotero.org/download/

• Zotero Desktop: where you can see and organize your citations

• Zotero Connector: a browser plugin that automatically detects content as you browse the web and allows you to save it to Zotero with a single click

The Zotero icon changes depending on what type of material you are looking at. A folder for a list of items and a paper for a journal article.

Adding Items to Zotero

There are two main ways to add item to Zotero:

• Zotero Connector
  • Use the Zotero button to automatically add materials to Zotoro
    • Click on the Zotero icon in your browser toolbar and the document will automatically be downloaded into the Library folder you have open on Zotero Desktop.
  • If you are on a webpage that includes the pdf of an article, Zotero will automatically download the article
  • Watch your Desktop Library to make sure the author and year are included in the metadata
• Adding pdfs
  • You can manually drag-and-drop pdfs into the main window of Zotero desktop to add those item to Zotero

Organizing with Zotero

In the Zotero desktop you can:

• Create folders to organize your resources
• Add tags and notes to items
• Search by keyword to find your resources

Citing with Zotero

To cite in Word or Open Office:

• Enable the Zotero plug in
  • Go to Tools > Add-ons and choose your plug in (you may need to restart your computer)

• Use the Zotero tool bar to add citations and create a bibliography

Zotero Help

Zotero Quick Start Guide – Guide to help you get started with Zotero

Zotero Support – For more in depth help

Hints for Using Zotero with Word on a Mac

After installing Zotero standalone to your computer, Word should show either a new Zotero tab next to the Home, Review, View tabs,

OR it will show a separate tool bar with 5 little red icons just below that tab bar (for Word 2011).

If it does not show up – Double check Word to see if the Zotero plugin is correctly installed:

If that STILL doesn’t work:

1. Open Word Preferences.
2. Click View.
3. Check “Show developer tab” and close the preferences dialog.
4. In the new Developer tab click “Word Add-ins”.
5. Make sure that “Zotero.dotm” is present under Global Templates and Add-ins” and is checked.
   If the “Zotero.dotm” entry isn’t present in the dialog, try reinstalling the Word plugin from Zotero. In Zotero, go to the Cite pane of the Zotero preferences, click the “Reinstall Word Components” button, and then restart Word.

References