Canadian Journal of Nursing Informatics

Information access and systematic reviews: A discussion

by Suzanne Fredericks RN, PhD

Jasna K. Schwind RN, PhD

Kateryna Aksenchuk RN, BScN

Ryerson University


Abstract

Systematic reviews are prevalent across various healthcare disciplines, yet they remain one of the least engaged methodologies in nursing. Increasingly, massive amounts of healthcare studies and information are published in various mediums, which may made it difficult for nurses to keep up with primary research evidence. The purpose of this discussion paper is to present a methodology that can be used to search for relevant materials, sort through large volumes of information, and make decisions regarding possible study selection for review. The intention of this paper is to describe the process involved in mapping out what is known from the existing literature about a specific area of interest, as well as the strategies used to delimit the number and type of materials to be included in a systematic review. An overview of the process of identifying relevant materials to include in a review is presented. Specifically, determining inclusion and exclusion criteria, search strategies, and selecting studies for inclusion in a systematic review are discussed. A case study of an existing systematic review that evaluated interventions for reducing the number of hospital readmissions following heart failure is used to guide this discussion.

Introduction

A systematic review is a research methodology that summarizes the best available evidence in response to specific research questions. Through the conduct of systematic reviews, researchers attempt to identify all relevant published and unpublished work related to a specific topic. The quality of selected work is then assessed to determine appropriateness for study inclusion; while the findings from individual studies are synthesized, interpreted, and presented in an impartial manner to respond to the main study questions (Hemingway, 2009). Depending on the nature of the systematic review, it may include qualitative and quantitative research evidence; be limited to only quantitative evidence (in particular randomized controlled trials), or allow all type of research evidence, even non-empirical based materials. Systematic review findings are often used by clinicians and policy makers when changing practices and to influence decision making.

Even though systematic reviews are prevalent across various healthcare disciplines, they remain one of the least engaged methodologies in nursing (Hemingway, 2009). This may be due to the overwhelmingly large quantity of existing healthcare related materials. The sheer magnitude of the task may seem colossal. The amount of healthcare research published annually is escalating, making it difficult to keep up with primary research evidence. Thus, many nurses may be overwhelmed by the sheer volume of information available and/or may lack the expertise to identify relevant materials suitable for a systematic review.

The purpose of this discussion paper is to present an example of the methodology used to collect relevant materials for an upcoming systematic review. The intention of this paper is to describe the process involved in mapping out what is known from the existing literature about a specific area of interest, and describe the strategies used to delimit the number and type of materials to be included in the review focused on evaluating interventions for hospital readmissions reduction following heart failure (HF).

Overview of the Review Process

 

Determining Inclusion Criteria

The first step in conducting a systematic review is the identification of specific inclusion criteria. A important criteria for this systematic review was that every study should contain a sample that represented patients who were recently (within one year) diagnosed with HF. The criteria included patients with Congestive Heart Failure (CHF) to reduce the likelihood of excluding relevant articles. The authors were aware this inclusion might result in an unmanageable number of references; however, they felt the broader approach would generate a greater breadth of coverage. Decisions about how to set the parameters on large numbers of bibliographic references were made once the volume and the general scope of the field were determined.

Materials were also included in the review if the authors examined or recommended an intervention to reduce hospital readmission rates. From a practical point of view, further parameters were delineated regarding the time span of the studies and the language used. Reflecting time and budget constraints, primary studies released (both published and unpublished) between 1991 and 2011 were considered for inclusion. Also, a 20 year time frame was chosen because the investigation into hospital readmissions has been relatively recent. Foreign language studies were excluded because of the cost and the time involved in translating the material. Due to these inclusion parameters, there was potential risk of missing relevant materials.

 Search Strategies

Following the identification of inclusion criteria, specific search strategies are developed to assist in the creation of search requests; aid in the access of search results; and to allow for a record of result changes and a search history. Search strategies are important to ensure a materials are collected and selected in a consistent manner. A variety of search strategies were used in our HF systemic review: searching electronic databases; hand-searching of prominent journals; and probing existing networks, organizations, and conferences for relevant materials.

The search strategy for the electronic databases was developed from the objectives of the systematic review. The research team worked with a qualified librarian to design and execute database-specific search strategies. The use of the librarian’s services was central to generating the breadth of coverage. Reference lists from studies obtained through the database searches for systematic reviews and traditional literature reviews were examined to ensure they had been included in the study. The citations within the identified studies yielded further references, thus expanding the search parameters.

The main strategy used to search the electronic databases was developed from the objectives of the systematic review. The research team worked with a qualified librarian to design and execute database-specific search strategies. The use of the librarian’s services was central to generating the breadth of coverage. Reference lists from studies obtained through the database searches for systematic reviews and traditional literature reviews were examined to ensure they had been included in the study. The citations within the identified studies yielded further references, thus expanding the search parameters.

If funding to hire a librarian is not available, study investigators need to conduct the electronic databases search themselves. Researchers engaged in search strategies would first need to identify key concepts relevant to the search and keywords to describe these concepts. Keyword identification should be supplemented with a search of any synonyms, related words, or variations of keywords. A consideration of any search features including truncation, proximity operators, or Boolean operators that may influence the search should also be incorporated into the search.

Once keywords have been identified, the researcher chooses a relevant search engine or database. To identify an appropriate search engine, the researcher is encouraged to read the search engine’s homepage for an overview of the particular engine. Upon selection of key search engines, search expressions are then created using syntax that is appropriate for each engine. Examples of syntax can be retrieved from various search engine homepages. The search expressions can then be inserted into the search field. The results obtained are evaluated for relevance. A record should be kept of the number of hits returned and whether or not the results were relevant to the search query. Based on the examination of the results obtained, the search can be modified by using different databases and search expressions (Ackermann & Hartman, 2004).

In addition to searching electronic databases, hand-searching of prominent journals in the field of interest can be employed. With regards to the systematic review in question, hand-searching of print journals including Clinical Nursing Research, Heart and Lung, Journal of Advanced Nursing, Journal of Cardiovascular Nursing, and The European Journal of Cardiovascular Surgery, was conducted to identify articles that may have been missed through the electronic database and reference lists searches. Also, existing networks (specifically, the Cardiac Care Network of Ontario and the Ontario Heart Health Network), societies (British Society for Heart Failure, European Society of Cardiology, European Society of Cardiology Council on Cardiovascular Nursing and Allied Professions [CCNAP], and the Heart Failure Society of America), relevant organizations (Heart and Stroke Foundation of Canada, Heart and Stroke Foundation of Ontario, and American Heart Association), and conferences (Chronic Heart Failure and Hypertension Conference, Canadian Conference of Cardiovascular Nursing, Consensus Conference: Management of Heart Disease, Asian Pacific Congress of Heart Failure, and Pan American Heart Failure Congress) were searched, as existing knowledge and networks served to generate information about interventions to reduce the number of hospital readmissions following HF. Additionally, these networks, societies, organizations, and conferences were contacted by email and telephone to identify pertinent unpublished work.

A total of 798 articles published between 1986 and 2011 were found that addressed heart failure and hospital readmissions. Of these, 736 articles were excluded because of their irrelevance to the topic:  1) they described the current health care environment in which heart failure was one of many conditions putting strain on the health care system (63.2 %), or 2) they described heart failure frameworks and policies guiding practice (36.8 %). A total of 62 studies met the selection criteria and were included in the systematic review.

As anticipated, the search generated hundreds of bibliographic references, which were then appraised to determine possible inclusion in the final study selection. The software, Reference Manager was used to manage the data, keep track of articles, and make requests for inter-library loans. This software was compatible with the word processing package that was used, which assisted in producing lists of references for inclusion in the final literature review report. A trained research assistant kept a record of the databases searched for each set of results imported into Reference Manager. This information was important as it was used to update and refine subsequent searches.

 Study Selection

Following retrieval of pertinent studies, inclusion criteria should be developed to guide the selection of relevant studies. The criteria for study selection should flow from the research question and be specified a priori (Khan, Kunz, Kleijnen & Antes, 2003). The study selection criteria should include selection principals that address the basic elements of a study such as type of design, sampling techniques, instruments, sample size, missing data, and key conclusions. These selection elements are applied to determine whether to accept or reject studies for review. As well, the selection of studies will be influenced by the initial inclusion and exclusion criteria related to HR and readmissions. A log of studies that are excluded should be kept including reasons for rejection.

Reliability assessment for study selection should be employed, in which a percentage of the studies considered for inclusion are evaluated by at least two independent raters. One of the raters should be knowledgeable in the area under review, while the second rater should be a content expert. This reduces relevance and validity bias. In cases of disagreement between reviewers, arbitration by another person is suggested or disagreements can be resolved through a consensus-building process (National Center for the Dissemination of Disability Research, 2007).

In the HF systematic review, two reviewers applied the inclusion criteria to all citations. Studies that met the inclusion criteria were downloaded for further review, including studies that showed potential for inclusion, but needed closer examination to ensure true relevance. Deadlines were set to provide a timeframe for study selection and review. Reviewers read the full articles carefully to determine whether the studies met all inclusion criteria for systematic review.

All data were extracted by two independent reviewers. Standard information was collected on each study. A data charting form was created to record general and specific information about each study. Specific information related to study design, including: study population, type of intervention, and outcome measures for hospital readmission rates. Information was recorded as follows: author(s), year of publication, study location, intervention type (and comparator, if any), duration of the intervention, study population, aims of the study, methodology, outcome measures, and important results. Based on the data extracted, decisions were made concerning study inclusion or exclusion in the final systematic review.

Consultation Exercise

 

A final step to ensure a quality systematic review is to incorporate a consultation stage.  A consultation element was included in or HF and readmissions systematic review. People consulted included practitioners (cardiovascular, surgical-focused advance practice nurses, cardiovascular surgeons, a family practitioner, a rehabilitation practitioner, an emergency room physician and nurses), patients, and hospital administrators from local organizations. It was anticipated that contributors to the consultation would provide additional references about potential studies to include in the review, and give valuable insights about effectiveness issues of interventions aimed at reducing hospital readmissions.

Conclusions

In conclusion, a detailed overview of the procedure involved in obtaining relevant materials for systematic review was presented. A case study approach was used to present the steps in conducting the HF and hospital readmissions review. This approach presented techniques involved in determining inclusion and exclusion criteria, search strategies, and study selection procedures. The intent of this case study approach is to outline the steps and address the challenges that may arise during the accessing, sorting through, and management of papers collected in a quality systematic review. An important aim of this paper is to encourage other nurses to engage in rigorous systematic reviews that facilitates access and adherence to primary health evidence.

 References

Ackermann, E. & Hartman, K. (2004). Search strategies for search engines. Franklin, Beedle and Associates, Incorporated, Wilsonville, OR. Retrieved from http://webliminal.com/search/search-web05.html

Hemingway, P. (2009). What is a systematic review? Evidence based medicine. Haywood Group. Ltd. Retrieved from http://www.medicine.ox.ac.uk/bandolier/painres/download/whatis/Syst-review.pdf.

Khan, K. S., Kunz, R., Kleijnen, J. & Antes, G. (2003). Five steps to conducting a systematic review. Journal of the Royal Society of Medicine, 93, 3, 118-121.

National Center for the Dissemination of Disability Research. (2007). Appraising the quality of systematic reviews. Technical brief number 17. Retrieved from  http://www.ncddr.org/kt/products/focus/focus17/

Author Details

 Suzanne Fredericks RN, PhD

* Please direct questions related to article to this author*

Associate Professor

Daphne Cockwell, School of Nursing, Ryerson University

350 Victoria St.

Toronto, ON; M5B 2K3

Office: 416-979-5000 ext. 7978; sfrederi@ryerson.ca

Fax: 416-979-5332

Jasna K. Schwind RN, PhD

Associate Professor

Daphne Cockwell, School of Nursing, Ryerson University

350 Victoria St.

Toronto, ON; M5B 2K3

Office: 416-979-5000 ext. 6321; jschwind@ryerson.ca

Fax: 416-979-5332

Kateryna Aksenchuk RN, BScN

Research Assistant

Daphne Cockwell, School of Nursing, Ryerson University

350 Victoria St.

CJNI Editor: June Kaminski

 

Back to Issue Index

Be Sociable, Share!

Comments are closed.