• Users Online: 454
  • Home
  • Print this page
  • Email this page
Home About us Editorial board Ahead of print Current issue Search Archives Submit article Instructions Subscribe Contacts Login 

 Table of Contents  
Year : 2017  |  Volume : 51  |  Issue : 3  |  Page : 220-232

Systematic reviews and meta-analysis in evidence-based orthodontics: An appraisal and methodology

Asso. Prof. and Head, Department of Orthodontics, Melaka Manipal Medical College, Melaka, Malaysia

Date of Submission11-Apr-2017
Date of Acceptance17-Apr-2017
Date of Web Publication17-Jul-2017

Correspondence Address:
Priti Mulimani
Department of Orthodontics, Melaka Manipal Medical College, Melaka
Login to access the Email id

Source of Support: None, Conflict of Interest: None

DOI: 10.4103/jios.jios_36_17

Rights and Permissions

Orthodontics is not only an art but also a science, and the key to practicing it likewise lies with implementing evidence-based orthodontics (EBO). Systematic reviews (SRs) and Meta-analyses (MAs) are considered to be the “gold standard” or highest level of evidence in EBO. The limitations of traditional, narrative reviews have led to the increased popularity of SRs due to their objective and transparent methodology, which makes the evidence generated by them more reliable and unbiased. With SRs and MAs becoming the mainstay of Orthodontic research and EBO, it is imperative to be in the know of the technicalities of these tools of research synthesis. The present paper provides an overview of the SR methodology and describes in detail the key steps in conducting SRs or MAs viz., formulating the review question, defining the criteria for inclusion of studies, identifying all relevant studies by a meticulous search process, including studies which meet pre-defined criteria, assessing quality of included studies and finally analysing and interpreting the collective findings from included individual studies. The paper can thus be used as a guide for the whats, whens, whys and hows of carrying out a systematic review and meta-analysis, and generate the highest quality of evidence available for delivering the best patient care.

Keywords: Evidence-based orthodontics, meta-analysis, methodology, systematic review

How to cite this article:
Mulimani P. Systematic reviews and meta-analysis in evidence-based orthodontics: An appraisal and methodology. J Indian Orthod Soc 2017;51:220-32

How to cite this URL:
Mulimani P. Systematic reviews and meta-analysis in evidence-based orthodontics: An appraisal and methodology. J Indian Orthod Soc [serial online] 2017 [cited 2019 May 25];51:220-32. Available from: http://www.jios.in/text.asp?2017/51/3/220/210911

  Introduction Top

Evidence-based practice(EBP) is defined as the conscientious, explicit, and judicious use of current best evidence when making decisions about the care of a patient.[1] Clinical expertise, best research evidence and patient preferences or values form the triad of evidence-based orthodontics(EBO). With time, as more and more studies are added to the scientific literature, the body of evidence keeps expanding and it becomes challenging to keep track of the constant developments and new findings. Quite often the varied and sometimes contradictory findings of research studies confound their clinical application by making the overall evidence unclear and difficult to interpret. Thus, a barrier is created between the existing research evidence and its utility in improving patient care.

  Appraisal of the Background and Context for Systematic Reviews Top

In EBP, systematic reviews(SRs) bridge the gap between research and practice by providing the missing link.[2] Reviews are studies of studies or secondary research tools which help to summarize existing data from individual studies, reduce research waste, refine hypothesis, develop questions for further research where answers are lacking or unclear, and make scientific evidence more accessible to clinicians and policy makers. They are said to have methodological superiority over primary research or single studies, which can be biased, methodologically flawed, time and context dependent, and can be misinterpreted and misrepresented.[3]

Historically, the need for synthesizing existing research or summarizing studies was found to exist since the 18thcentury itself. Publication of a review paper by Karl Pearson in 1904 in British Medical Journal and the methodical review and statistical analysis of studies on typhoid fever by Goldberger in 1907 are considered to be the beginning of synthesizing research in a scientific way, as we know it today.[4] However, in the early days, reviews were more of a descriptive nature and lacked rigorous methodology, prestated objectives or data analysis which made them prone to bias, thus raising concerns about the reliability of their conclusions. Such reviews are known as traditional or narrative reviews, and these have been criticized for being unsystematic, failing to critically evaluate findings or methodology of included studies and selectively reporting studies supporting authors' hypothesis thus leading to questionable or erroneous conclusions.[5]

With the rise of EBP in the 1990s, SRs started to gain prominence as reliable instruments of evidence detection. Although the term SR was used first in the 1970s, it was only in the late 1990s that it became more widely used.[4] The terms research synthesis, overview, and meta-analysis have been used interchangeably with SR due to the lack of clarity and ambiguity in the past. Last's Dictionary of Epidemiology(2001) defines a “Systematic Review(SR)” as the application of strategies that limit bias in the assembly, critical appraisal, and synthesis of all relevant studies on a specific topic.[6] Thus, the core principle, which differentiates SRs from narrative reviews, is the emphasis on an objective, meticulous, unprejudiced and transparent approach in conducting, and reporting the methodology. Other important differences between SRs and narrative reviews are shown in [Table1].[7] Meta-analysis may be, but is not necessarily, used as part of this process. The term “meta-analysis” was coined by Glass in 1976 to refer to the quantitative synthesis of the results of primary studies.[8] “Meta-Analysis” is defined as the statistical synthesis of the data from separate but similar, i.e.,comparable studies, leading to a quantitative summary of the pooled results.[6]
Table 1: Difference in methodology between systematic reviews and traditional narrative reviews

Click here to view

The key to producing a good SR is by implementing a standardized methodology, which ensures that the resultant conclusions are comparable, unbiased, and reliable. Such guidelines are provided by international bodies working to enhance the quality of SRs such as the Cochrane Collaboration [9] and Centre for Reviews and Dissemination(CRD).[10] Cochrane was founded as an international, independent, not-for-profit collaboration of researchers, professionals, patients, caregivers, and people interested in health by Iain Chalmers in 1993 in honor of Archie Cochrane, a British medical researcher who had first envisioned and highlighted the necessity of collecting and publishing critical reviews of knowledge from randomized controlled trials(RCTs). Its objective itself is to gather, summarize, and support generation of the best evidence from research to help make informed choices about treatment through Cochrane reviews. To fulfill this objective Cochrane maintains, an online library of which the Cochrane Database of Systematic Reviews(CDSR) is the most important part and it consists of high-quality Cochrane reviews on various important health topics.[9],[11] The Cochrane Handbook for Systematic Reviews [12] and CRD's guidance for undertaking reviews in health care [13] are extremely comprehensive sources which provide the most intricate and scientifically evaluated framework of guidelines on every aspect of planning and executing a SR. To propagate and support high-quality reviews, leading scientific journals have made it mandatory for reviews that are submitted for publication, to fulfill a checklist of 27 items which is known as the Preferred Reporting Items for Systematic reviews and Meta-Analyses(PRISMA) statement.[14] The checklist[Figure1][14] outlines the necessary methodological steps a review should report so that it can be comprehensively assessed for validity.
Figure 1: PRISMA checklist of items to include when reporting a systematic review or meta‑analysis (reproduced from Moher et al., 2009[14] open access)

Click here to view

  Methodology for Conducting a Systematic Review Top

Select a review topic

Before undertaking a SR, it is necessary to ask the question “Is a review really required on this topic?” Check whether there are already existing or ongoing reviews by searching scientific literature including specialty journals, PubMed, and internet, particularly databases such as CDSR [15] and Database of Abstracts of Reviews of Effects.[16] Advice can be obtained from clinical or methodological experts or stakeholders, patients, etc., to identify lacunae or gaps in knowledge where answers from a review will be beneficial. Once the need to produce a new review is justified, one can proceed to the next step.

Establish a review group

This implies organizing a group of authors to carry out the SR. There should be at least two authors in the group so that measures to minimize bias and error can be implemented at all stages of the review. The group would benefit to have authors who can provide expertise in SR methods, information retrieval, the relevant clinical/topic area, and statistics. Conflicts of interest of individual authors should be taken into account, as these have to be declared as part of the review writing process to ensure transparency. In case of Cochrane systematic reviews, the next step is to register the “title” of the review and its authors, with the relevant Cochrane review group, by following specific guidelines lay down by them. Cochrane Oral Health Group [17] is the one that handles all dentistry and orthodontic Cochrane reviews.

Develop a protocol

Good SRs are carried out in two stages–protocol stage and review stage. The protocol stage represents the planning stage and the review stage represents the execution stage. Areview “protocol” outlines the methods which will be used to carry out the review a priori or in advance. The objective of this is to eliminate bias, arbitrariness, or changing the outcomes being investigated according to findings available or not available. Aprotocol usually lays down methodology for the following:

  1. Developing the review question
  2. Defining inclusion criteria for studies in terms of

    • Types of studies
    • Types of participants
    • Types of interventions
    • Types of outcome measures.

  3. Searching for studies
  4. Selection of studies to be included in the review
  5. Data extraction from individual studies
  6. Quality assessment
  7. Data synthesis and analysis.

Developing the review question

The methodological section of a protocol begins with asking the right review question, which can be of two types [18]–”Background” questions make very broad-based, general enquiries regarding a disease, condition or a specific topic. These have two essential components; first, a question root(who, what, when, etc.) with a verb and second, the disorder, test, treatment, or other related aspects. Example of a review asking such a question is–”What are patients' expectations of orthodontic treatment.”[19] “Foreground” questions are very narrow, and they seek specific and focused knowledge about managing patients with certain conditions and have three or four essential components. Sackett et al. suggested the use of PICO, a mnemonic, for formulating such questions, whereP stands for participants or patients, I for intervention, C for comparisons, and O for outcome. An example, of such a question is–”In patients undergoing orthodontic treatment for congenitally missing lateral incisors, will space closure and substitution with canines produce a better esthetic and functional result as opposed to space opening and replacement with prosthesis?” In this scenario, the participants are patients with congenitally missing lateral incisors, intervention is orthodontic treatment, comparison is between the two methods, viz., space closure and substitution with canines versus space opening and replacement with prosthesis, and outcomes are esthetic results and functional efficiency. Articulation of clinical problem in PICO format is said to generate the closest, most relevant and best matches while searching the literature for evidence.[20]

Defining inclusion criteria for studies

Next, the criteria determining inclusion of studies in the review have to be stated explicitly and as specifically as possible, in terms of study design and PICO. Since the underlying principle of a review, is that its conclusions should be based on best available evidence, most of the reviews include only RCTs since they are considered to be the gold standard for testing interventions. However, other study designs may also be included depending on the topic of investigation. However, including less rigorous study designs introduces confounding variables, hence their inclusion should have a sound justification rooted in scientific rationale and not merely due to the reason that relevant randomized studies are lacking on that topic in literature. Identification of gaps and lack of good trials in the current evidence base in itself can be a significant finding of the review, since many international funding bodies often look for documented confirmation that evidence is lacking in a certain area, before they can fund more studies in that area.

Provide specific details for each of the PICO components such as age of participants(ex. adults as more than 18years of age), parameters for verifying the condition of interest(ex. skeletal ClassIII as ANB below 1°), levels of severity of condition that need to be addressed (ex. severe skeletal Class III as ANB less than-6°), specific dose of intervention (ex. face-mask wear for at least 12h in a day), time periods being tested(ex. over a period of at least 6 months), definition and thresholds of primary and secondary outcomes being assessed including valid instruments to measure these(ex. increase in ANB by 3° in cephalometric analysis or reduction of reverse overjet by at least 2mm measured clinically or on study models). The nature of the intervention(s) (ex. Delaire face mask without RME) and comparator(s) (ex. tandem traction bow or Delaire face mask without RME) should be specified in detail. An operational definition describing the content and delivery of the intervention will usually be helpful. It is usually recommended to also include measures of adverse effects (ex. decalcification or root resorption while testing efficacy of certain appliance) and measures important to patients (ex. cost of intervention, quality of life, perception) within the review. An illustration of how this is written in the protocol is provided in [Table2],[21] which has been adapted from our Cochrane protocol–”Ergonomic interventions for preventing musculoskeletal disorders in dental care practitioners.”[21]
Table 2: Detailed criteria for including studies in the review to be laid down in protocol stage – illustration

Click here to view

Searching for studies

After detailing which studies will be included, the protocol has to state how the authors are going to search for them. This entails clarifying which resources or databases will be searched–both electronic and physical, and the “search terms” that will be used to retrieve the search results electronically. Search sources may include electronic databases such as MEDLINE,[22] EMBASE,[23] registries of clinical trials like Cochrane Central Register of Controlled Trials,[24] WHO International Clinical Trials Registry Platform;[25] specialty specific databases and journals; reference lists from relevant studies; citations; conference proceedings; relevant resources from internet; peers, study authors, experts, manufacturers, and other organizations.

Electronic databases need to be communicated with, using a syntax specific to them during the search process, to retrieve all (sensitivity) and relevant(specificity) results. This involves using keywords, all possible synonyms, specific symbols, and BOOLEAN operators (such as AND, OR, and NOT). Designing a method including all these components is known as the “search strategy.” Different databases will have different search strategies customized to their search platform and syntax. Designing a proper search strategy can be quite technical. The database help sections often provide a guide on how to carry out the search, also the Cochrane handbook comprehensively describes how to search various resources for studies. Cochrane review groups may have dedicated Trial Search Coordinators to help authors conduct the search process. Apreliminary search strategy for at least one database should be included in the protocol, including all the other sources that will be searched relevant to the research question. The search strategy, terms used, databases to be screened should be stated from the outset so that the methodology is transparent and can be replicated by anyone to obtain same results. Ashort version of the sample search strategy for Medline for the protocol on ergonomic interventions is shown in [Table3].[21]
Table 3: An example of search strategy

Click here to view

Selection of studies to be included in the review

Studies retrieved by the search results have to be assessed for their eligibility to be included in the review. The protocol should outline how these decisions will be made by clearly stating the inclusion and exclusion criteria(based on PICO), number of researchers who will screen titles and abstracts and then full papers, and the method for resolving disagreements about study eligibility.

Data extraction from individual studies

The studies identified to be included in the review now to need to be read in depth and research data from each of them collected. This is known as data extraction. In this section, provide details of what all information will be retrieved from included studies, which authors will do it and how they will do it, i.e.,the method used for recording data-either software or data extraction forms. It is important to identify authors who will extract the data a priori and also to put in place a method to resolve disagreements and discrepancies which are known to commonly arise. Information should also be provided on how missing data in the included studies will be dealt with–whether authors of primary studies will be contacted to provide missing or additional data and how much time will be allotted to wait for their replies. If there are no language restrictions for included studies, then arrangements for translation of foreign language papers have to be specified.

Quality assessment

Quality assessment is necessary to answer the question whether the studies included are robust enough to guide treatment, prevention, diagnostic, or policy decisions. It may include evaluation of appropriateness of study design to the research objective, quality of intervention, manner or experience of operators delivering intervention, choice of outcome measure, or statistical issues. An important part of this assessment is also to determine the risk of bias of included studies. Abias is a systematic error, or deviation from the truth, in results or inferences. It could lead to underestimation or overestimation of the true intervention effect and may be due to poor study design or conduct in the collection, analysis, interpretation, publication or review of data. Various tools and methods are available to assess quality and risk of bias and which exactly will be used for the particular review has to be highlighted.[12],[13]

Data synthesis and analysis

In this section, methodology has to be laid down for how data from results of included studies will be treated and which statistics will be used to summarize findings of individual studies. Choice of statistics depends on the type of variables used to assess the outcomes like continuous or categorical. Results generated may also be subjected to additional analyses to detect variations in observed findings(heterogeneity analysis) or to ascertain exact causes of observed variation among included studies(by doing subgroup analysis or meta-regression). Detailed elaboration on data synthesis and analysis is provided in “Undertaking the review” section of this article.

Approval and registration of protocol

Once the protocol is ready, it may or may not undergo the registration process, before proceeding to the actual review stage. Databases which accept registration and maintain records of registered protocols are the Cochrane library [16](for review titles or topics registered with Cochrane) and PROSPERO, which is an international database of prospectively registered SRs(also includes Cochrane protocols).[26] Although registration of protocols is not a mandatory requirement for scientific publications as of now, the registration process in an international database helps in eliminating unplanned duplication of SRs by other authors, reducing research waste and resources and encourages transparency in the review process. For Cochrane reviews, the protocol undergoes the peer review process. Based on feedback and suggestions obtained from the process, corrections are made at this stage to further refine the methodology which ensures production of a high-quality review.

Undertaking the review

In the protocol stage, the methodology is planned, and in review stage, the planned methodology is executed. The blueprint laid down in the protocol is now implemented to build the review through the following stages:

  1. Searching databases
  2. Study selection
  3. Data extraction
  4. Quality assessment of included studies
  5. Data synthesis
  6. Data analysis
  7. Results
  8. Discussion
  9. Conclusions
  10. Dissemination.

Searching databases

Execute the search methodology mentioned in the protocol to generate search results, from the planned sources. All the prespecified resources are thoroughly searched by implementing the search strategy for each. Whether the process constitutes searching electronic databases, internet websites, hand searching of journals, or emails or letters for asking information from experts or authors, maintaining records of the same is extremely important for future references and to verify the appropriateness and thoroughness of the entire search process. For each database that is explored, dates, terms used, entire search strategy, and number of records retrieved should be meticulously documented so that it could be rerun or cross-checked at a later date. Search results can be downloaded and stored either in reference management software or Microsoft Word or Excel.

Study selection

The number of studies thrown up in the search results may vary depending on the review topic, ranging from few thousands to none. In any case, the search result would consist of many irrelevant, unrelated or unsuitable studies, which require to be screened based on the prespecified inclusion and exclusion criteria and obtain studies appropriate for the review. The aim of a good selection process is to ensure that all relevant studies are included using a process which is explicit, objective and minimizes risk of errors and biases. The study selection is usually done in two stages. In the first stage, the search results are given a preliminary screening to reject obviously irrelevant ones and to identify those relevant for further detailed assessment. The second stage consists of retrieving full reports of apparently relevant studies identified in the first stage for a detailed assessment and from this list, studies which meet requirements for inclusion are finalized. Care should also be taken to identify and eliminate duplicate publications and multiple papers of the same study at this stage.

Obtaining full text of studies for assessment from various sources can be very labor and time intensive. The difficulty level of this part depends on the affiliations and organizational arrangements for accessing scientific literature of the authors. It is usually preferable to have one person from the team responsible for the library of references. Selection of studies should be implemented through parallel independent assessments by at least two reviewers to minimize the risk of errors. Disagreements between them regarding inclusions have to be resolved either using consensus or arbitration by a third person, whichever method has been prestated in the protocol.

Data extraction

For data extraction, standardized data extraction forms customized for their review can be designed by authors. These can be pilot tested first to check their adequacy and to see if all authors are understanding and using the form correctly. Usually, one researcher extracts the data while at least one other researcher independently checks the data extracted for accuracy and detail. If disagreements occur between assessors, they should be resolved according to a predefined strategy using consensus and arbitration as appropriate. Asample outline of a data extraction form is provided in [Table4].[13]
Table 4: Details to be included in a data extraction form Example information requirements for data extraction

Click here to view

Quality assessment of included studies

Not all studies of the same design are equal. There may be discrepancies in methodology and adherence to format true to the type of study being carried out. Hence, quality assessment of included studies is important to check the methodological soundness and reliability of results, since this determines the strength of evidence, which is instrumental to make clinical recommendations based on the findings of the review. Grading of Recommendations Assessment, Development and Evaluation tool used in Cochrane reviews, rates quality of evidence as high, moderate, low, or very low quality and strength of recommendations as strong and weak. This assessment takes into account the risk of bias of the included studies, the directness of the evidence, the inconsistency of the results, the precision of the estimates, and the risk of publication bias.[27]

In clinical trials, biases can be broadly categorized as selection bias, performance bias, detection bias, attrition bias, reporting bias, and other biases that do not fit into these categories. Biases can be assessed using scales or checklists. However, the Cochrane Collaboration's recommended tool for assessing the risk of bias consists of a domain-based evaluation and is neither a scale nor a checklist. The domains in a standard “Risk of bias” table of a Cochrane review are sequence generation(selection bias), allocation sequence concealment(selection bias), blinding of participants and personnel(performance bias), blinding of outcome assessment(detection bias), incomplete outcome data(attrition bias), selective outcome reporting(reporting bias), and other potential sources of bias. Each study can be classified as having low, unclear or high risk of bias, based on assessment of these domains, which represent plausible bias unlikely to seriously alter the results, plausible bias that raises some doubt about the results, and plausible bias that seriously weakens confidence in the results, respectively.[12]

Data synthesis

Once research data from individual studies have been extracted and collected, it now has to be analyzed as a whole, to draw meaningful conclusions. This is known as data synthesis and it usually involves summarizing of the findings of individual studies to provide summary statistics in the first stage. Summary statistics used depend on the type of outcomes being assessed and these are presented in [Table5]. In the second stage, if it is possible to do so, then data from all studies are pooled together and subjected to formal statistical techniques to detect combined effect estimate of all studies which is known as meta-analysis(quantitative synthesis). [Table6] provides detailed information about the why, what, when, and how of meta-analysis.
Table 5: Statistical tests used for summarizing study outcomes

Click here to view
Table 6: Meta‑analysis

Click here to view

If formal pooling of results is inappropriate, then a narrative discussion of findings should be presented. The general framework for narrative synthesis consists of four elements:[12]

  1. Developing a theory of how the intervention works, why and for whom
  2. Developing a preliminary synthesis of findings of included studies
  3. Exploring relationships within and between studies
  4. Assessing the robustness of the synthesis.

Meta-analysis provides a statistical method to address elements 1–3. Narrative synthesis uses subjective (rather than statistical) methods to follow through elements 1–4.

Data analysis

The synthesized data can now be scrutinized, to look for similarities or differences in the treatment effects in individual studies, to understand how the treatment works. Different studies can never be identical and will always have some variations between them, which is referred to as “heterogeneity” and is said to be of three types–(1) clinical heterogeneity–due to variations in participants, interventions, and outcomes; (2) methodological heterogeneity–due to variability in study design and risk of bias;(3) statistical heterogeneity is the variability in effects or results observed in different studies, which is more than one would expect due to random error or chance alone; could be attributed to both clinical and methodological diversity, and this is often simply referred to as “heterogeneity.”[12]

The purpose of detecting this is because presence of heterogeneity indicates variation and variation could be due to different reasons or just by chance. Tests for heterogeneity can indicate whenever variation is arising due to some factor in the study and not due to some random chance. Knowing this helps to explore these factors and trace their effects on intervention or treatment. Heterogeneity can be detected with statistical tests like Chi-squared test and I2. To analyze if variation is due to specific participants or intervention, subgroup analysis can be undertaken. And to zero down on factors causing these variations, meta-regression can be carried out.[12],[13]

Often unexpected issues are encountered while executing the methodology and decisions have to be taken which may not be entirely clear. For example, where a majority but not all people in a study meet the age range, should the study be included? Should analyses be based on change of scores or final values? This often creates the sense of doubt as to how would the results be affected if an alternative decision was taken instead. To explore these possibilities, a sensitivity analysis is performed. For this, the meta-analysis is repeated informally, by substituting alternate decisions or ranges of values for unclear decisions. For example, if one suspects a particular study is biased, then that study is removed and meta-analysis is repeated with other included studies. If the analyses show that the overall result and conclusions are not affected by these substitutions, then results of review can be regarded as robust, reliable or certain. Otherwise, they have to be treated with caution.[12],[13]

  Results Top

The following is the to-do list for presenting the results:

  1. Summarize search results–by using PRISMA study flow diagrams to illustrate the results of the search and the process of screening and selecting studies for inclusion in the review[Figure2][14]
  2. Describe included studies–under subheadings of trial settings, participant and investigator characteristics, interventions used, comparisons made, findings of treatment effects and their
  3. Quality or risk of bias assessment
  4. Cite reasons for excluding certain studies or other clarifications
  5. Provide tables and graphs to depict the following–

    1. Details of included studies
    2. Outcome data from individual studies
    3. Quality assessment of individual studies
    4. Summary of findings of the review including cumulative information, data, and quality of evidence for the most important outcomes
    5. Summary statistics and meta-analysis-forest plots are one of the most commonly used tools to graphically display summary statistics and meta-analysis results in SRs[Figure3].[30] Aforest plot provides a visual representation of the main findings of individual studies including statistical significance of effects, confidence intervals, effect size, and also their pooled effect. Its horizontal axis represents effect size and the vertical axis represents no effect and is known as “line of null effect.” Effect estimates and confidence intervals for individual studies are then displayed with reference to these axes. Each study is represented by a block at the point estimate of intervention effect with a horizontal line extending on either side of the block. The area of the block indicates the weight assigned to that study, larger the block, larger is its weight in the meta-analysis while the horizontal line depicts the confidence interval(usually with a 95% level of confidence). The pooled effect generated from the meta-analysis is represented as diamond instead of a block with confidence intervals.[12],[30]

  Discussion Top

In discussion, provide a brief overview of main results and findings of the review, comment on the overall completeness and applicability of evidence, present the quality of evidence, state potential biases or limitations of the review, and discuss agreements or disagreements of review with other similar reviews.
Figure 2: PRISMA recommended flow of information through the different phases of a systematic review (reproduced from Moher et al., 2009[14] open access)

Click here to view
Figure 3: Interpretation of a forest plot (reproduced from Bigby, 2014[30] open access)

Click here to view

  Conclusions–implications for Practice and Future Research Top

Conclusions should be made with care and caution since it has far-reaching implications for clinical application. The decision can be based on judicious consideration of four factors-the quality of evidence, the balance of benefits and harms, values and preferences, and resource utilization.[12] If there is a lack of evidence or studies, the same should be mentioned and suggestions for refining future research should be made. It is important to use precise terminology and interpretation. Acommon error is to confuse “evidence of no effect” with “no evidence of effect.” “Evidence of no effect” implies evidence exists and it proves that there is no effect, whereas “no evidence of effect” implies that there is a lack of evidence to either prove or disprove the effect.

  Dissemination Top

After review completion, findings have to be disseminated by publishing it and making it visible to provide new information to stakeholders.

Additional considerations

Producing a review is a laborious, complex, and meticulous process. Documenting decisions made by each reviewer during selection process will help in reminding why certain decisions were taken, rationale behind them and also to maintain the records future reference. Reference management software-EndNote, Reference Manager, or ProCite to record and manage references will help in documenting the process, streamline document management and to make the production of reference lists for reports and journal papers easier. Web-based systems, such as Covidence and EPPI-Reviewer, help in managing all stages of review process. RevMan and TrialStat SRS are other software that provides a platform supporting review preparation in all stages. Although these softwares make review writing very convenient by providing the required format and tools, some of them may not be accessible to everyone or may require subscription. In such cases general word processing packages, spreadsheets and databases can be used and customized by review authors.

Sometimes, during the review process, one may encounter issues not anticipated during the protocol stage. In such cases, a deviation from the protocol may be required. Any changes or deviation from protocol should be scientifically justifiable and clearly documented in the final review. At no cost, modifications should be made due to awareness of the results of individual studies or to redirect the course of review toward biased outcomes.

Good statistical support is very important in executing an SR where data have been analyzed and evaluated in an appropriate and precise manner. Astatistician with the necessary understanding and experience can hence be included as a coauthor in the review group or alternatively the review authors should regularly consult one throughout the review process.

Concluding remarks

It is a common grouse that SRs do not reach any conclusion, and most of the times they cite that there is insufficient evidence or there are no good studies. Grant and Booth [31] have mentioned that a meta-analysis cannot be better than its included studies allow. Similarly, a SR can only provide the answers that its included studies allow. It cannot generate any new data on its own unless the primary research has detected it. Hence, the onus is not only on generating good SRs but also on improving the quality of primary research which add to the quality of existing knowledge base and are not merely increasing the quantity and contributing to research waste. Our specialty of orthodontics is not only the first specialty created in the field of dentistry but also a pioneer in research and EBP. In current times with easy access to technology and internet, our patients are very aware and well informed. What differentiates real orthodontists from short-course orthodontic service providers is our ability to provide best answers and solutions to our patients based on our deep and strong foundations of orthodontic practice, theory and research rooted in sound principles of EBO, which makes it all the more important for us to be well-versed with the highest level of evidence in the form of systematic reviews and meta-analysis.

Financial support and sponsorship


Conflicts of interest

There are no conflicts of interest.

  References Top

SackettDL, RosenbergWM, GrayJA, HaynesRB, RichardsonWS. Evidence based medicine: What it is and what it isn't. BMJ 1996;312:71-2.  Back to cited text no. 1
AnkemK. Evaluation of method in systematic reviews and meta-analyses published in LIS. Libr Inf Sci 2008;32:91-104.  Back to cited text no. 2
WilsonP, Petticrew M; Medical Research Council's Population Health Sciences Research Network Knowledge Transfer Project Team, CalnanM, NazarethI. Why promote the findings of single research studies? BMJ 2008;336:722.  Back to cited text no. 3
ChalmersI, HedgesLV, CooperH. Abrief history of research synthesis. Eval Health Prof 2002;25:12-37.  Back to cited text no. 4
MulrowCD. The medical review article: State of the science. Ann Intern Med 1987;106:485-8.  Back to cited text no. 5
LastJM. ADictionary of Epidemiology. Oxford: Oxford University Press; 2001.  Back to cited text no. 6
PetticrewM. Systematic reviews from astronomy to zoology: Myths and misconceptions. BMJ 2001;322:98-101.  Back to cited text no. 7
GlassGV. Primary, secondary and meta-analysis of research. Educ Res 1976;10:3-8.  Back to cited text no. 8
Cochrane. Available from: http://www.cochrane.org/.[Last accessed on 2017Feb15].  Back to cited text no. 9
University of York, Centre for Reviews and Dissemination. Available from: https://www.york.ac.uk/crd/.[Last accessed on 2017Feb15].  Back to cited text no. 10
Cochrane Library. Available from: http://www.cochranelibrary.com/.[Last accessed on 2017Feb15].  Back to cited text no. 11
Cochrane Handbook for Systematic Reviews of Interventions. Available from: . [Last accessed on 2017Feb15].  Back to cited text no. 12
Systematic Reviews. CRD's Guidance for Undertaking Reviews in Health Care. Available from: https://www.york.ac.uk/media/crd/Systematic_Reviews.pdf.[Last accessed on 2017Feb15].  Back to cited text no. 13
MoherD, LiberatiA, TetzlaffJ, Altman DG; PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. PLoS Med 2009;6:e1000097.  Back to cited text no. 14
The Cochrane Library. Cochrane Database of Systematic Reviews(CDSR). Available from: http://www.cochranelibrary.com/cochrane-database-of-systematic-reviews/.[Last accessed on 2017Feb15].  Back to cited text no. 15
CRD Database. Available from: https://www.crd.york.ac.uk/CRDWeb/Homepage.asp.[Last accessed on 2017Feb15].  Back to cited text no. 16
Cochrane Oral Health Group. Available from: http://www.oralhealth.cochrane.org/.[Last accessed on 2017Feb15].  Back to cited text no. 17
Asking a Research Question–FGDP(UK). Available from: http://www.fgdp.org.uk/_assets/part_1%20asking%20a%20research%20question.pdf.[Last accessed on 2017Feb15].  Back to cited text no. 18
YaoJ, LiDD, YangYQ, McGrathCP, MattheosN. What are patients' expectations of orthodontic treatment: A systematic review. BMC Oral Health 2016;16:19.  Back to cited text no. 19
RichardsonWS, WilsonMC, NishikawaJ, HaywardRS. The well-built clinical question: A key to evidence-based decisions. ACP J Club 1995;123:A12-3.  Back to cited text no. 20
MulimaniP, HoeVC, HayesMJ, IdicullaJJ, AbasAB, KaranthL. Ergonomic interventions for preventing musculoskeletal disorders in dental care practitioners(Protocol). Cochrane Database Syst Rev 2014;8:CD011261. doi: 10.1002/14651858.CD011261.  Back to cited text no. 21
MEDLINE–Bibliographic Medical Database. Available from: https://www.nlm.nih.gov/bsd/pmresources.html.[Last accessed on 2017Feb15].  Back to cited text no. 22
Elsevier. EMBASE–Biomedical Database. Available from https://www.elsevier.com/solutions/embase-biomedical-research. [Last accessed on 2017Feb15].  Back to cited text no. 23
Cochrane Central Register of Controlled Trials(CENTRAL). Available from: http://www.cochranelibrary.com/about/central- landing-page.html. [Last accessed on 2017Feb15].  Back to cited text no. 24
World Health Organization. International Clinical Trials Registry Platform(ICTRP). Available from: http://www.who.int/ictrp/search/en/.[Last accessed on 2017Feb15].  Back to cited text no. 25
PROSPERO–International Prospective Register of Systematic Reviews. Available from: https://www.crd.york.ac.uk/PROSPERO/.[Last accessed on 2017Feb15].  Back to cited text no. 26
OxmanAD, GRADE Working Group. Grading quality of evidence and strength of recommendations. BMJ 2004;328:1490-4.  Back to cited text no. 27
ThiruvenkatachariB, HarrisonJE, WorthingtonHV, O'BrienKD. Orthodontic treatment for prominent upper front teeth(classII malocclusion) in children. Cochrane Database Syst Rev 2013;11:CD003452. doi: 10.1002/14651858.CD003452.pub3.  Back to cited text no. 28
Minami-SugayaH, Lentini-OliveiraDA, CarvalhoFR, MachadoMA, MarzolaC, SaconatoH, etal. Treatments for adults with prominent lower front teeth. Cochrane Database Syst Rev 2012;5:CD006963. doi: 10.1002/14651858.CD006963.pub2.  Back to cited text no. 29
BigbyM. Understanding and evaluating systematic reviews and meta-analyses. Indian J Dermatol 2014;59:134-9.  Back to cited text no. 30
[PUBMED]  [Full text]  
GrantMJ, BoothA. Atypology of reviews: An analysis of 14 review types and associated methodologies. Health Info Libr J 2009;26:91-108.  Back to cited text no. 31


  [Figure1], [Figure2], [Figure3]

  [Table1], [Table2], [Table3], [Table4], [Table5], [Table6]


Similar in PUBMED
   Search Pubmed for
   Search in Google Scholar for
 Related articles
Access Statistics
Email Alert *
Add to My List *
* Registration required (free)

  In this article
Appraisal of the...
Methodology for ...
Article Figures
Article Tables

 Article Access Statistics
    PDF Downloaded195    
    Comments [Add]    

Recommend this journal