Virtual reality training for surgical trainees in laparoscopic surgery

Myura Nagendran; Kurinchi Selvan Gurusamy; Rajesh Aggarwal; Marilena Loizidou; Brian R Davidson

doi:10.1002/14651858.CD006575.pub3

. 2013 Aug 27;2013(8):CD006575. doi: 10.1002/14651858.CD006575.pub3

Virtual reality training for surgical trainees in laparoscopic surgery

Myura Nagendran ¹, Kurinchi Selvan Gurusamy ^2,^✉, Rajesh Aggarwal ³, Marilena Loizidou ⁴, Brian R Davidson ²

Editor: Cochrane Hepato‐Biliary Group

PMCID: PMC7388923 PMID: 23980026

Abstract

Background

Standard surgical training has traditionally been one of apprenticeship, where the surgical trainee learns to perform surgery under the supervision of a trained surgeon. This is time‐consuming, costly, and of variable effectiveness. Training using a virtual reality simulator is an option to supplement standard training. Virtual reality training improves the technical skills of surgical trainees such as decreased time for suturing and improved accuracy. The clinical impact of virtual reality training is not known.

Objectives

To assess the benefits (increased surgical proficiency and improved patient outcomes) and harms (potentially worse patient outcomes) of supplementary virtual reality training of surgical trainees with limited laparoscopic experience.

Search methods

We searched the Cochrane Central Register of Controlled Trials (CENTRAL) in The Cochrane Library, MEDLINE, EMBASE and Science Citation Index Expanded until July 2012.

Selection criteria

We included all randomised clinical trials comparing virtual reality training versus other forms of training including box‐trainer training, no training, or standard laparoscopic training in surgical trainees with little laparoscopic experience. We also planned to include trials comparing different methods of virtual reality training. We included only trials that assessed the outcomes in people undergoing laparoscopic surgery.

Data collection and analysis

Two authors independently identified trials and collected data. We analysed the data with both the fixed‐effect and the random‐effects models using Review Manager 5 analysis. For each outcome we calculated the mean difference (MD) or standardised mean difference (SMD) with 95% confidence intervals based on intention‐to‐treat analysis.

Main results

We included eight trials covering 109 surgical trainees with limited laparoscopic experience. Of the eight trials, six compared virtual reality versus no supplementary training. One trial compared virtual reality training versus box‐trainer training and versus no supplementary training, and one trial compared virtual reality training versus box‐trainer training. There were no trials that compared different forms of virtual reality training. All the trials were at high risk of bias. Operating time and operative performance were the only outcomes reported in the trials. The remaining outcomes such as mortality, morbidity, quality of life (the primary outcomes of this review) and hospital stay (a secondary outcome) were not reported.

Virtual reality training versus no supplementary training: The operating time was significantly shorter in the virtual reality group than in the no supplementary training group (3 trials; 49 participants; MD ‐11.76 minutes; 95% CI ‐15.23 to ‐8.30). Two trials that could not be included in the meta‐analysis also showed a reduction in operating time (statistically significant in one trial). The numerical values for operating time were not reported in these two trials. The operative performance was significantly better in the virtual reality group than the no supplementary training group using the fixed‐effect model (2 trials; 33 participants; SMD 1.65; 95% CI 0.72 to 2.58). The results became non‐significant when the random‐effects model was used (2 trials; 33 participants; SMD 2.14; 95% CI ‐1.29 to 5.57). One trial could not be included in the meta‐analysis as it did not report the numerical values. The authors stated that the operative performance of virtual reality group was significantly better than the control group.

Virtual reality training versus box‐trainer training: The only trial that reported operating time did not report the numerical values. In this trial, the operating time in the virtual reality group was significantly shorter than in the box‐trainer group. Of the two trials that reported operative performance, only one trial reported the numerical values. The operative performance was significantly better in the virtual reality group than in the box‐trainer group (1 trial; 19 participants; SMD 1.46; 95% CI 0.42 to 2.50). In the other trial that did not report the numerical values, the authors stated that the operative performance in the virtual reality group was significantly better than the box‐trainer group.

Authors' conclusions

Virtual reality training appears to decrease the operating time and improve the operative performance of surgical trainees with limited laparoscopic experience when compared with no training or with box‐trainer training. However, the impact of this decreased operating time and improvement in operative performance on patients and healthcare funders in terms of improved outcomes or decreased costs is not known. Further well‐designed trials at low risk of bias and random errors are necessary. Such trials should assess the impact of virtual reality training on clinical outcomes.

Plain language summary

Virtual reality training for supplementing standard training in surgical trainees with limited prior laparoscopic experience

Standard surgical training has traditionally been one of apprenticeship, where the surgical trainee learns to perform the surgery under the supervision of a trained surgeon. This is costly, time‐consuming, and is of variable effectiveness. Laparoscopic surgery involves the use of instruments using keyhole and is generally considered more difficult than open surgery. Training using a virtual reality simulator (computer simulation) is an option to supplement standard laparoscopic surgical training. Virtual reality training improves the technical skills of surgical trainees. The impact of virtual reality training in supplementing standard laparoscopic surgical training in surgical trainees with limited prior laparoscopic experience on patients is not known. We define surgical trainees with limited prior laparoscopic experience as those who have helped senior surgeons in laparoscopic operations and would need supervision for performing laparoscopic operations on their own. We sought to answer the question of whether virtual reality training is useful for such surgical trainees in terms of improving surgical results and for improving the operative performance by performing a thorough search of the medical literature for randomised clinical trials. Randomised clinical trials are commonly called randomised controlled trials and are the best study design to answer such questions. If conducted well, they provide the most accurate answer.

Two authors searched the medical literature available until July 2012 and obtained the information from the identified trials. The use of two authors to identify studies and obtain information decreases the errors in obtaining the information. We identified and included eight trials covering 109 surgical trainees in this review. The trials compared virtual reality with no supplementary training or with box‐trainer training (physical simulator using a camera to display the inside of the box and instruments). There were no trials that compared different forms of virtual reality training. All the trials were at high risk of bias (defects in study design that can lead to arriving at wrong conclusions with overestimation of benefits and underestimation of harms of virtual reality training or standard training). Operating time and operative performance were the only outcomes reported in the trials. The remaining outcomes such as death, complications, quality of life, and hospital stay after the operation were not reported in any of the trials. Overall virtual reality training appears to decrease the operating time (by about 10 minutes) and improve the operative performance of surgical trainees (difficult to quantify from the available reports) with limited laparoscopic experience when compared with no supplementary training or with box‐trainer training. However, the impact of this decreased operating time and improvement in operative performance on patients or healthcare funders in terms of improved health or decreased costs is not known. Further well‐designed trials are necessary, with less risk of arriving at wrong conclusions because of poor study design or because of chance. Such trials should assess the impact of virtual reality training on patients and healthcare funders.

Summary of findings

for the main comparison.

Virtual reality training for surgical trainees in laparoscopic surgery
Patient or population: surgical trainees training in laparoscopic surgery. Settings: secondary care. Intervention: virtual reality training. Comparison: either no supplementary training or box‐trainer training.
Outcomes	Control	Virtual reality training	No of participants  (studies)	Quality of the evidence  (GRADE)	Comments
Virtual reality training versus no supplementary training
Operating time	The median operating time in the control group was 39 minutes.	The mean operating time in the intervention group was 11.76 minutes lower (8.30 to 15.23 lower).	49 (3)	⊕⊝⊝⊝  very low^1,2	Two other trials also showed reduction in operating time in the virtual reality training group (statistically significant in one trial and not statistically significant in the second trial). The magnitude of the effect was not reported.
Operative performance		The mean performance score in the intervention group was 1.65 standard deviations higher (0.72 to 2.58 higher).	33 (2)	⊕⊝⊝⊝  very low^1,2,3	One other trial also showed statistically significant improvement in operating performance. The magnitude of the effect was not reported.
Virtual reality training versus box trainer training
Operative performance		The mean performance score in the intervention group was 1.46 standard deviations higher (0.42 to 2.5 higher).	19  (1)	⊕⊝⊝⊝  very low^1,2	One other trial also showed statistically significant improvement in operating performance. The magnitude of the effect was not reported.
GRADE Working Group grades of evidence  High quality: Further research is very unlikely to change our confidence in the estimate of effect.  Moderate quality: Further research is likely to have an important impact on our confidence in the estimate of effect and may change the estimate.  Low quality: Further research is very likely to have an important impact on our confidence in the estimate of effect and is likely to change the estimate.  Very low quality: We are very uncertain about the estimate.

Open in a new tab

¹The trial(s) was (were) of low risk of bias.  ²There were too few trials to assess publication bias.  ³There was significant heterogeneity.

Background

Description of the condition

Standard surgical training has traditionally been one of apprenticeship, where the surgical trainee learns to perform surgery under the supervision of a trained surgeon. Different procedures have different learning curves (Herrell 2005; Tekkis 2005a; Tekkis 2005b). Surgeons experienced in one procedure may not be experienced in another and results improve with experience in an individual procedure (Herrell 2005; Tekkis 2005a; Tekkis 2005b).

An increasing number of surgical procedures are being done laparoscopically (by abdominal keyhole surgery). This includes laparoscopic cholecystectomy (removal of gallbladder), laparoscopic anti‐reflux procedures (surgery for heartburn), laparoscopic hysterectomy (removal of uterus), and laparoscopic nephrectomy (removal of kidney) (Ghezzi 2006; Keus 2006; Salminen 2007; Venkatesh 2007).

Description of the intervention

The different methods of laparoscopic surgical training include live animal training, human and animal cadaver training, training using box‐trainer (also called video trainer), and virtual reality training (training using computer simulation) (Munz 2004). Box trainers are currently being used for laparoscopic training in various courses run by the Royal College of Surgeons of England and have been shown to be superior to standard surgical training (Scott 2000). Virtual reality training has been reported to improve the learning outcomes in different surgical procedures (Hyltander 2002; Seymour 2002; Watterson 2002; Grantcharov 2004; Munz 2004). It also offers an ethical way of assessing the competency of a surgeon in performing a procedure without risk to the patient (Moorthy 2004).

How the intervention might work

There are other reports that suggest that virtual reality training alone is inferior to traditional training for certain procedures (Gerson 2003). Virtual reality training has been mainly used for development of technical skills and not training in decision‐making skills. As opposed to the limited variability of data available during a flight on which a pilot requires to be trained using a custom‐designed simulator, anatomical variations are common throughout the human body (Heloury 1985; Lamah 1999; Izuishi 2005), and skills acquired on a single computer simulation programme may not be applicable to patients (Gerson 2003).

The price of the simulators can vary depending upon the learning outcome. However, traditional surgical training is not without costs. The operating time increases significantly for junior surgeons compared to senior surgeons (Farnworth 2001; Babineau 2004; Wilkiemeyer 2005; Kauvar 2006). Bridges and Diamond report the average costs of this increased operating time to be about USD 12,000 per year per resident during the period 1993 to 1997 (Bridges 1999). The complication rate is also higher for junior surgeons compared to senior surgeons (Wilkiemeyer 2005; Kauvar 2006). Bridges and Diamond did not include the cost of the complications in their cost analysis. Thus, the additional cost of the virtual reality training system has to be balanced against the cost of increased operating time and complication rates during traditional surgical training.

Why it is important to do this review

There have been previous reviews related to virtual reality training in surgery (Gaba 2004; Carter 2005; Haque 2006; Sutherland 2006). This review is an update of a review by this group (Gurusamy 2009). It was clear from the first review that virtual reality had the potential to supplement standard surgical training.

This updated review includes only trials of surgical trainees with limited prior laparoscopic experience and where the impact on patient outcomes was assessed. Real operations have the added complexity of anatomical variations (Heloury 1985; Lamah 1999; Izuishi 2005), decision‐making skills (Gurusamy 2009), and mental stress (Prabhu 2010). Considering that patient welfare is the main reason for surgical training, it is important to focus on operative performance in the real situation and its impact on patients rather than on improvement in technical skills during virtual reality training.

Objectives

Methods

Criteria for considering studies for this review

Types of studies

We considered only randomised clinical trials, irrespective of language, blinding, or publication status. We excluded quasi‐randomised studies (where the method of allocating participants to a treatment are not strictly random; for eg, date of birth, hospital record number, alternation), cohort studies, and case‐control studies with regards to benefits, but included them for any harms that could be attributed to virtual reality training. We did not anticipate any such harm that could be attributed to virtual reality training or inclusion of any such study.

Types of participants

Surgical trainees with prior limited laparoscopic experience. Although various terms have been used, such trainees are usually called residents. In general, one would expect that these surgical trainees would need supervision for laparoscopic operations. One would also expect that these surgical trainees have at least assisted in one or more laparoscopic procedures as the laparoscopic camera holder before being allowed to operate on patients.

Types of interventions

We included trials comparing virtual reality training versus any other method of training, including traditional training, in‐job training, or training using a box‐trainer. We also included trials comparing one method of virtual reality training versus another method of virtual reality training (eg, comparison of two different systems).

Types of outcome measures

Primary outcomes

Mortality.
Serious adverse events: these are defined as any event that would increase mortality, is life‐threatening, requires inpatient hospitalisation, results in a persistent or significant disability, or any important medical event which might have jeopardised the patient or required intervention to prevent it (ICH‐GCP 1997).
Quality of life.

Secondary outcomes

Length of hospital stay (days).
Operating time (post‐training in minutes).
Global rating score (post‐training and however defined by authors).

We have presented the results of all the outcomes in Table 1.

Search methods for identification of studies

Electronic searches

We searched the Cochrane Central Register of Controlled Trials (CENTRAL) in The Cochrane Library, MEDLINE, EMBASE, and Science Citation Index Expanded (Royle 2003). We also searched the references of the identified trials to identify further relevant trials. We have given the search strategies in Appendix 1 with the time span for the searches.

Searching other resources

We searched the references of the identified trials to find further relevant trials. We also searched the meta‐Register of Controlled Trials (mRCT) (www.controlled‐trials.com/mrct/). The meta‐Register includes the International Standard Randomised Control Trial Number (ISRCTN) Register and National Institutes of Health (NIH) ClinicalTrials.gov Register among others.

Data collection and analysis

We performed the systematic review according to the recommendations of The Cochrane Collaboration (Higgins 2011) and the Cochrane Hepato‐Biliary Group Module (Gluud 2013).

Selection of studies

MN and KG identified the trials for inclusion independently of each other. MN and KG have also listed the excluded trials with the reasons for the exclusion. BRD adjudicated any differences in opinion.

Data extraction and management

MN and KG extracted the following data independently.  1. Year and language of publication.  2. Country.  3. Date and duration of the trial.  4. Inclusion and exclusion criteria.  5. Sample size.  6. Experience of laparoscopic surgery in surgical participants.  7. Name of virtual reality software.  8. Tasks in training in each group.  9. Duration of training in each group.  10. Outcomes (mentioned above).  11. Risk of bias (described below).

We sought any unclear or missing information by contacting the authors of the individual trials. If there was any doubt whether the trials shared the same participants, completely or partially (by identifying common authors and centres), we planned to contact the authors of the trials to clarify whether the trial report had been duplicated. We resolved any differences in opinion through discussion.

Assessment of risk of bias in included studies

We followed the instructions given in the Cochrane Handbook for Systematic Reviews of Intervention (Higgins 2011) and the Cochrane Hepato‐Biliary Group Module (Gluud 2013). According to empirical evidence (Schulz 1995; Moher 1998; Kjaergard 2001; Wood 2008; Lundh 2012; Savovic 2012; Savovic 2012a), the risks of bias of the trials were assessed based on the following bias risk domains.

Sequence generation

Low risk of bias (the method used was either adequate (eg, computer‐generated random numbers, table of random numbers) or unlikely to introduce confounding).
Uncertain risk of bias (there was insufficient information to assess whether the method used was likely to introduce confounding).
High risk of bias (the method used (eg, quasi‐randomised studies) was improper and likely to introduce confounding).

Allocation concealment

Low risk of bias (the method used (eg, central allocation) was unlikely to introduce bias into the final observed effect).
Uncertain risk of bias (there was insufficient information to assess whether the method used was likely to introduce bias into the estimate of effect).
High risk of bias (the method used (eg, open random allocation schedule) was likely to introduce bias into the final observed effect).

Blinding of participants and personnel

In this situation, both the participants and personnel are the same, ie, the surgical trainees operate on the patients. It is impossible to blind the surgical trainees. We therefore consider this domain to be at high risk of bias for all trials.

Blinding of outcome assessors

Since it is not possible to blind the surgical trainee (participant), it is important that outcome assessors are different from the participants and have no knowledge about the group of the participant.

Low risk of bias (blinding was performed adequately, or the outcome measurement was not likely to be influenced by lack of blinding).
Uncertain risk of bias (there is insufficient information to assess whether the type of blinding used was likely to introduce bias into the estimate of effect).
High risk of bias (no blinding or incomplete blinding, and the outcome or the outcome measurement was likely to be influenced by lack of blinding).

Incomplete outcome data

Low risk of bias (the underlying reasons for missingness are unlikely to make treatment effects depart from plausible values, or proper methods have been employed to handle missing data).
Uncertain risk of bias (there was insufficient information to assess whether the missing data mechanism in combination with the method used to handle missing data was likely to introduce bias into the estimate of effect).
High risk of bias (the crude estimate of effects (eg, complete case estimate) was clearly biased due to the underlying reasons for missingness, and the methods used to handle missing data are unsatisfactory).

Selective outcome reporting

Low risk of bias (the trial protocol is available and all of the trial's pre‐specified outcomes that are of interest in the review had been reported or similar; if the trial protocol was not available, mortality and morbidity were reported).
Uncertain risk of bias (there was insufficient information to assess whether the magnitude and direction of the observed effect was related to selective outcome reporting).
High risk of bias (not all of the trial's pre‐specified primary outcomes had been reported or similar).

Vested interest bias

Low risk of bias (the trial was not performed or supported by any parties that might have conflicting interest, eg, virtual reality trainer manufacturer).
Uncertain risk of bias (any conflicts of interest of the trialist or trial funder was not clear).
High risk of bias (the trial was performed or supported by any parties that might have conflicting interest, eg, virtual reality trainer manufacturer).

Since all the trials have high risk of bias due to lack of blinding of participants/personnel, all trials were classified as being at high risk of bias.

Measures of treatment effect

For binary outcomes, we planned to calculate the risk ratio (RR) with 95% confidence interval (CI). Risk ratio calculations do not include trials in which no events occurred in either group, whereas risk difference calculations do. We planned to report the risk difference if the results using this association measure were different from risk ratio. For continuous outcomes, we calculated the mean difference (MD) with 95% CI for outcomes such as operating time and the standardised mean difference (SMD) with 95% CI for quality of life and operative performance (where different scales might be used).

Unit of analysis issues

The unit of analysis was the individual surgical trainee who performed the individual operations and underwent virtual reality training (or the equivalent control).

Dealing with missing data

We sought any unclear or missing information by contacting the authors of the individual trials.

We performed an intention‐to‐treat analysis (Newell 1992) whenever possible (ie, including all participants originally randomised). We planned to impute data for binary outcomes using various scenarios such as good outcome analysis, bad outcome analysis, 'best‐case' scenario, and 'worst‐case' scenario (Gurusamy 2009a; Gluud 2013).

For continuous outcomes, we used available‐case analysis. We imputed the standard deviation from P values according to the instructions given in the Cochrane Handbook for Systematic Reviews of Intervention (Higgins 2011), and used the median for the meta‐analysis when the mean was not available. If it was not possible to calculate the standard deviation from the P value or the confidence intervals, we imputed the standard deviation as the highest standard deviation in the other trials included under that outcome, fully recognising that this form of imputation will decrease the weight of the study for calculation of mean differences and bias the effect estimate to no effect in case of standardised mean difference (Higgins 2011).

Assessment of heterogeneity

We explored heterogeneity by Chi² test with significance set at P value 0.10, and measured the quantity of heterogeneity by I² (Higgins 2002). We also used overlapping of confidence intervals in the forest plot to determine heterogeneity.

Assessment of reporting biases

We planned to use visual asymmetry in a funnel plot to explore reporting bias if 10 or more trials were identified (Egger 1997; Macaskill 2001). We also planned to perform linear regression approach described by Egger 1997 to determine the funnel plot asymmetry.

Data synthesis

We performed the meta‐analyses using the software package Review Manager 5 (RevMan 2012) and following the recommendations of The Cochrane Collaboration (Higgins 2011) and the Cochrane Hepato‐Biliary Group Module (Gluud 2013). We used both a random‐effects model (DerSimonian 1986) and a fixed‐effect model (DeMets 1987) for the meta‐analyses. In case of discrepancy between the two models we have reported both results; otherwise we have reported the results of the fixed‐effect model. We summarised the evidence in the summary of findings table using GRADEpro (http://ims.cochrane.org/revman/other‐resources/gradepro).

Trial sequential analysis

We planned to use trial sequential analysis to control for random errors due to sparse data and repetitive testing of the accumulating data for the primary outcomes (CTU 2011; Higgins 2011; Thorlund 2011). We planned to add the trials according to the year of publication, and if more than one trial was published in a year, add the trials in alphabetical order according to the last name of the first author. We planned to construct the trial sequential monitoring boundaries on the basis of the diversity‐adjusted required information size (Brok 2008; Wetterslev 2008; Brok 2009; Thorlund 2009, Wetterslev 2009; Thorlund 2010).

We planned to apply trial sequential analysis (CTU 2011; Thorlund 2011) using a diversity adjusted required sample size calculated from an alpha error of 0.05, a beta error of 0.20, a control group proportion obtained from the results of the meta‐analysis, and a relative risk reduction of 20% for binary outcomes with two or more trials to determine whether more trials are necessary on this topic (if the trial sequential monitoring boundary and the required information size is reached or the futility zone is crossed, then more trials may be unnecessary) (Brok 2008; Wetterslev 2008; Brok 2009; Thorlund 2009, Wetterslev 2009; Thorlund 2010). For quality of life, the required sample size was calculated from an alpha error of 0.05, a beta error of 0.20, the variance estimated from the meta‐analysis results of low risk of bias trials, and a minimal clinically relevant difference of 0.25.

Subgroup analysis and investigation of heterogeneity

We planned to perform the following subgroup analyses.

Trials at a high risk of bias compared to trials at a low risk of bias in all domains other than blinding of participants.
Different methods of virtual reality training.
Different levels of prior laparoscopic experience.
Different types of operations.

Sensitivity analysis

We planned to perform a sensitivity analysis by imputing data for binary outcomes using various scenarios such as good outcome analysis, bad outcome analysis, 'best‐case' scenario, and 'worst‐case' scenario (Gurusamy 2009a; Gluud 2013). We performed a sensitivity analysis by excluding the trials in which the mean and the standard deviation were imputed.

Results

Description of studies

Results of the search

We identified a total of 908 references through electronic searches of The Cochrane Central Register of Controlled Trials (CENTRAL) in The Cochrane Library (n = 107), MEDLINE (n = 283), EMBASE (n = 255), Science Citation Index Expanded (n = 246), and randomised controlled trials registers (n = 17). We excluded 416 duplicates and 454 clearly irrelevant references through screening titles and reading abstracts. We retrieved 35 full‐text articles for further assessment. Three were references of ongoing trials (Aggarwal 2010; Gala 2011; Farley 2012), which we have detailed in Characteristics of ongoing studies. No additional studies were identified through scanning reference lists of the identified randomised trials. We excluded 25 references (23 studies) for the reasons listed under the table 'Characteristics of excluded studies'. One reference is awaiting classification (Neary 2008). Although we made attempts to contact the author, we did not receive any reply. In total, nine references for eight completed randomised clinical trials met the inclusion criteria. This is summarised in the study flow diagram (Figure 1).

Included studies

Of the eight trials, six compared virtual reality versus no supplementary training (Seymour 2002; Grantcharov 2004; McClusky 2004; Ahlberg 2007; Larsen 2009; Hogle 2009) and one trial compared virtual reality versus box‐trainer training (Hamilton 2002). One trial compared virtual reality training versus box‐trainer training, and versus no supplementary training (Sendag 2009). There were no trials that compared different forms of virtual reality training. A total of 109 participants were included in this review. Where reported, the average age of trainees in the studies ranged from 32 to 37 years and the proportion of women ranged from 31% to 92%. The details of the trials such as inclusion and exclusion criteria, details of the intervention and control (including the training regimen), and the outcomes measured are shown in the table Characteristics of included studies.

Excluded studies

None of the excluded studies met the inclusion criteria. The reasons for exclusion are shown in the table Characteristics of excluded studies.

Risk of bias in included studies

All the trials were at high risk of bias. The risk of bias in the included trials is summarised in the 'Risk of bias' graph (Figure 2) and 'Risk of bias' summary (Figure 3).

Risk of bias summary: review authors' judgements about each risk of bias item for each included study.

Risk of bias graph: review authors' judgements about each risk of bias item presented as percentages across all included studies.

Effects of interventions

See: Table 1

The findings are summarised in Table 1.

Virtual reality training versus no supplementary training

Operating time and operative performance were the only outcomes reported in the trials. The remaining outcomes were not reported. Trial sequential analysis therefore could not be performed.

Operating time

Three trials could be included in the meta‐analysis (Grantcharov 2004; McClusky 2004; Larsen 2009). The meta‐analysis showed that the operating time was significantly shorter in the virtual reality group than in the no supplementary training group (mean difference (MD) ‐11.76 minutes; 95% confidence interval (CI) ‐15.23 to ‐8.30) (Analysis 1.1). There was no significant heterogeneity (I² = 0; Chi² test for heterogeneity P value = 0.75). There was no change in the results by using the fixed‐effect or the random‐effects model. The standard deviation was imputed in one trial (McClusky 2004). Exclusion of this trial did not alter the results (MD ‐11.82 minutes; 95% CI ‐15.31 to ‐8.33) (Analysis 1.2).

1.1 — Comparison 1 Virtual reality training versus no training, Outcome 1 Operating time.

1.2 — Comparison 1 Virtual reality training versus no training, Outcome 2 Operating time (sensitivity analysis).

Two trials could not be included in the meta‐analysis (Ahlberg 2007; Sendag 2009). In one trial, the operating time was 58% shorter in the virtual reality group than in the control group. However, this difference did not reach statistical significance (P = 0.0586) (Ahlberg 2007). In the other trial, the operating time was significantly shorted in the virtual reality group than in the control group (P < 0.003) (Sendag 2009). This trial did not report the magnitude of the difference.

Operative performance

Two trials could be included for the meta‐analysis (Hogle 2009; Larsen 2009). The meta‐analysis showed that the operative performance was significantly better in the virtual reality group than in the no supplementary training group using the fixed‐effect model (standardised mean difference (SMD) 1.65; 95% CI 0.72 to 2.58) (Analysis 1.3). However, there was significant heterogeneity in this meta‐analysis (I² = 92%; Chi² test for heterogeneity P value < 0.00001). The results became non‐significant when the random‐effects model was used (SMD 2.14; 95% CI ‐1.29 to 5.57). In one trial, the overall operative performance was obtained by adding the individual scores (Hogle 2009). This involved the assumption that the distribution was parametric. A sensitivity analysis excluding this trial showed that the operative performance was significantly better in the virtual reality group than in the control group (SMD 3.93; 95% CI 2.36 to 5.51) (Analysis 1.4).

1.3 — Comparison 1 Virtual reality training versus no training, Outcome 3 Operative performance.

1.4 — Comparison 1 Virtual reality training versus no training, Outcome 4 Operative performance (sensitivity analysis).

One trial could not be included in the meta‐analysis as it did not report the values (Sendag 2009). The authors stated that the operative performance was significantly better in the virtual reality group than in the control group (P < 0.003) (Sendag 2009).

Virtual reality training versus box‐trainer training

Operating time and operative performance were the only outcomes reported in the trials. The remaining outcomes were not reported. Trial sequential analysis therefore could not be performed.

Operating time

Only one trial reported this outcome and this trial did not report the magnitude of the difference (Sendag 2009). In this trial, the operating time in the virtual reality group was significantly shorter than in the box‐trainer group (P < 0.004) (Sendag 2009).

Operative performance

Two trials reported this outcome (Hamilton 2002; Sendag 2009). However, only one trial reported the magnitude of the difference. The operative performance was significantly better in the virtual reality group than in the box‐trainer group (SMD 1.46; 95% CI 0.42 to 2.50) (Analysis 2.1). Since this was the only trial that reported the magnitude of the difference, the heterogeneity could not be assessed. The issue of fixed‐effect model versus random‐effects model does not arise for the same reason. The trial reported the mean and the standard deviation and so sensitivity analysis was not performed.

2.1 — Comparison 2 Virtual reality training versus box‐trainer training, Outcome 1 Operative performance.

In Sendag 2009, the magnitude of the difference in the operative performance between the groups was not reported. However, the authors stated that the operative performance in the virtual reality group was significantly better than the box‐trainer group (P < 0.004).

Subgroup analysis

We did not perform any subgroup analysis because of the few trials available in each category.

Funnel plot

We did not assess the reporting bias because of the few trials included in the review.

Discussion

Summary of main results

Laparoscopic surgery is different from open surgery because of: increased need for hand‐eye‐co‐ordination to perform tasks looking at a screen to compensate for not being able to operate under direct vision; increased need for manual dexterity to compensate for the use of long instruments, which can amplify any error in movement; fulcrum effect of the body wall, ie, when the surgeon moves his hand to the patient's right, the operating end of the instrument moves to the patient's left on the monitor (Gallagher 1999); the need for handling tissues carefully (to compensate for the lack of sensation of touch using hands); and the lack of three‐dimensional images. Virtual reality training is one of the many methods of laparoscopic surgical training and is currently aimed at improving psychomotor skills (Gallagher 1999).

An increasing number of procedures are being performed laparoscopically. With the decreasing time to train surgeons because of European Working Time Directive (Chikwe 2004) and modernising medical careers (MMC) initiative by the Department of Health (Payne 2005), training structured to improve surgical skills in the least time with maximum efficiency is necessary. This is applicable to surgical trainees with no prior experience in laparoscopic surgery and in those who have started their laparoscopic career but have not achieved proficiency. Because of the shortened working hours, the trainees may be exposed to fewer surgical procedures and hence may lack experience. Thus, it is necessary to develop generic skills, such as suturing or cutting and also procedure‐specific skills, such as cannulating the common bile duct. In the previous version of this review, we demonstrated that virtual reality training improved generic skills such as suturing or cutting (Gurusamy 2009). In this update, we have focused on the effect of virtual reality training on operative performance.

Eight trials with 109 surgical trainees were included in this review. The training regimens included training in basic tasks such as cutting, suturing, transfer of objects, or diathermy in five trials (Hamilton 2002; Seymour 2002; Grantcharov 2004; McClusky 2004; Hogle 2009) whereas they included dissection in anatomical models in addition to basic tasks in two trials (Ahlberg 2007; Larsen 2009). The details of training were not available in one trial (Sendag 2009).

Assessing the evidence, it appears that virtual reality training decreases the operating time compared with no supplementary training. However, the difference is approximately 10 minutes per procedure. Although a formal subgroup analysis was not performed, the decrease in operating time appears to be more pronounced in the trial in which the trainees were trained on anatomical models (in addition to basic tasks) (Larsen 2009) compared to the trials in which the surgical trainees were trained only in basic tasks (Analysis 1.1). However, it should be noted that only three trials reported this outcome (Grantcharov 2004; McClusky 2004; Larsen 2009), and we cannot be certain that the difference observed in the magnitude of the effect was due to the difference in the training regimen. Irrespective of the reason for the difference in the magnitude of the effect, the difference in operating time is unlikely to benefit patients in a major way. Whether this difference will decrease the costs by increasing the number of procedures performed in the theatre list depends upon the type of procedures and the duration of the theatre list. Virtual reality training improves the operative performance compared with no supplementary training. As in the case of operating time, the magnitude of difference is greater in the trial in which the trainees were trained on anatomical models (in addition to basic tasks) compared to the trials in which the surgical trainees were trained only in basic tasks (Analysis 1.3). Again, we cannot be certain that the difference observed in the magnitude of the effect was due to the difference in the training regimen as this outcome was reported by two trials only (Hogle 2009; Larsen 2009). There is considerable uncertainty as to the magnitude of the improvement and what this improvement means to the patient and to the healthcare funder. One would generally equate better operative performance with better patient outcomes. However, there is currently no evidence to demonstrate the correlation between better operative performance and better patient outcomes. The likely reason for this is the sensitivity of the issue. Another reason may be that the performance can change (either improve because of more meticulous surgery or decrease because of the stress of the assessment) when a formal assessment is made, which makes the issue quite a difficult one to prove. However, if we agree to the common logical notion that patient outcomes are likely to be better if the operative performance improves, virtual reality training may improve patient outcomes by improving operative performance. While the longevity of this difference in operative performance between virtual reality‐trained surgeons and those with no supplementary training is an unknown quantity, as the surgeons with no supplementary training may catch up with their virtual reality‐trained counterparts as they gain more surgical experience, the difference during the learning curve in surgical training may benefit patients.

The version of the software used was reported in only one trial (Ahlberg 2007). It was therefore not possible to determine whether the magnitude of effect was greater with later versions of the software compared to the earlier versions. Because of the limited number of trials, it was not possible to perform a subgroup analysis based on the levels of experience of the surgical trainees, so it was not possible to determine whether the magnitude of effect was greater in surgical trainees with higher number of procedures performed under supervision.

Virtual reality training also appears to decrease the operating time and improve the operative performance when compared with box‐trainer training. Again the magnitude and the impact of these differences between virtual reality training and box‐trainer training is not known. In the United Kingdom, training is largely based on box‐trainers (rather than virtual reality trainers) in addition to the standard laparoscopic training model of apprenticeships. A survey of satisfaction of the trainees in the virtual reality group and box‐trainer group (Madan 2005) found that the majority of the trainees preferred box‐trainers to virtual reality trainer and a significant number felt that the virtual reality training model was not realistic. This study did not have a haptic feedback interface. Haptic feedback is a tactile feedback technology which mimics the sense of touch by applying forces or vibrations to the user. None of the trials included in this review employed a haptic feedback facility. In the next few years, haptic feedback is likely to form an integral part of the virtual reality simulator and it is likely that the trainee satisfaction will increase with the better simulation. However, the degree of fidelity or realism does not alter the effectiveness in training (Grober 2004). This might explain the reason for the effectiveness of virtual reality training in spite of being a low‐fidelity model. However, improving the fidelity may increase trainee satisfaction and the enthusiasm to learn on virtual reality models.

Some potential advantages of virtual reality over box‐trainer trainer include:  1. Two‐handed tasks need to be followed closely using a second person for training. In virtual reality trainers which follow the instrument tips, there is no need for the second person. In box‐trainers with a fixed video camera, a distance has to be chosen so that the task can be viewed closely. The introduction of instruments cannot be followed. This violates the rule of keeping the business end of the instrument under vision always, which is particularly important in those who are beginning their laparoscopic career.  2. One of the other major problems with box‐trainer training is the 'trainer' time. An expert is necessary for evaluation and feedback in box trainer training. In virtual reality training, the computer evaluates every movement of the trainee and provides feedback after completion of the task (eg, reports the number of movements, distance moved by each hand, traces the path of movements, etc). These can even be used for monitoring the improvement in skills. Thus, the virtual reality software can act as a 'virtual tutor' and a regular training session every week is feasible. However, this advantage of a virtual reality trainer over a box‐trainer has been questioned by some since it is not easy, even for experts, to distinguish reliably that a task was completed without problems (Greco 2010). So if experts cannot come to an agreement about successful completion of a task, it is not possible to program this into a virtual reality trainer in order to determine whether the task was completed without problems.

The potential advantages of box‐trainer training over virtual reality training include:  1. Cheaper cost of the model which enables training multiple trainees simultaneously in short training courses.  2. Better realism (use of real tissue and presence of haptic feedback) compared to currently evaluated virtual reality models (Madan 2005).

The recent hybrid simulators with camera trackers to follow the instruments (Botden 2007) combine some of the advantages of virtual reality training and box‐trainer training. Further research is needed into whether such hybrid simulators are better than virtual reality trainers.

Recent advances in virtual reality technology has made it possible to import images into virtual reality software from external sources (Jaselskis 2013). It is possible to reconstruct the three‐dimensional images if the x, y, z co‐ordinates and colour information of each pixel is available (Cyberware 2013). In the near future, it might be possible to import these three‐dimensional images into the virtual reality software. Once these images are imported into the virtual reality software, it should be possible to manipulate the images. That would mean that it is possible to train the surgical trainees in three‐dimensional reconstructions of actual patients rather than train in their component skills only. Training on numerous such models with anatomic variations can also help with the improvement of decision‐making skills and procedure‐specific skills.

Overall completeness and applicability of evidence

The results of this review are applicable only to surgical trainees with limited laparoscopic experience and confine to the types of virtual reality training used in the trials. The evidence only shows that the operating time is decreased and the operative performance is increased by virtual reality training when compared with no supplementary training or with box‐trainer training. There is currently no evidence that virtual reality training improved patient outcomes.

Quality of the evidence

All the trials were at high risk of bias. While blinding of outcome assessors was performed in three of the four trials that reported this outcome (Hamilton 2002; Hogle 2009; Larsen 2009), the lack of blinding of the participants can result in bias. However, very few of the trials were able to provide data for our meta‐analyses, which were based on few participants. Accordingly, we cannot exclude random errors. Overall the quality of evidence is very low as indicated in Table 1. Nevertheless, this is the best evidence that is currently available.

Potential biases in the review process

Although study selection and data collection were performed in a non‐blinded manner, the potential for bias and errors is largely reduced by the use of two independent data extractors. We were unable to assess publication bias by funnel plot but went through the trial registers to identify the trials. Thus there is unlikely to be publication bias but there appears to be evidence of reporting bias since many of the trials did not report the common outcomes that are likely to be measured during the conduct of the trial. The inclusion of data from such trials may result in a change in conclusions. We imputed the standard deviation when it was not available from the studies. A sensitivity analysis demonstrated that the impact of such imputation was low. The alternative was to exclude the information from the trial which would have made the interpretation of data even more difficult.

Agreements and disagreements with other studies or reviews

The previous version of this review (Gurusamy 2009) found that virtual reality training could supplement the standard laparoscopic surgical training model of apprenticeship, and was at least as effective as box trainer training in supplementing standard laparoscopic training. Most reviews on this topic including the ones mentioned in the Background section arrive at similar conclusions.

Authors' conclusions

Implications for practice.

Implications for research.

Further well‐designed trials at low risk of bias and random errors are necessary. Such trials should also assess the impact of virtual reality training on clinical outcomes.  The conduct and reporting of trials using the SPIRIT (www.spirit‐statement.org) and the CONSORT statements (www.consort‐statement.org) is likely to result in better information from the trials.

What's new

Date	Event	Description
13 February 2013	Amended	Author list: Myura Nagendran, Kurinchi Selvan Gurusamy, Rajesh Aggarwal, Marilena Loizidou, Brian R Davidson
5 February 2013	New citation required and conclusions have changed	We revised the eligibility criteria and updated the methods according to the Cochrane Handbook for Systematic Reviews of Interventions 5.2, re‐analysed the data, and revised conclusions. The previous conclusions were "Virtual reality training can supplement standard laparoscopic surgical training of apprenticeship and is at least as effective as video trainer training in supplementing standard laparoscopic training. Further research of better methodological quality and more patient‐relevant outcomes are needed". The conclusions in the current version are "Virtual reality training appears to decrease the operating time and improve the operative performance of surgical trainees with limited laparoscopic experience when compared with no training or with box‐trainer training. However, the impact of this decreased operating time and improvement in operative performance on patients and healthcare funders in terms of improved outcomes or decreased costs is not known. Further well‐designed trials of low risk of bias and random errors are necessary. Such trials should assess the impact of virtual reality training on clinical outcomes".
21 July 2012	New search has been performed	We updated the searches and identified three new trials (Hogle 2009; Larsen 2009; Sendag 2009).

Open in a new tab

Acknowledgements

To the Cochrane Hepato‐Biliary Group for the support that they have provided.

To L Palanivelu, who identified studies for inclusion and independently extracted data from some studies in the first published review version.    Peer Reviewers: Elisa Greco, Canada; Sonja Buzink, The Netherlands; Tim Horeman, The Netherlands.  Contact Editor: Christian Gluud, Denmark.

This project was funded by the National Institute for Health Research.  Disclaimer of the Department of Health: 'The views and opinions expressed in the review are those of the authors and do not necessarily reflect those of the National Institute for Health Research (NIHR), National Health Services (NHS), or the Department of Health'.

Appendices

Appendix 1. Search strategies for identification of studies

Database	Time span	Search strategy
The Cochrane Central Register of Controlled  Trials (CENTRAL) in The Cochrane Library	Issue 3, 2012	#1 MeSH descriptor Therapy, Computer‐Assisted explode all trees in MeSH products  #2 MeSH descriptor Surgery, Computer‐Assisted explode all trees in MeSH products  #3 MeSH descriptor Computer‐Assisted Instruction explode all trees in MeSH products  #4 virtual realit* OR simulat* in All Fields in all products  #5 (#1 OR #2 OR #3 OR #4)  #6 train* in All Fields in all products  #7 MeSH descriptor Laparoscopy explode all trees in MeSH products  #8 laparoscop* OR coelioscop* OR celioscop* OR peritoneoscop* in All Fields in all products  #9 (#7 OR #8)  #10 (#5 AND #6 AND #9)
MEDLINE	1946 to July 2012	("Therapy, Computer‐Assisted"[MeSH] OR "Surgery, Computer‐Assisted"[MeSH] OR "Computer‐Assisted Instruction"[MeSH] OR virtual realit* OR simulat) AND train AND ("laparoscopy"[MeSH] OR laparoscop* OR coelioscop* OR celioscop* OR peritoneoscop*) AND ((randomized controlled trial [pt] OR controlled clinical trial [pt] OR randomized [tiab] OR placebo [tiab] OR drug therapy [sh] OR randomly [tiab] OR trial [tiab] OR groups [tiab]) NOT (animals [mh] NOT humans [mh]))
EMBASE	1974 to July 2012	#1 exp virtual reality/ or exp computer simulation/ or exp computer assisted surgery/  #2 (virtual realit* or simulat).af.  #3 #1 OR #2  #4 exp medical education/ or exp surgical training/ or exp training/ or exp postgraduate education/  #5 (train or training or trainer).af.  #6 #4 OR #5  #7 exp laparoscopic surgery/ or exp laparoscopy/ #8 (laparoscop* or coelioscop* or celioscop* or peritoneoscop).af.  #9 #7 or #8 #10 #3 AND #6 AND #9  #11 exp crossover‐procedure/ or exp double‐blind procedure/ or exp randomized controlled trial/ or single‐blind procedure/  #12 (random or factorial* or crossover* or cross over* or cross‐over* or placebo* or double* blind* or single* blind* or assign* or allocat* or #13 #11 or #12 #14 #10 and #13
Science Citation Index Expanded	1900 to July 2012	#1 TS=(virtual realit* OR simulat)  #2 TS=(train)  #3 TS=(lapa1roscop* OR coelioscop* OR celioscop* OR peritoneoscop)  #4 TS=(random OR rct* OR crossover OR masked OR blind* OR placebo* OR meta‐analysis OR systematic review* OR meta‐analys*)  #5 #4 AND #3 AND #2 AND #1

Open in a new tab

Data and analyses

Comparison 1. Virtual reality training versus no training.

Outcome or subgroup title	No. of studies	No. of participants	Statistical method	Effect size
1 Operating time	3	49	Mean Difference (IV, Fixed, 95% CI)	‐11.76 [‐15.23, ‐8.30]
2 Operating time (sensitivity analysis)	2	37	Mean Difference (IV, Fixed, 95% CI)	‐11.82 [‐15.31, ‐8.33]
3 Operative performance	2	33	Std. Mean Difference (IV, Fixed, 95% CI)	1.65 [0.72, 2.58]
4 Operative performance (sensitivity analysis)	1		Std. Mean Difference (IV, Fixed, 95% CI)	Subtotals only

Open in a new tab

Comparison 2. Virtual reality training versus box‐trainer training.

Outcome or subgroup title	No. of studies	No. of participants	Statistical method	Effect size
1 Operative performance	1		Std. Mean Difference (IV, Fixed, 95% CI)	Subtotals only

Open in a new tab

Characteristics of studies

Characteristics of included studies [ordered by study ID]

Ahlberg 2007.

Methods	Randomised clinical trial.
Participants	Country: Sweden.  Number randomised: 13.  Postrandomisation drop‐outs: not stated.  Revised sample size: 13.  Average age: 32 years.  Women: 7 (53.8%). Inclusion criteria:  1. Surgical residents from postgraduate year 1 or 2.  2. Experience in assisting with laparoscopic procedures. Exclusion criteria:  1. Previous experience of performing laparoscopic cholecystectomy.
Interventions	Participants were randomly assigned to two groups.  Group 1: virtual reality training (n = 7).  Group 2: no supplementary training (n = 6). Details of virtual reality training: 1. LapSim virtual reality simulator (version 2.0).  2. Basic skills (camera navigation, co‐ordination, clip application, lifting and grasping, cutting, and suturing) and dissection programmes (different anatomic variations of the hepatoduodenal ligament) with no haptic feedback.  3. The participants practised under supervision and received feedback given by the simulator as well as oral feedback given by the supervisor after each completed task until they showed proficiency on each of the 6 examination tasks at least twice.
Outcomes	The outcome reported was operating time.
Notes	Assessment: The participants performed 10 laparoscopic cholecystectomies each after the period of training and the outcomes of 10th surgery were considered except for 2 where the outcomes of the 5th surgery were considered. The operating time was 58% longer in the control group than the virtual reality group (P = 0.0586). We attempted to contact authors in October 2012. No replies were received.
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	Comment: This information was not available.
Allocation concealment (selection bias)	Unclear risk	Quote: "...sealed‐envelope method…." Comment: Further details were not available.
Blinding of participants and personnel (performance bias)   All outcomes	High risk	Comment: It is impossible to blind the participants to the groups.
Blinding of outcome assessment (detection bias)   All outcomes	Unclear risk	Quote: "...Video assessments were performed by 2 observers...blinded concerning the subjects’ training status…" Comment: Although the video recording of the procedures was assessed by blinded observers, there is no information on whether any of the outcomes of interest for this review were assessed by observers blinded to the group of the participants.
Incomplete outcome data (attrition bias)   All outcomes	Unclear risk	Comment: This information was not available.
Selective reporting (reporting bias)	High risk	Comment: Some important outcomes which will generally be assessed were not reported.
Source of funding bias	Unclear risk	Comment: This information was not available.

Open in a new tab

Grantcharov 2004.

Methods	Randomised clinical trial.
Participants	Country: Denmark.  Number randomised: 20.  Postrandomisation drop‐outs: 4 (20%).  Revised sample size: 16.  Average age: 37 years.  Women: 6 (37.5%). Inclusion criteria:  1. Surgeons with limited experience in laparoscopic surgery (< 8 cholecystectomies).
Interventions	Participants were randomly assigned to two groups.  Group 1: virtual reality training (n = 8).  Group 2: no supplementary training (n = 8). Details of virtual reality training: 1. MIST‐virtual reality (Minimally Invasive Surgical Trainer – Virtual Reality).  2. Ten repetitions of all six tasks of progressive complexity and designed to simulate the techniques used during laparoscopic cholecystectomy.
Outcomes	The outcome reported was operating time.
Notes	Assessment: The participants performed a laparoscopic cholecystectomy each after the period of training. Only part of the procedure was assessed, starting from the point at which clips were applied to the cystic artery and cystic duct, and ﬁnishing with dissection of the gallbladder from the liver bed. Two surgical trainees from each group were not included for analysis because there was a fault in the video‐recording which was used for assessment. We attempted to contact authors in October 2012. No replies were received.
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	Comment: This information was not available.
Allocation concealment (selection bias)	Unclear risk	Quote: "using sealed envelopes" Comment: Further details were not available.
Blinding of participants and personnel (performance bias)   All outcomes	High risk	Comment: It is impossible to blind the participants to the groups.
Blinding of outcome assessment (detection bias)   All outcomes	Unclear risk	Quote: "The reviewers were blinded to the training status of the trainees" Comment: Although the video recording of the procedures was assessed by blinded observers, there is no information on whether operating time was measured by observers blinded to the group of the participants.
Incomplete outcome data (attrition bias)   All outcomes	High risk	Comment: There were postrandomisation drop‐outs.
Selective reporting (reporting bias)	High risk	Comment: Some important outcomes which will generally be assessed were not reported.
Source of funding bias	Unclear risk	Quote: "This work was supported by Sygekassernes Helsefond, Copenhagen, Denmark." Comment: Unable to confirm whether funding source linked to private sector or charitable funds.

Open in a new tab

Hamilton 2002.

Methods	Randomised clinical trial.
Participants	Country: United States of America.  Number randomised: 19.  Postrandomisation drop‐outs: 0 (0%).  Revised sample size: 19.  Average age: not stated.  Women: not stated. Inclusion criteria:  1. Second‐year general surgical residents (the study authors had included first‐year and second‐year surgical residents but the evaluation of laparoscopic cholecystectomies was performed in only the second‐year surgical residents).
Interventions	Participants were randomly assigned to two groups.  Group 1: virtual reality training (n = 10).  Group 2: box‐trainer training (n = 9). Details of virtual reality training: 1. Minimally Invasive surgical Trainer ‐ virtual reality trainer.  2. All 6 tasks (grasping, grasping and transfer, hand co‐ordination, replacing the laparoscopic instrument, diathermy ‐ one hand, two hands) ‐ 2 repetitions with each hand (except 3rd task which involved both hands).  3. Ten 30‐minute sessions in 2 weeks. Details of box‐trainer training: 1. Video trainer.  2. Five tasks (suture foam, bean drop, triangle transfer, rope drill, checkerboard).  3. Ten 30‐minute sessions in 2 weeks.
Outcomes	The outcome reported was operating time.
Notes	Assessment: The participants performed a laparoscopic cholecystectomy each after the period of training. We attempted to contact authors in October 2012. Authors replied in October 2012.
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	Quote: "Randomization was performed using a random digit assignment method" Comment: Further details were not available.
Allocation concealment (selection bias)	Low risk	Quote: "Randomization was performed using a random digit assignment method" Communication from author: "placed in sealed envelopes which were opened and said which group randomized to".
Blinding of participants and personnel (performance bias)   All outcomes	High risk	Comment: It is impossible to blind the participants to the groups.
Blinding of outcome assessment (detection bias)   All outcomes	Low risk	Quote: "Faculty evaluators were blinded to the resident's given training modality."
Incomplete outcome data (attrition bias)   All outcomes	Low risk	Comment: There were no postrandomisation drop‐outs.
Selective reporting (reporting bias)	High risk	Comment: Some important outcomes which will generally be assessed were not reported.
Source of funding bias	High risk	Quote: "Funding was provided by the Southwestern Center for Minimally Invasive Surgery as supported in part by an educational grant from United States Surgical Corporation, a division of Tyco Healthcare Group." Comment: The trial was funded by a party with a vested interest in the results.

Open in a new tab

Hogle 2009.

Methods	Randomised clinical trial.
Participants	Country: United States of America.  Number randomised: 13.  Postrandomisation drop‐outs: 1 (7.7%).  Revised sample size: 12.  Average age: not stated  Women: not stated Inclusion criteria:  1. Surgical residents in their first year of surgical experience.
Interventions	Participants were randomly assigned to two groups.  Group 1: virtual reality training (n = 6).  Group 2: no supplementary training (n = 6). Details of virtual reality training: 1. LapSim virtual reality trainer (camera navigation, instrument navigation, co‐ordination, grasping, lifting and grasping, cutting, and clip applying).  2. The training curriculum was fully completed when level 3 was passed for each module.
Outcomes	The outcome reported was operative performance.
Notes	Assessment: Global Operative Assessment of Laparoscopic Skills (GOALS) during laparoscopic cholecystectomy. We attempted to contact authors in October 2012. Authors replied in October 2012.
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	Communication from author: "Random sequence was generated using a random number generator found in the back of a statistics text book. Odd numbers were decided a priori to be assigned to the no‐training group. Even numbers were assigned to training group."
Allocation concealment (selection bias)	Low risk	Communication from author: "Randomization envelopes were produced. The outside of each envelope was labelled with the name of the study, the name of the Principal Investigator and the envelope Number. A card on the inside of each envelope stated “Training Group” or “No Training Group”. Envelopes were assigned sequentially. Study participants were asked not to discuss their study group designation with anyone else including other study participants, surgeons, residents, fellows or technical/study support staff. Only the subject and the study coordinator knew the randomization assignment."
Blinding of participants and personnel (performance bias)   All outcomes	High risk	Comment: It is impossible to blind the participants to the groups.
Blinding of outcome assessment (detection bias)   All outcomes	Low risk	Quote: "The supervising attending surgeon evaluated their performance using GOALS. The video tapes then were used for subsequent blinded evaluation and scoring with GOALS" Communication from author: "Only the subject and the study coordinator knew the randomization assignment."
Incomplete outcome data (attrition bias)   All outcomes	High risk	Comment: There were postrandomisation drop‐outs.
Selective reporting (reporting bias)	High risk	Comment: Some important outcomes which will generally be assessed were not reported.
Source of funding bias	Low risk	Communication from author: "All three studies were non‐funded. Internal resources were used and no monetary compensation was given to investigators or participants."

Open in a new tab

Larsen 2009.

Methods	Randomised clinical trial.
Participants	Country: Denmark.  Number randomised: 24.  Postrandomisation drop‐outs: 3 (12.5%).  Revised sample size: 21.  Average age: 33 years.  Women: 22 (91.7%). Inclusion criteria:  1. Trainees in gynaecological specialty training years 1 and 2 with no experience of advanced laparoscopy (defined as all laparoscopic procedures involving co‐ordination of more than 1 instrument).
Interventions	Participants were randomly assigned to two groups.  Group 1: virtual reality training (n = 11).  Group 2: no supplementary training (n = 10). Details of virtual reality training: 1. LapSim Gyn 3.0.1.  2. Training in the 2 basic skills of “lifting and grasping” and “cutting” and one procedure‐specific task in which the trainee had to carry out a complete right‐sided salpingectomy while preserving the ovary.  3. The training in basic skills was done once in each training cycle of 45 ‐ 60 minutes and the salpingectomy repeated continually during the remainder of the cycle.  4. The simulator provided the trainees with instant feedback on time, path length and angular path of the instruments’ movements, bleeding, cutting of uncoagulated arteries, and use of diathermy on non‐_target tissue.  5. The training sessions were repeated until the expert criterion level was reached in 2 consecutive and independent simulations.
Outcomes	The outcomes reported were operating time and operative performance.
Notes	Assessment: Surgical performance during elective laparoscopic salphingenctomy.
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	Quote: "The Clinical Trial Unit at Copenhagen University independently randomised the trainees by computer to intervention or control groups."
Allocation concealment (selection bias)	Low risk	Quote: "The randomisation procedure was concealed and achieved by using the trainees’ unique personal identification number".
Blinding of participants and personnel (performance bias)   All outcomes	High risk	Quote: "it was not possible to blind the trainees to their allocated group, but all involved departments, supervisors, and staff in the operating theatres were blinded to the trainee’s group". Comment: It is impossible to blind the participants to the groups.
Blinding of outcome assessment (detection bias)   All outcomes	Low risk	Quote: "All involved departments, supervisors, and staff in the operating theatres were blinded to the trainee’s group, and the assessors of outcome were blinded to both the trainee and their allocated group."
Incomplete outcome data (attrition bias)   All outcomes	High risk	Comment: There were postrandomisation drop‐outs.
Selective reporting (reporting bias)	Low risk	Comment: All important outcomes were reported.
Source of funding bias	Low risk	Quote: "This project was supported by Copenhagen University Rigshospitalet Hospital. Trygfondet supplied various materials including computer hardware. Det Calssenske Fidecommis’ Jubilaeumsfond provided travel expenses. Aase and Ejner Danielsens foundation provided software maintenance and updates, DVD recorders, and a TV monitor. The Danish Society for the Protection of Laboratory Animals provided computer hardware and software. All phases of the present work including design and conduct of the study; collection, management, analysis, and interpretation of the data; and preparation, review, and approval of the final manuscript were done independent of the funders." Comment: Funded by various organisations which do not appear to have a vested interest in the results of the trial

Open in a new tab

McClusky 2004.

Methods	Randomised clinical trial.
Participants	Country: United States of America.  Number randomised: 12.  Postrandomisation drop‐outs: not stated.  Revised sample size: 12.  Average age: not stated.  Women: not stated. Inclusion criteria:  1. Surgical residents (postgraduate years 1 and 2).
Interventions	Participants were randomly assigned to two groups.  Group 1: virtual reality training (n = 6).  Group 2: no supplementary training (n = 6). Details of virtual reality training: 1. Minimally Invasive Surgical Trainer ‐ Virtual Reality trainer (MIST‐VR).  2. Expert established performance criterion levels on the manipulation diathermy task of the MIST‐VR.
Outcomes	The outcome reported was operating time.
Notes	Assessment: The participants performed a laparoscopic cholecystectomy each after the period of training. We attempted to contact authors in October 2012. No replies were received.
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	Comment: This information was not available.
Allocation concealment (selection bias)	Unclear risk	Comment: This information was not available.
Blinding of participants and personnel (performance bias)   All outcomes	High risk	Comment: It is impossible to blind the participants to the groups.
Blinding of outcome assessment (detection bias)   All outcomes	Unclear risk	Quote: "Recordings were assessed by two blinded surgeon investigators" Comment: Although the video recording of the procedures was assessed by blinded observers, there is no information on whether any of the outcomes of interest for this review were assessed by observers blinded to the group of the participants.
Incomplete outcome data (attrition bias)   All outcomes	Unclear risk	Comment: This information was not available.
Selective reporting (reporting bias)	High risk	Comment: Some important outcomes which will generally be assessed were not reported.
Source of funding bias	Unclear risk	Comment: This information was not available.

Open in a new tab

Sendag 2009.

Methods	Randomised clinical trial.
Participants	Country: Turkey.  Number randomised: 24.  Postrandomisation drop‐outs: not stated.  Revised sample size: 24.  Average age: not stated.  Women: not stated. Inclusion criteria:  1. Novice residents with no prior experience of laparoscopy.
Interventions	Participants were randomly assigned to three groups.  Group 1: virtual reality training (n = not stated).  Group 2: box‐trainer training (n = not stated). Group 3: no supplementary training (n = not stated). Details of virtual reality training: 1. LapSim.  2. Training for 3 weeks (60 minutes each week).
Outcomes	The outcomes reported were operating time and operative performance.
Notes	Assessment: The participants performed a laparoscopic bilateral tubal ligation each after the period of training.  The global rating scores were statistically significantly higher and the operating time was statistically lower in the virtual reality group than control group. The operating time was statistically significantly lower in the virtual reality group than the box‐trainer training group. It is not clear from the report whether the global rating score was significantly different between virtual reality group and box‐trainer training group. We attempted to contact authors in October 2012. No replies were received.
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	Comment: This information was not available.
Allocation concealment (selection bias)	Unclear risk	Comment: This information was not available.
Blinding of participants and personnel (performance bias)   All outcomes	High risk	Comment: It is impossible to blind the participants to the groups
Blinding of outcome assessment (detection bias)   All outcomes	Unclear risk	Comment: This information was not available.
Incomplete outcome data (attrition bias)   All outcomes	Unclear risk	Comment: This information was not available.
Selective reporting (reporting bias)	High risk	Comment: Some important outcomes which will generally be assessed were not reported.
Source of funding bias	Unclear risk	Comment: This information was not available.

Open in a new tab

Seymour 2002.

Methods	Randomised clinical trial.
Participants	Country: United States of America.  Number randomised: 16.  Postrandomisation drop‐outs: not stated.  Revised sample size: 16.  Average age: not stated.  Women: 5 (31.3%). Inclusion criteria:  1. Surgical residents in postgraduate years 1 to 4.
Interventions	Participants were randomly assigned to two groups.  Group 1: virtual reality training (n = 8).  Group 2: no supplementary training (n = 8). Details of virtual reality training: 1. Minimally Invasive Surgical Trainer ‐ Virtual Reality (MIST‐VR).  2. Expert established performance criterion levels on the manipulation diathermy task of theMIST‐VR.  3. Training sessions lasted approximately 1 hour.  4. The criteria levels were achieved in 3 ‐ 8 training sessions.
Outcomes	None of the outcomes of interest for this review were reported in this trial.
Notes	Assessment: The participants performed a laparoscopic cholecystectomy each after the period of training. We attempted to contact authors in October 2012. No replies were received.
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	Comment: This information was not available.
Allocation concealment (selection bias)	Unclear risk	Comment: This information was not available.
Blinding of participants and personnel (performance bias)   All outcomes	High risk	Comment: It is impossible to blind the participants to the groups
Blinding of outcome assessment (detection bias)   All outcomes	Unclear risk	Quote: "Each procedural video was viewed without audio by two surgeon‐investigators blinded to operating team members." Comment: Although the video recording of the procedures was assessed by blinded observers, there is no information on whether any of the outcomes of interest for this review were assessed by observers blinded to the group of the participants.
Incomplete outcome data (attrition bias)   All outcomes	Unclear risk	Comment: This information was not available.
Selective reporting (reporting bias)	High risk	Comment: Some important outcomes which will generally be assessed were not reported.
Source of funding bias	Unclear risk	Comment: This information was not available.

Open in a new tab

Characteristics of excluded studies [ordered by study ID]

Study	Reason for exclusion
Banks 2007	Not virtual reality training.
Bensalah 2007	No assessment of operative procedure performed by the 2 groups.
Beyer 2011	Not a randomised clinical trial.
Catalyud 2010	Some of the participants had experience of performing more than 100 laparoscopic cholecystectomies and hence cannot be considered as surgical trainees as far as this procedure is concerned.
Cosman 2007	The trainees were asked to perform only a minor part of the procedure.
Gala 2009	Not virtual reality training.
Gauger 2010	Training included virtual reality training and box‐trainer training.
Gobern 2010	Not virtual reality training.
Grantcharov 2007	Not a randomised clinical trial.
Hamilton 2001	Not virtual reality training.
Howells 2008	Not virtual reality training.
Lee 2012	Training involved a combination of virtual reality trainer and box trainer.
Mohan 2009	Unclear whether this is virtual reality training or box‐trainer training.
Moldovanu 2011	Not assessing the effect of virtual reality training in surgical trainees.
Orejuela 2010	Not virtual reality training.
Orzech 2012	The participants performed only a very small part of the procedure in humans.
Palter 2011	Not virtual reality training.
Palter 2012	Training includes virtual reality training and cadaveric training.
Parent 2010	Not virtual reality training.
Sroka 2010	Not virtual reality training.
Van Sickle 2008	Training includes virtual reality training and box‐trainer training.
Zendejas 2011	Not virtual reality training.
Zorn 2011	Comment about a trial which did not meet the inclusion criteria.

Open in a new tab

Characteristics of studies awaiting assessment [ordered by study ID]

Neary 2008.

Methods	Randomised clinical trial.
Participants	Years 3 ‐ 5 postgraduation registrars, specialist registrars or residents in surgery.
Interventions	Proficiency‐based virtual reality simulation training versus standard training.
Outcomes	Laparoscopic colectomy operation duration and number of errors.
Notes	Authors contacted, awaiting response.

Open in a new tab

Characteristics of ongoing studies [ordered by study ID]

Aggarwal 2010.

Trial name or title	An evaluation of the cost effectiveness of virtual reality surgical simulation to shorten the learning curve for real laparoscopic procedures.
Methods	Randomised clinical trial.
Participants	1. Specialist general surgery trainee doctors, i.e. ST1 through to ST5.  2. Have performed fewer than 50 laparoscopic cases (ie appendicectomy and cholecystectomy) as primary operator.
Interventions	The LapMentor™ virtual reality surgical simulator shall be used for training the intervention group in laparoscopic technical skills, under the guise of an evidence‐based training curriculum.    The training arm will undergo a training curriculum. This will be in 3 phases, ie:  1. Knowledge ‐ a structured knowledge‐based online training and assessment tool (including text, diagrams and video).  2. Technical Skills ‐ a step‐wise, structured and proficiency‐based virtual reality training curriculum (incorporating technical skills, procedural tasks and full procedures).  3. Attitudes ‐ a 1‐day session in the simulated Operating Room to perform 2 complete laparoscopic cases with a full operative team. This will be scheduled in the simulated operating room.    The control arm will not undergo any of the above.    The duration of treatment for the training arm will be approximately 2 weeks. The duration of follow‐up for each subject (ie, both arms of the trial) will be approximately 2 months.
Outcomes	Primary outcomes: 1. Procedure time taken (for intra‐abdominal part of procedure), measured during the operative intervention.  2. Quality of operative procedure by Global Rating Scale, measured within 1 week of the operative intervention Secondary outcomes: 1. Knowledge (multiple‐choice test), measured within2 weeks of recruitment to the study for the control group, and within 2 weeks of completion of the training curriculum for the intervention group.  2. Attitudes (surgical team measurements, ie NOTTS and OTAS), measured during operative intervention.  3. Clinical ‐ death, deep vein thrombosis or pulmonary embolism, re‐intervention (percutaneous, endoscopic or surgical), blood transfusion, unplanned ITU/HDU admission, failure to be discharged within 30 days, bile duct injury within 30 days of intervention.
Starting date	February 2011.
Contact information	Dr Rajesh Aggarwal Department of Surgery and Cancer  10th Floor, QEQM Building  St Mary's Hospital Campus  Imperial College London  Praed Street UK.
Notes

Open in a new tab

Farley 2012.

Trial name or title	Part‐ vs whole‐task mastery simulation training for laparoscopic hernia repair: A randomised trial.
Methods	Randomised clinical trial.
Participants	Only residents who have achieved mastery of the TEP training curriculum (in either arm) will be eligible for Operating Room assessment
Interventions	Group 1: Online video curriculum.  Group 2: Whole task mastery training.  Group 3: Part task mastery training.
Outcomes	Operation duration.
Starting date	June 2012.
Contact information	David R. Farley, Co‐Director of the Mayo Simulation Center, Professor of Surgery, Mayo Clinic.
Notes	May not be virtual reality training.

Open in a new tab

Gala 2011.

Trial name or title	An evaluation of validated laparoscopic skills simulators and the impact on operating room performance.
Methods	Randomised clinical trial.
Participants	Ob/Gyn residents in postgraduate years 1 to 4 from ACGME accredited programmes.
Interventions	Group 1: Laparoscopic simulation training (5 x 30‐minute faculty‐directed sessions at the laparoscopic simulator lab). Group 2: Traditional surgical training.
Outcomes	Operative performance.
Starting date	August 2005.
Contact information	Rajiv B Gala, University of Texas Southwestern Medical Center.
Notes	This may be the same trial as Gala 2009, a trial excluded because the simulator was not a virtual reality training simulator.

Open in a new tab

Ob/Gyn = obstetrics and gynaecological trainees  ACGME = Accreditation Council for Graduate Medical Education  NOTTS = non‐technical skills for surgeons  OTAS = observational teamwork assessment for surgery  TEP = totally extraperitoneal repair

Differences between protocol and review

The outcome 'Composite score' was added and the outcome 'Improvement in task performance' was deleted as the final task performance was considered to be more important.

Differences between first review version and second review version

This version includes only surgical trainees with limited prior laparoscopic experience and where the impact of virtual reality training was assessed in the clinical setting.
The methods have been updated to follow the Cochrane Handbook for Systematic Reviews of Interventions version 5.2 (Higgins 2011).
A search update was performed.

Contributions of authors

M Nagendran identified the studies for inclusion, extracted the data, performed the analysis, and wrote the first draft of the update. K Gurusamy independently identified the studies for inclusion, extracted the data, helped with the analysis and wrote sections of the update. R Aggarwal independently extracted the data for some studies in the first version. He commented critically on the review. L Marilena and BR Davidson critically commented on the review.

Sources of support

Internal sources

None, Not specified.

External sources

None, Not specified.

Declarations of interest

None known.

New search for studies and content updated (conclusions changed)

References

References to studies included in this review

Ahlberg 2007 {published data only}

Ahlberg G, Enochsson L, Gallagher AG, Hedman L, Hogman C, McClusky DA, et al. Proficiency‐based virtual reality training significantly reduces the error rate for residents during their first 10 laparoscopic cholecystectomies. American Journal of Surgery 2007;193(6):797‐804. [DOI] [PubMed] [Google Scholar]

Grantcharov 2004 {published data only}

Grantcharov TP, Kristiansen VB, Bendix J, Bardram L, Rosenberg J, Funch‐Jensen P. Randomized clinical trial of virtual reality simulation for laparoscopic skills training. British Journal of Surgery 2004;91(2):146‐50. [DOI] [PubMed] [Google Scholar]

Hamilton 2002 {published data only}

Hamilton EC, Scott DJ, Fleming JB, Rege RV, Laycock R, Bergen PC, et al. Comparison of video trainer and virtual reality training systems on acquisition of laparoscopic skills. Surgical Endoscopy 2002;16(3):406‐11. [DOI] [PubMed] [Google Scholar]

Hogle 2009 {published data only}

Hogle NJ, Chang L, Strong VE, Welcome AO, Sinaan M, Bailey R, et al. Validation of laparoscopic surgical skills training outside the operating room: a long road. Surgical Endoscopy 2009;23:1476‐82. [DOI] [PubMed] [Google Scholar]

Larsen 2009 {published data only}

Larsen CR, Soerensen JL, Grantcharov TP, Dalsgaard T, Schouenborg L, Ottosen C, et al. Effect of virtual reality training on laparoscopic surgery: randomised controlled trial. BMJ 2009;338:b1802. [DOI] [PMC free article] [PubMed] [Google Scholar]
Larsen CR, Soerensen JL, Grantcharov TP, Dalsgaard T, Schouenborg L, Ottosen C, et al. Impact of virtual reality training in laparoscopic gynaecology. International Journal of Gynaecology and Obstetrics 2009;107(Suppl 2):S237. [Google Scholar]

McClusky 2004 {published data only}

McClusky DA, Gallagher AG, Ritter EM, Lederman AB, Sickle KR, Baghai M, et al. Virtual reality training improves junior residents' operating room performance: results of a prospective, randomized, double‐blinded study of the complete laparoscopic cholecystectomy. Journal of the American College of Surgeons 2004;199(3):S73. [Google Scholar]

Sendag 2009 {published data only}

Sendag F, Akdemir A, Bilgin O, Oztekin K. Virtual reality technology in laparoscopic surgical education. Gynecological Surgery 2009;6(1 Suppl):S83. [Google Scholar]

Seymour 2002 {published data only}

Seymour NE, Gallagher AG, Roman SA, O'Brien MK, Bansal VK, Andersen DK, et al. Virtual reality training improves operating room performance: results of a randomized, double‐blinded study. Annals of Surgery 2002;236(4):458‐64. [DOI] [PMC free article] [PubMed] [Google Scholar]

References to studies excluded from this review

Banks 2007 {published data only}

Banks EH, Chudnoff S, Karmin I, Wang C, Pardanani S. Does a surgical simulator improve resident operative performance of laparoscopic tubal ligation?. American Journal of Obstetrics and Gynecology 2007;197(5):541 e1‐e5. [DOI] [PubMed] [Google Scholar]

Bensalah 2007 {published data only}

Bensalah K, Lucas SM, Zeltser IS, Tuncel A, Jenkins A, Pearle MS, et al. Training on the virtual reality laparoscopic simulator improves laparoscopic performance in a virtual environment: a randomized controlled trial. Journal of Endourology 2007;21:A137‐8. [Google Scholar]
Zeltser IS, Bensalah K, Tuncel A, Lucas S, Jenkins A, Pearle MS, et al. Training on the virtual reality laparoscopic simulator improves performance of an unfamiliar live surgical laparoscopic procedure: a randomized, controlled trial. Journal of Endourology 2007;21:A137. [Google Scholar]

Beyer 2011 {published data only}

Beyer L, Troyer JD, Mancini J, Bladou F, Berdah SV, Karsenty G. Impact of laparoscopy simulator training on the technical skills of future surgeons in the operating room: a prospective study. American Journal of Surgery 2011;202:265‐72. [DOI] [PubMed] [Google Scholar]

Catalyud 2010 {published data only}

Calatayud D, Arora S, Aggarwal R, Kruglikova I, Schulze S, Funch‐Jensen P, et al. Warm‐up in a virtual reality environment improves performance in the operating room. Annals of Surgery 2010;251(6):1181‐5. [DOI] [PubMed] [Google Scholar]

Cosman 2007 {published data only}

Cosman PH, Hugh TJ, Shearer CJ, Merrett ND, Biankin AV, Cartmill JA. Skills acquired on virtual reality laparoscopic simulators transfer into the operating room in a blinded, randomised, controlled trial. Studies in Health Technology and Informatics 2007;125:76‐81. [PubMed] [Google Scholar]

Gala 2009 {published data only}

Gala RB, Orejuela F, Gerten K, Lockrow E, Schaffer J. An evaluation of validated laparoscopic skills simulators and the impact on operating room performance. Journal of Pelvic Medicine and Surgery 2009;15(2):46‐7. [Google Scholar]

Gauger 2010 {published data only}

Gauger PG, Hauge LS, Andreatta PB, Hamstra SJ, Hillard ML, Arble EP, et al. Laparoscopic simulation training with proficiency _targets improves practice and performance of novice surgeons. American Journal of Surgery 2010;199(1):72‐80. [DOI] [PubMed] [Google Scholar]

Gobern 2010 {published data only}

Gobern JM, Oliver KE, Gala RB, Chohan L, Kilpatrick CC, Orejuela FJ, et al. Correlation of fine motor skills testing with laparoscopic surgical skills in obstetrics and gynecology residents. Journal of Minimally Invasive Gynecology 2010;1(6 Suppl):S70‐S71. [Google Scholar]

Grantcharov 2007 {published data only}

Grantcharov TP. Virtual reality simulation in training and assessment of laparoscopic skills. European Clinics in Obstetrics and Gynaecology 2007;21(12):197‐200. [Google Scholar]

Hamilton 2001 {published data only}

Hamilton EC, Scott DJ, Kapoor A, Nwariaku F, Bergen PC, Rege RV, et al. Improving operative performance using a laparoscopic hernia simulator. American Journal of Surgery 2001;182(6):725‐8. [DOI] [PubMed] [Google Scholar]

Howells 2008 {published data only}

Howells NR, Gill HS, Carr AJ, Price AJ, Rees JL. Transferring simulated arthroscopic skills to the operating theatre: a randomised blinded study. Journal of Bone and Joint Surgery. British Volume 2008;90(4):494‐9. [DOI] [PubMed] [Google Scholar]

Lee 2012 {published data only}

Lee J, Kahol K, Kerbl DC, Alipanah N, Hsueh TY, Mucksavage P, et al. Impact of pre‐operative warm‐up exercises on surgical performance in urology. Journal of Endourology 2010;24:A60. [Google Scholar]
Lee JY, Mucksavage P, Kerbl DC, Osann KE, Winfield HN, Kahol K, et al. Laparoscopic warm‐up exercises improve performance of senior‐level trainees during laparoscopic renal surgery. Journal of Endourology 2012;26(5):545‐50. [DOI] [PMC free article] [PubMed] [Google Scholar]

Mohan 2009 {published data only}

Mohan PVR, Chaudhry R. Laparoscopic simulators: Are they useful!. Medical Journal of the Armed Forces of India 2009;65(2):113‐7. [DOI] [PMC free article] [PubMed] [Google Scholar]

Moldovanu 2011 {published data only}

Moldovanu R, Tarcoveanu E, Dimofte G, Lupascu C, Bradea C. Preoperative warm‐up using a virtual reality simulator. Journal of the Society of Laparoendoscopic Surgeons 2011;15(4):533‐8. [DOI] [PMC free article] [PubMed] [Google Scholar]

Orejuela 2010 {published data only}

Orejuela F, Gerten K, Lockrow E, Noel KA, Chohan L, Kilpatrick CC, et al. An evaluation of validated laparoscopic skills simulators and the impact on operating room performance in obstetrics and gynecology residents. Journal of Pelvic Medicine and Surgery 2010;16(2 Suppl):S7. [Google Scholar]

Orzech 2012 {published data only}

Orzech N, Palter VN, Reznick RK, Aggarwal R, Grantcharov TP. A comparison of 2 ex vivo training curricula for advanced laparoscopic skills: a randomized controlled trial. Annals of Surgery 2012;255(5):833‐9. [DOI] [PubMed] [Google Scholar]

Palter 2011 {published data only}

Palter VN, Grantcharov T, Harvey A, Macrae HM. Ex vivo technical skills training transfers to the operating room and enhances cognitive learning: a randomized controlled trial. Annals of Surgery 2011;253(5):886‐9. [DOI] [PubMed] [Google Scholar]

Palter 2012 {published data only}

Palter VN, Grantcharov TP. Development and validation of a comprehensive curriculum to teach an advanced minimally invasive procedure: a randomized controlled trial. Annals of Surgery 2012;256(1):25‐32. [DOI] [PubMed] [Google Scholar]

Parent 2010 {published data only}

Parent RJ, Plerhoples TA, Long EE, Zimmer DM, Teshome M, Mohr CJ, et al. Early, intermediate, and late effects of a surgical skills "boot camp" on an objective structured assessment of technical skills: a randomized controlled study. Journal of American College of Surgeons 2010;210(6):984‐9. [DOI] [PubMed] [Google Scholar]

Sroka 2010 {published data only}

Sroka G, Feldman LS, Vassiliou MC, Kaneva PA, Fayez R, Fried GM. Fundamentals of laparoscopic surgery simulator training to proficiency improves laparoscopic performance in the operating room ‐ a randomized controlled trial. American Journal of Surgery 2010;199(1):115‐20. [DOI] [PubMed] [Google Scholar]

Van Sickle 2008 {published data only}

Sickle KR, Ritter EM, Baghai M, Goldenberg AE, Huang IP, Gallagher AG, et al. Prospective, randomized, double‐blind trial of curriculum‐based training for intracorporeal suturing and knot tying. Journal of American College of Surgeons 2008;207(4):560‐8. [DOI] [PubMed] [Google Scholar]

Zendejas 2011 {published data only}

Zendejas B, Cook DA, Bingener J, Huebner M, Dunn WF, Sarr MG, et al. Simulation‐based mastery learning improves patient outcomes in laparoscopic inguinal hernia repair: a randomized controlled trial. Annals of Surgery 2011;254(3):502‐9. [DOI] [PubMed] [Google Scholar]

Zorn 2011 {published data only}

Zorn KC. Randomized controlled trial of virtual reality and hybrid simulation for robotic surgical training. BJU International 2011;108(10):1657. [DOI] [PubMed] [Google Scholar]

References to studies awaiting assessment

Neary 2008 {published data only}

Neary P. Creating and implementing a proficiency‐based progression virtual reality training programme for higher surgical trainees for laparoscopic assisted sigmoid colectomy. clinicaltrials.gov/show/NCT00752817 (accessed 15th December 2012).

References to ongoing studies

Aggarwal 2010 {published data only}

Aggarwal R. An evaluation of the cost‐effectiveness of virtual reality surgical simulation to shorten the learning curve for real laparoscopic procedures. www.controlled‐trials.com/ISRCTN44012204 (accessed 15th December 2012).

Farley 2012 {published data only}

Farley DR. Part vs whole task mastery simulation training for laparoscopic hernia repair: a randomized trial. clinicaltrials.gov/show/NCT01629485 (accessed 15th December 2012).

Gala 2011 {published data only}

Gala RB. An evaluation of validated laparoscopic skills simulators and the impact on operating room performance. clinicaltrials.gov/show/NCT00555243 (accessed 15th December 2012).

Additional references

Babineau 2004

Babineau TJ, Becker J, Gibbons G, Sentovich S, Hess D, Robertson S, et al. The "cost" of operative training for surgical residents. Archives of Surgical Research 2004;139(4):366‐70. [DOI] [PubMed] [Google Scholar]

Botden 2007

Botden SM, Buzink SN, Schijven MP, Jakimowicz JJ. Augmented versus virtual reality laparoscopic simulation: what is the difference? A comparison of the ProMIS augmented reality laparoscopic simulator versus LapSim virtual reality laparoscopic simulator. World Journal of Surgery 2007;31(4):764‐72. [DOI] [PMC free article] [PubMed] [Google Scholar]

Bridges 1999

Bridges M, Diamond DL. The financial impact of teaching surgical residents in the operating room. American Journal of Surgery 1999;177(1):28‐32. [DOI] [PubMed] [Google Scholar]

Brok 2008

Brok J, Thorlund K, Gluud C, Wetterslev J. Trial sequential analysis reveals insufficient information size and potentially false positive results in many meta‐analyses. Journal of Clinical Epidemiology 2008;61:763‐9. [DOI] [PubMed] [Google Scholar]

Brok 2009

Brok J, Thorlund K, Wetterslev J, Gluud C. Apparently conclusive meta‐analyses may be inconclusive ‐ Trial sequential analysis adjustment of random error risk due to repetitive testing of accumulating data in apparently conclusive neonatal meta‐analyses. International Journal of Epidemiology 2009;38(1):287‐98. [DOI] [PubMed] [Google Scholar]

Carter 2005

Carter FJ, Schijven MP, Aggarwal R, Grantcharov T, Francis NK, Hanna GB, et al. Consensus guidelines for validation of virtual reality surgical simulators. Surgical Endoscopy 2005;19(12):1523‐32. [DOI] [PubMed] [Google Scholar]

Chikwe 2004

Chikwe J, Souza AC, Pepper JR. No time to train the surgeons. BMJ (Clinical Research Ed.) 2004;328(7437):418‐9. [DOI] [PMC free article] [PubMed] [Google Scholar]

CTU 2011

Copenhagen Trial Unit. TSA ‐ Trial Sequential Analysis. http://ctu.dk/tsa/ 2011 (accessed 20 August 2013).

Cyberware 2013

Cyberware. Head & Face Color 3D Scanner. www.cyberware.com/products/scanners/ps.html (accessed 20 August 2013).

DeMets 1987

DeMets DL. Methods for combining randomized clinical trials: strengths and limitations. Statistics in Medicine 1987;6(3):341‐50. [DOI] [PubMed] [Google Scholar]

DerSimonian 1986

DerSimonian R, Laird N. Meta‐analysis in clinical trials. Controlled Clinical Trials 1986;7(3):177‐88. [DOI] [PubMed] [Google Scholar]

Egger 1997

Egger M, Davey Smith G, Schneider M, Minder C. Bias in meta‐analysis detected by a simple, graphical test. BMJ (Clinical Research Ed.) 1997;315(7109):629‐34. [DOI] [PMC free article] [PubMed] [Google Scholar]

Farnworth 2001

Farnworth LR, Lemay DE, Wooldridge T, Mabrey JD, Blaschak MJ, DeCoster TA, et al. A comparison of operative times in arthroscopic ACL reconstruction between orthopaedic faculty and residents: the financial impact of orthopaedic surgical training in the operating room. Iowa Orthopedic Journal 2001;21:31‐5. [PMC free article] [PubMed] [Google Scholar]

Gaba 2004

Gaba DM. The future vision of simulation in health care. Quality and Safety in Health Care 2004;13 Suppl 1:i2‐i10. [DOI] [PMC free article] [PubMed] [Google Scholar]

Gallagher 1999

Gallagher AG, McClure N, McGuigan J, Crothers I, Browning J. Virtual reality training in laparoscopic surgery: a preliminary assessment of minimally invasive surgical trainer virtual reality (MIST VR). Endoscopy 1999;31(4):310‐3. [DOI] [PubMed] [Google Scholar]

Gerson 2003

Gerson LB, Dam J. A prospective randomized trial comparing a virtual reality simulator to bedside teaching for training in sigmoidoscopy. Endoscopy 2003;35(7):569‐75. [DOI] [PubMed] [Google Scholar]

Ghezzi 2006

Ghezzi F, Cromi A, Bergamini V, Uccella S, Beretta P, Franchi M, et al. Laparoscopic‐assisted vaginal hysterectomy versus total laparoscopic hysterectomy for the management of endometrial cancer: a randomized clinical trial. Journal of Minimally Invasive Gynecology 2006;13(2):114‐20. [DOI] [PubMed] [Google Scholar]

Gluud 2013

Gluud C, Nikolova D, Klingenberg SL, Alexakis N, Als‐Nielsen B, Colli A, et al. Cochrane Hepato‐Biliary Group. About The Cochrane Collaboration (Cochrane Review Groups (CRGs)). 2013, Issue 7. Art. No.: LIVER.

Greco 2010

Greco EF, Regehr G, Okrainec A. Identifying and classifying problem areas in laparoscopic skills acquisition: can simulators help?. Academic Medicine 2010;85(10 Suppl):S5‐S8. [DOI] [PubMed] [Google Scholar]

Grober 2004

Grober ED, Hamstra SJ, Wanzel KR, Reznick RK, Matsumoto ED, Sidhu RS, et al. The educational impact of bench model fidelity on the acquisition of technical skill: the use of clinically relevant outcome measures. Annals of Surgery 2004;240(2):374‐81. [DOI] [PMC free article] [PubMed] [Google Scholar]

Gurusamy 2009a

Gurusamy KS, Gluud C, Nikolova D, Davidson BR. Assessment of risk of bias in randomized clinical trials in surgery. British Journal of Surgery 2009;96(4):342‐9. [DOI] [PubMed] [Google Scholar]

Haque 2006

Haque S, Srinivasan S. A meta‐analysis of the training effectiveness of virtual reality surgical simulators. IEEE Transactions on Information Technology in Biomedicine 2006;10(1):51‐8. [DOI] [PubMed] [Google Scholar]

Heloury 1985

Heloury Y, Leborgne J, Rogez JM, Robert R, Lehur PA, Pannier M, et al. Radiological anatomy of the bile ducts based on intraoperative investigation in 250 cases. Anatomia Clinica 1985;7(2):93‐102. [DOI] [PubMed] [Google Scholar]

Herrell 2005

Herrell SD, Smith JA Jr. Robotic‐assisted laparoscopic prostatectomy: what is the learning curve?. Urology 2005;66(5 Suppl):105‐7. [DOI] [PubMed] [Google Scholar]

Higgins 2002

Higgins JP, Thompson SG. Quantifying heterogeneity in a meta‐analysis. Statistics in Medicine 2002;21(11):1539‐58. [DOI] [PubMed] [Google Scholar]

Higgins 2011

Higgins JPT, Green S (editors). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration, 2011. Available from www.cochrane‐handbook.org.

Hyltander 2002

Hyltander A, Liljegren E, Rhodin PH, Lonroth H. The transfer of basic skills learned in a laparoscopic simulator to the operating room. Surgical Endoscopy 2002;16(9):1324‐8. [DOI] [PubMed] [Google Scholar]

ICH‐GCP 1997

International Conference on Harmonisation Expert Working Group. International conference on harmonisation of technical requirements for registration of pharmaceuticals for human use. ICH harmonised tripartite guideline. Guideline for good clinical practice CFR & ICH Guidelines. Vol. 1, PA 19063‐2043, USA: Barnett International/PAREXEL, 1997. [Google Scholar]

Izuishi 2005

Izuishi K, Toyama Y, Nakano S, Goda F, Usuki H, Masaki T, et al. Preoperative assessment of the aberrant bile duct using multislice computed tomography cholangiography. American Journal of Surgery 2005;189(1):53‐5. [DOI] [PubMed] [Google Scholar]

Jaselskis 2013

Jaselskis E, Andrle S, Anderson‐Wilk M, Bernard JE, Bryden M, Vance JM. Virtual reality and laser scanning applications. www.ctre.iastate.edu/pubs/vrls.pdf (accessed 20 August 2013).

Kauvar 2006

Kauvar DS, Braswell A, Brown BD, Harnisch M. Influence of resident and attending surgeon seniority on operative performance in laparoscopic cholecystectomy. Journal of Surgical Research 2006;132(2):159‐63. [DOI] [PubMed] [Google Scholar]

Keus 2006

Keus F, Jong JAF, Gooszen HG, Laarhoven CJHM. Laparoscopic versus open cholecystectomy for patients with symptomatic cholecystolithiasis. Cochrane Database of Systematic Reviews 2006, Issue 4. [DOI: 10.1002/14651858.CD006231] [DOI] [PubMed] [Google Scholar]

Kjaergard 2001

Kjaergard LL, Villumsen J, Gluud C. Reported methodologic quality and discrepancies between large and small randomized trials in meta‐analyses. Annals of Internal Medicine 2001;135(11):982‐9. [DOI] [PubMed] [Google Scholar]

Lamah 1999

Lamah M, Dickson GH. Congenital anatomical abnormalities of the extrahepatic biliary duct: a personal audit. Surgical and Radiologic Anatomy 1999;21(5):325‐7. [DOI] [PubMed] [Google Scholar]

Lundh 2012

Lundh A, Sismondo S, Lexchin J, Busuioc OA, Bero L. Industry sponsorship and research outcome. Cochrane Database of Systematic Reviews 2012, Issue 12. [DOI: 10.1002/14651858.MR000033.pub2] [DOI] [PubMed] [Google Scholar]

Macaskill 2001

Macaskill P, Walter SD, Irwig L. A comparison of methods to detect publication bias in meta‐analysis. Statistics in Medicine 2001;20(4):641‐54. [DOI] [PubMed] [Google Scholar]

Madan 2005

Madan AK, Frantzides CT, Tebbit C, Quiros RM. Participants' opinions of laparoscopic training devices after basic laparoscopic training course. American Journal of Surgery 2005;189(6):758‐61. [DOI] [PubMed] [Google Scholar]

Moher 1998

Moher D, Pham B, Jones A, Cook DJ, Jadad AR, Moher M, et al. Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta‐analyses?. Lancet 1998;352(9128):609‐13. [DOI] [PubMed] [Google Scholar]

Moorthy 2004

Moorthy K, Munz Y, Jiwanji M, Bann S, Chang A, Darzi A. Validity and reliability of a virtual reality upper gastrointestinal simulator and cross validation using structured assessment of individual performance with video playback. Surgical Endoscopy 2004;18(2):328‐33. [DOI] [PubMed] [Google Scholar]

Munz 2004

Munz Y, Kumar BD, Moorthy K, Bann S, Darzi A. Laparoscopic virtual reality and box trainers: is one superior to the other?. Surgical Endoscopy 2004;18(3):485‐94. [DOI] [PubMed] [Google Scholar]

Newell 1992

Newell DJ. Intention‐to‐treat analysis: implications for quantitative and qualitative research. International Journal of Epidemiology 1992;21(5):837‐41. [DOI] [PubMed] [Google Scholar]

Payne 2005

Payne SR, Shaw MB. What impact will shortened training have on urological service delivery?. Annals of the Royal College of Surgeons of England 2005;87(5):373‐8. [DOI] [PMC free article] [PubMed] [Google Scholar]

Prabhu 2010

Prabhu A, Smith W, Yurko Y, Acker C, Stefanidis D. Increased stress levels may explain the incomplete transfer of simulator‐acquired skill to the operating room. Surgery 2010;147(5):640‐5. [DOI] [PubMed] [Google Scholar]

RevMan 2012 [Computer program]

The Nordic Cochrane Centre, The Cochrane Collaboration. Review Manager (RevMan). Version 5.2. Copenhagen: The Nordic Cochrane Centre, The Cochrane Collaboration, 2012.

Royle 2003

Royle P, Milne R. Literature searching for randomized controlled trials used in Cochrane reviews: rapid versus exhaustive searches. International Journal of Technology Assessment in Health Care 2003;19(4):591‐603. [DOI] [PubMed] [Google Scholar]

Salminen 2007

Salminen PT, Hiekkanen HI, Rantala AP, Ovaska JT. Comparison of long‐term outcome of laparoscopic and conventional nissen fundoplication: a prospective randomized study with an 11‐year follow‐up. Annals of Surgery 2007;246(2):201‐6. [DOI] [PMC free article] [PubMed] [Google Scholar]

Savovic 2012

Savovic J, Jones HE, Altman DG, Harris RJ, Jüni P, Pildal J, et al. Influence of reported study design characteristics on intervention effect estimates from randomized, controlled trials. Health Technology Assessment 2012;16(35):1‐82. [DOI] [PubMed] [Google Scholar]

Savovic 2012a

Savovic J, Jones HE, Altman DG, Harris RJ, Jüni P, Pildal J, et al. Influence of reported study design characteristics on intervention effect estimates from randomized, controlled trials. Annals of Internal Medicine 2012;157(6):429‐38. [DOI] [PubMed] [Google Scholar]

Schulz 1995

Schulz KF, Chalmers I, Hayes RJ, Altman DG. Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA 1995;273(5):408‐12. [DOI] [PubMed] [Google Scholar]

Scott 2000

Scott DJ, Bergen PC, Rege RV, Laycock R, Tesfay ST, Valentine RJ, et al. Laparoscopic training on bench models: better and more cost effective than operating room experience?. Journal of the American College of Surgeons 2000;191(3):272‐83. [DOI] [PubMed] [Google Scholar]

Sutherland 2006

Sutherland LM, Middleton PF, Anthony A, Hamdorf J, Cregan P, Scott D, et al. Surgical simulation ‐ A systematic review. Annals of Surgery 2006;243(3):291‐300. [DOI] [PMC free article] [PubMed] [Google Scholar]

Tekkis 2005a

Tekkis PP, Fazio VW, Lavery IC, Remzi FH, Senagore AJ, Wu JS, et al. Evaluation of the learning curve in ileal pouch‐anal anastomosis surgery. Annals of Surgery 2005;241(2):262‐8. [DOI] [PMC free article] [PubMed] [Google Scholar]

Tekkis 2005b

Tekkis PP, Senagore AJ, Delaney CP, Fazio VW. Evaluation of the learning curve in laparoscopic colorectal surgery: comparison of right‐sided and left‐sided resections. Annals of Surgery 2005;242(1):83‐91. [DOI] [PMC free article] [PubMed] [Google Scholar]

Thorlund 2009

Thorlund K, Devereaux PJ, Wetterslev J, Guyatt G, Ioannidis JP, Thabane L, et al. Can trial sequential monitoring boundaries reduce spurious inferences from meta‐analyses. International Journal of Epidemiology 2009;38(1):276‐86. [DOI] [PubMed] [Google Scholar]

Thorlund 2010

Thorlund K, Anema A, Mills E. Interpreting meta‐analysis according to the adequacy of sample size. An example using isoniazid chemoprophylaxis for tuberculosis in purified protein derivative negative HIV‐infected individuals. Clinical Epidemiology 2010;2:57‐66. [DOI] [PMC free article] [PubMed] [Google Scholar]

Thorlund 2011

Thorlund K, Engstrøm J, Wetterslev J, Brok J, Imberger G, Gluud C. User manual for Trial Sequential Analysis (TSA). ctu.dk/tsa/files/tsa_manual.pdf 2011 (accessed 20 August 2013).

Venkatesh 2007

Venkatesh R, Belani JS, Chen C, Sundaram CP, Bhayani SB, Figenshau RS, et al. Prospective randomized comparison of laparoscopic and hand‐assisted laparoscopic radical nephrectomy. Urology 2007;70(5):873‐7. [DOI] [PubMed] [Google Scholar]

Watterson 2002

Watterson JD, Beiko DT, Kuan JK, Denstedt JD. Randomized prospective blinded study validating acquisition of ureteroscopy skills using computer based virtual reality endourological simulator. Journal of Urology 2002;168(5):1928‐32. [DOI] [PubMed] [Google Scholar]

Wetterslev 2008

Wetterslev J, Thorlund K, Brok J, Gluud C. Trial sequential analysis may establish when firm evidence is reached in cumulative meta‐analysis. Journal of Clinical Epidemiology 2008;61(1):64‐75. [DOI] [PubMed] [Google Scholar]

Wetterslev 2009

Wetterslev J, Thorlund K, Brok J, Gluud C. Estimating required information size by quantifying diversity in random‐effects model meta‐analyses. BMC Medical Research Methodology 2009;9:86. [DOI] [PMC free article] [PubMed] [Google Scholar]

Wilkiemeyer 2005

Wilkiemeyer M, Pappas TN, Giobbie‐Hurder A, Itani KM, Jonasson O, Neumayer LA. Does resident post graduate year influence the outcomes of inguinal hernia repair?. Annals of Surgery 2005;241(6):879‐84. [DOI] [PMC free article] [PubMed] [Google Scholar]

Wood 2008

Wood L, Egger M, Gluud LL, Schulz KF, Juni P, Altman DG, et al. Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: meta‐epidemiological study. BMJ (Clinical Research Ed.) 2008;336(7644):601‐5. [PUBMED: 18316340] [DOI] [PMC free article] [PubMed] [Google Scholar]

References to other published versions of this review

Gurusamy 2008

Gurusamy K, Aggarwal R, Palanivelu L, Davidson BR. Systematic review of randomized controlled trials on the effectiveness of virtual reality training for laparoscopic surgery. British Journal of Surgery 2008;95(9):1088‐97. [PUBMED: 18690637] [DOI] [PubMed] [Google Scholar]

Gurusamy 2009

Gurusamy KS, Aggarwal R, Palanivelu L, Davidson BR. Virtual reality training for surgical trainees in laparoscopic surgery. Cochrane Database of Systematic Reviews 2009, Issue 1. [DOI: 10.1002/14651858.CD006575.pub2] [DOI] [PubMed] [Google Scholar]

[CD006575-bib-0001] Ahlberg G, Enochsson L, Gallagher AG, Hedman L, Hogman C, McClusky DA, et al. Proficiency‐based virtual reality training significantly reduces the error rate for residents during their first 10 laparoscopic cholecystectomies. American Journal of Surgery 2007;193(6):797‐804. [DOI] [PubMed] [Google Scholar]

[CD006575-bib-0002] Grantcharov TP, Kristiansen VB, Bendix J, Bardram L, Rosenberg J, Funch‐Jensen P. Randomized clinical trial of virtual reality simulation for laparoscopic skills training. British Journal of Surgery 2004;91(2):146‐50. [DOI] [PubMed] [Google Scholar]

[CD006575-bib-0003] Hamilton EC, Scott DJ, Fleming JB, Rege RV, Laycock R, Bergen PC, et al. Comparison of video trainer and virtual reality training systems on acquisition of laparoscopic skills. Surgical Endoscopy 2002;16(3):406‐11. [DOI] [PubMed] [Google Scholar]

[CD006575-bib-0004] Hogle NJ, Chang L, Strong VE, Welcome AO, Sinaan M, Bailey R, et al. Validation of laparoscopic surgical skills training outside the operating room: a long road. Surgical Endoscopy 2009;23:1476‐82. [DOI] [PubMed] [Google Scholar]

[CD006575-bib-0005] Larsen CR, Soerensen JL, Grantcharov TP, Dalsgaard T, Schouenborg L, Ottosen C, et al. Effect of virtual reality training on laparoscopic surgery: randomised controlled trial. BMJ 2009;338:b1802. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD006575-bib-0006] Larsen CR, Soerensen JL, Grantcharov TP, Dalsgaard T, Schouenborg L, Ottosen C, et al. Impact of virtual reality training in laparoscopic gynaecology. International Journal of Gynaecology and Obstetrics 2009;107(Suppl 2):S237. [Google Scholar]

[CD006575-bib-0007] McClusky DA, Gallagher AG, Ritter EM, Lederman AB, Sickle KR, Baghai M, et al. Virtual reality training improves junior residents' operating room performance: results of a prospective, randomized, double‐blinded study of the complete laparoscopic cholecystectomy. Journal of the American College of Surgeons 2004;199(3):S73. [Google Scholar]

[CD006575-bib-0008] Sendag F, Akdemir A, Bilgin O, Oztekin K. Virtual reality technology in laparoscopic surgical education. Gynecological Surgery 2009;6(1 Suppl):S83. [Google Scholar]

[CD006575-bib-0009] Seymour NE, Gallagher AG, Roman SA, O'Brien MK, Bansal VK, Andersen DK, et al. Virtual reality training improves operating room performance: results of a randomized, double‐blinded study. Annals of Surgery 2002;236(4):458‐64. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD006575-bib-0010] Banks EH, Chudnoff S, Karmin I, Wang C, Pardanani S. Does a surgical simulator improve resident operative performance of laparoscopic tubal ligation?. American Journal of Obstetrics and Gynecology 2007;197(5):541 e1‐e5. [DOI] [PubMed] [Google Scholar]

[CD006575-bib-0011] Bensalah K, Lucas SM, Zeltser IS, Tuncel A, Jenkins A, Pearle MS, et al. Training on the virtual reality laparoscopic simulator improves laparoscopic performance in a virtual environment: a randomized controlled trial. Journal of Endourology 2007;21:A137‐8. [Google Scholar]

[CD006575-bib-0012] Zeltser IS, Bensalah K, Tuncel A, Lucas S, Jenkins A, Pearle MS, et al. Training on the virtual reality laparoscopic simulator improves performance of an unfamiliar live surgical laparoscopic procedure: a randomized, controlled trial. Journal of Endourology 2007;21:A137. [Google Scholar]

[CD006575-bib-0013] Beyer L, Troyer JD, Mancini J, Bladou F, Berdah SV, Karsenty G. Impact of laparoscopy simulator training on the technical skills of future surgeons in the operating room: a prospective study. American Journal of Surgery 2011;202:265‐72. [DOI] [PubMed] [Google Scholar]

[CD006575-bib-0014] Calatayud D, Arora S, Aggarwal R, Kruglikova I, Schulze S, Funch‐Jensen P, et al. Warm‐up in a virtual reality environment improves performance in the operating room. Annals of Surgery 2010;251(6):1181‐5. [DOI] [PubMed] [Google Scholar]

[CD006575-bib-0015] Cosman PH, Hugh TJ, Shearer CJ, Merrett ND, Biankin AV, Cartmill JA. Skills acquired on virtual reality laparoscopic simulators transfer into the operating room in a blinded, randomised, controlled trial. Studies in Health Technology and Informatics 2007;125:76‐81. [PubMed] [Google Scholar]

[CD006575-bib-0016] Gala RB, Orejuela F, Gerten K, Lockrow E, Schaffer J. An evaluation of validated laparoscopic skills simulators and the impact on operating room performance. Journal of Pelvic Medicine and Surgery 2009;15(2):46‐7. [Google Scholar]

[CD006575-bib-0017] Gauger PG, Hauge LS, Andreatta PB, Hamstra SJ, Hillard ML, Arble EP, et al. Laparoscopic simulation training with proficiency _targets improves practice and performance of novice surgeons. American Journal of Surgery 2010;199(1):72‐80. [DOI] [PubMed] [Google Scholar]

[CD006575-bib-0018] Gobern JM, Oliver KE, Gala RB, Chohan L, Kilpatrick CC, Orejuela FJ, et al. Correlation of fine motor skills testing with laparoscopic surgical skills in obstetrics and gynecology residents. Journal of Minimally Invasive Gynecology 2010;1(6 Suppl):S70‐S71. [Google Scholar]

[CD006575-bib-0019] Grantcharov TP. Virtual reality simulation in training and assessment of laparoscopic skills. European Clinics in Obstetrics and Gynaecology 2007;21(12):197‐200. [Google Scholar]

[CD006575-bib-0020] Hamilton EC, Scott DJ, Kapoor A, Nwariaku F, Bergen PC, Rege RV, et al. Improving operative performance using a laparoscopic hernia simulator. American Journal of Surgery 2001;182(6):725‐8. [DOI] [PubMed] [Google Scholar]

[CD006575-bib-0021] Howells NR, Gill HS, Carr AJ, Price AJ, Rees JL. Transferring simulated arthroscopic skills to the operating theatre: a randomised blinded study. Journal of Bone and Joint Surgery. British Volume 2008;90(4):494‐9. [DOI] [PubMed] [Google Scholar]

[CD006575-bib-0022] Lee J, Kahol K, Kerbl DC, Alipanah N, Hsueh TY, Mucksavage P, et al. Impact of pre‐operative warm‐up exercises on surgical performance in urology. Journal of Endourology 2010;24:A60. [Google Scholar]

[CD006575-bib-0023] Lee JY, Mucksavage P, Kerbl DC, Osann KE, Winfield HN, Kahol K, et al. Laparoscopic warm‐up exercises improve performance of senior‐level trainees during laparoscopic renal surgery. Journal of Endourology 2012;26(5):545‐50. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD006575-bib-0024] Mohan PVR, Chaudhry R. Laparoscopic simulators: Are they useful!. Medical Journal of the Armed Forces of India 2009;65(2):113‐7. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD006575-bib-0025] Moldovanu R, Tarcoveanu E, Dimofte G, Lupascu C, Bradea C. Preoperative warm‐up using a virtual reality simulator. Journal of the Society of Laparoendoscopic Surgeons 2011;15(4):533‐8. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD006575-bib-0026] Orejuela F, Gerten K, Lockrow E, Noel KA, Chohan L, Kilpatrick CC, et al. An evaluation of validated laparoscopic skills simulators and the impact on operating room performance in obstetrics and gynecology residents. Journal of Pelvic Medicine and Surgery 2010;16(2 Suppl):S7. [Google Scholar]

[CD006575-bib-0027] Orzech N, Palter VN, Reznick RK, Aggarwal R, Grantcharov TP. A comparison of 2 ex vivo training curricula for advanced laparoscopic skills: a randomized controlled trial. Annals of Surgery 2012;255(5):833‐9. [DOI] [PubMed] [Google Scholar]

[CD006575-bib-0028] Palter VN, Grantcharov T, Harvey A, Macrae HM. Ex vivo technical skills training transfers to the operating room and enhances cognitive learning: a randomized controlled trial. Annals of Surgery 2011;253(5):886‐9. [DOI] [PubMed] [Google Scholar]

[CD006575-bib-0029] Palter VN, Grantcharov TP. Development and validation of a comprehensive curriculum to teach an advanced minimally invasive procedure: a randomized controlled trial. Annals of Surgery 2012;256(1):25‐32. [DOI] [PubMed] [Google Scholar]

[CD006575-bib-0030] Parent RJ, Plerhoples TA, Long EE, Zimmer DM, Teshome M, Mohr CJ, et al. Early, intermediate, and late effects of a surgical skills "boot camp" on an objective structured assessment of technical skills: a randomized controlled study. Journal of American College of Surgeons 2010;210(6):984‐9. [DOI] [PubMed] [Google Scholar]

[CD006575-bib-0031] Sroka G, Feldman LS, Vassiliou MC, Kaneva PA, Fayez R, Fried GM. Fundamentals of laparoscopic surgery simulator training to proficiency improves laparoscopic performance in the operating room ‐ a randomized controlled trial. American Journal of Surgery 2010;199(1):115‐20. [DOI] [PubMed] [Google Scholar]

[CD006575-bib-0032] Sickle KR, Ritter EM, Baghai M, Goldenberg AE, Huang IP, Gallagher AG, et al. Prospective, randomized, double‐blind trial of curriculum‐based training for intracorporeal suturing and knot tying. Journal of American College of Surgeons 2008;207(4):560‐8. [DOI] [PubMed] [Google Scholar]

[CD006575-bib-0033] Zendejas B, Cook DA, Bingener J, Huebner M, Dunn WF, Sarr MG, et al. Simulation‐based mastery learning improves patient outcomes in laparoscopic inguinal hernia repair: a randomized controlled trial. Annals of Surgery 2011;254(3):502‐9. [DOI] [PubMed] [Google Scholar]

[CD006575-bib-0034] Zorn KC. Randomized controlled trial of virtual reality and hybrid simulation for robotic surgical training. BJU International 2011;108(10):1657. [DOI] [PubMed] [Google Scholar]

[CD006575-bib-0035] Neary P. Creating and implementing a proficiency‐based progression virtual reality training programme for higher surgical trainees for laparoscopic assisted sigmoid colectomy. clinicaltrials.gov/show/NCT00752817 (accessed 15th December 2012).

[CD006575-bib-0036] Aggarwal R. An evaluation of the cost‐effectiveness of virtual reality surgical simulation to shorten the learning curve for real laparoscopic procedures. www.controlled‐trials.com/ISRCTN44012204 (accessed 15th December 2012).

[CD006575-bib-0037] Farley DR. Part vs whole task mastery simulation training for laparoscopic hernia repair: a randomized trial. clinicaltrials.gov/show/NCT01629485 (accessed 15th December 2012).

[CD006575-bib-0038] Gala RB. An evaluation of validated laparoscopic skills simulators and the impact on operating room performance. clinicaltrials.gov/show/NCT00555243 (accessed 15th December 2012).

[CD006575-bib-0039] Babineau TJ, Becker J, Gibbons G, Sentovich S, Hess D, Robertson S, et al. The "cost" of operative training for surgical residents. Archives of Surgical Research 2004;139(4):366‐70. [DOI] [PubMed] [Google Scholar]

[CD006575-bib-0040] Botden SM, Buzink SN, Schijven MP, Jakimowicz JJ. Augmented versus virtual reality laparoscopic simulation: what is the difference? A comparison of the ProMIS augmented reality laparoscopic simulator versus LapSim virtual reality laparoscopic simulator. World Journal of Surgery 2007;31(4):764‐72. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD006575-bib-0041] Bridges M, Diamond DL. The financial impact of teaching surgical residents in the operating room. American Journal of Surgery 1999;177(1):28‐32. [DOI] [PubMed] [Google Scholar]

[CD006575-bib-0042] Brok J, Thorlund K, Gluud C, Wetterslev J. Trial sequential analysis reveals insufficient information size and potentially false positive results in many meta‐analyses. Journal of Clinical Epidemiology 2008;61:763‐9. [DOI] [PubMed] [Google Scholar]

[CD006575-bib-0043] Brok J, Thorlund K, Wetterslev J, Gluud C. Apparently conclusive meta‐analyses may be inconclusive ‐ Trial sequential analysis adjustment of random error risk due to repetitive testing of accumulating data in apparently conclusive neonatal meta‐analyses. International Journal of Epidemiology 2009;38(1):287‐98. [DOI] [PubMed] [Google Scholar]

[CD006575-bib-0044] Carter FJ, Schijven MP, Aggarwal R, Grantcharov T, Francis NK, Hanna GB, et al. Consensus guidelines for validation of virtual reality surgical simulators. Surgical Endoscopy 2005;19(12):1523‐32. [DOI] [PubMed] [Google Scholar]

[CD006575-bib-0045] Chikwe J, Souza AC, Pepper JR. No time to train the surgeons. BMJ (Clinical Research Ed.) 2004;328(7437):418‐9. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD006575-bib-0046] Copenhagen Trial Unit. TSA ‐ Trial Sequential Analysis. http://ctu.dk/tsa/ 2011 (accessed 20 August 2013).

[CD006575-bib-0047] Cyberware. Head & Face Color 3D Scanner. www.cyberware.com/products/scanners/ps.html (accessed 20 August 2013).

[CD006575-bib-0048] DeMets DL. Methods for combining randomized clinical trials: strengths and limitations. Statistics in Medicine 1987;6(3):341‐50. [DOI] [PubMed] [Google Scholar]

[CD006575-bib-0049] DerSimonian R, Laird N. Meta‐analysis in clinical trials. Controlled Clinical Trials 1986;7(3):177‐88. [DOI] [PubMed] [Google Scholar]

[CD006575-bib-0050] Egger M, Davey Smith G, Schneider M, Minder C. Bias in meta‐analysis detected by a simple, graphical test. BMJ (Clinical Research Ed.) 1997;315(7109):629‐34. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD006575-bib-0051] Farnworth LR, Lemay DE, Wooldridge T, Mabrey JD, Blaschak MJ, DeCoster TA, et al. A comparison of operative times in arthroscopic ACL reconstruction between orthopaedic faculty and residents: the financial impact of orthopaedic surgical training in the operating room. Iowa Orthopedic Journal 2001;21:31‐5. [PMC free article] [PubMed] [Google Scholar]

[CD006575-bib-0052] Gaba DM. The future vision of simulation in health care. Quality and Safety in Health Care 2004;13 Suppl 1:i2‐i10. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD006575-bib-0053] Gallagher AG, McClure N, McGuigan J, Crothers I, Browning J. Virtual reality training in laparoscopic surgery: a preliminary assessment of minimally invasive surgical trainer virtual reality (MIST VR). Endoscopy 1999;31(4):310‐3. [DOI] [PubMed] [Google Scholar]

[CD006575-bib-0054] Gerson LB, Dam J. A prospective randomized trial comparing a virtual reality simulator to bedside teaching for training in sigmoidoscopy. Endoscopy 2003;35(7):569‐75. [DOI] [PubMed] [Google Scholar]

[CD006575-bib-0055] Ghezzi F, Cromi A, Bergamini V, Uccella S, Beretta P, Franchi M, et al. Laparoscopic‐assisted vaginal hysterectomy versus total laparoscopic hysterectomy for the management of endometrial cancer: a randomized clinical trial. Journal of Minimally Invasive Gynecology 2006;13(2):114‐20. [DOI] [PubMed] [Google Scholar]

[CD006575-bib-0056] Gluud C, Nikolova D, Klingenberg SL, Alexakis N, Als‐Nielsen B, Colli A, et al. Cochrane Hepato‐Biliary Group. About The Cochrane Collaboration (Cochrane Review Groups (CRGs)). 2013, Issue 7. Art. No.: LIVER.

[CD006575-bib-0057] Greco EF, Regehr G, Okrainec A. Identifying and classifying problem areas in laparoscopic skills acquisition: can simulators help?. Academic Medicine 2010;85(10 Suppl):S5‐S8. [DOI] [PubMed] [Google Scholar]

[CD006575-bib-0058] Grober ED, Hamstra SJ, Wanzel KR, Reznick RK, Matsumoto ED, Sidhu RS, et al. The educational impact of bench model fidelity on the acquisition of technical skill: the use of clinically relevant outcome measures. Annals of Surgery 2004;240(2):374‐81. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD006575-bib-0059] Gurusamy KS, Gluud C, Nikolova D, Davidson BR. Assessment of risk of bias in randomized clinical trials in surgery. British Journal of Surgery 2009;96(4):342‐9. [DOI] [PubMed] [Google Scholar]

[CD006575-bib-0060] Haque S, Srinivasan S. A meta‐analysis of the training effectiveness of virtual reality surgical simulators. IEEE Transactions on Information Technology in Biomedicine 2006;10(1):51‐8. [DOI] [PubMed] [Google Scholar]

[CD006575-bib-0061] Heloury Y, Leborgne J, Rogez JM, Robert R, Lehur PA, Pannier M, et al. Radiological anatomy of the bile ducts based on intraoperative investigation in 250 cases. Anatomia Clinica 1985;7(2):93‐102. [DOI] [PubMed] [Google Scholar]

[CD006575-bib-0062] Herrell SD, Smith JA Jr. Robotic‐assisted laparoscopic prostatectomy: what is the learning curve?. Urology 2005;66(5 Suppl):105‐7. [DOI] [PubMed] [Google Scholar]

[CD006575-bib-0063] Higgins JP, Thompson SG. Quantifying heterogeneity in a meta‐analysis. Statistics in Medicine 2002;21(11):1539‐58. [DOI] [PubMed] [Google Scholar]

[CD006575-bib-0064] Higgins JPT, Green S (editors). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration, 2011. Available from www.cochrane‐handbook.org.

[CD006575-bib-0065] Hyltander A, Liljegren E, Rhodin PH, Lonroth H. The transfer of basic skills learned in a laparoscopic simulator to the operating room. Surgical Endoscopy 2002;16(9):1324‐8. [DOI] [PubMed] [Google Scholar]

[CD006575-bib-0066] International Conference on Harmonisation Expert Working Group. International conference on harmonisation of technical requirements for registration of pharmaceuticals for human use. ICH harmonised tripartite guideline. Guideline for good clinical practice CFR & ICH Guidelines. Vol. 1, PA 19063‐2043, USA: Barnett International/PAREXEL, 1997. [Google Scholar]

[CD006575-bib-0067] Izuishi K, Toyama Y, Nakano S, Goda F, Usuki H, Masaki T, et al. Preoperative assessment of the aberrant bile duct using multislice computed tomography cholangiography. American Journal of Surgery 2005;189(1):53‐5. [DOI] [PubMed] [Google Scholar]

[CD006575-bib-0068] Jaselskis E, Andrle S, Anderson‐Wilk M, Bernard JE, Bryden M, Vance JM. Virtual reality and laser scanning applications. www.ctre.iastate.edu/pubs/vrls.pdf (accessed 20 August 2013).

[CD006575-bib-0069] Kauvar DS, Braswell A, Brown BD, Harnisch M. Influence of resident and attending surgeon seniority on operative performance in laparoscopic cholecystectomy. Journal of Surgical Research 2006;132(2):159‐63. [DOI] [PubMed] [Google Scholar]

[CD006575-bib-0070] Keus F, Jong JAF, Gooszen HG, Laarhoven CJHM. Laparoscopic versus open cholecystectomy for patients with symptomatic cholecystolithiasis. Cochrane Database of Systematic Reviews 2006, Issue 4. [DOI: 10.1002/14651858.CD006231] [DOI] [PubMed] [Google Scholar]

[CD006575-bib-0071] Kjaergard LL, Villumsen J, Gluud C. Reported methodologic quality and discrepancies between large and small randomized trials in meta‐analyses. Annals of Internal Medicine 2001;135(11):982‐9. [DOI] [PubMed] [Google Scholar]

[CD006575-bib-0072] Lamah M, Dickson GH. Congenital anatomical abnormalities of the extrahepatic biliary duct: a personal audit. Surgical and Radiologic Anatomy 1999;21(5):325‐7. [DOI] [PubMed] [Google Scholar]

[CD006575-bib-0073] Lundh A, Sismondo S, Lexchin J, Busuioc OA, Bero L. Industry sponsorship and research outcome. Cochrane Database of Systematic Reviews 2012, Issue 12. [DOI: 10.1002/14651858.MR000033.pub2] [DOI] [PubMed] [Google Scholar]

[CD006575-bib-0074] Macaskill P, Walter SD, Irwig L. A comparison of methods to detect publication bias in meta‐analysis. Statistics in Medicine 2001;20(4):641‐54. [DOI] [PubMed] [Google Scholar]

[CD006575-bib-0075] Madan AK, Frantzides CT, Tebbit C, Quiros RM. Participants' opinions of laparoscopic training devices after basic laparoscopic training course. American Journal of Surgery 2005;189(6):758‐61. [DOI] [PubMed] [Google Scholar]

[CD006575-bib-0076] Moher D, Pham B, Jones A, Cook DJ, Jadad AR, Moher M, et al. Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta‐analyses?. Lancet 1998;352(9128):609‐13. [DOI] [PubMed] [Google Scholar]

[CD006575-bib-0077] Moorthy K, Munz Y, Jiwanji M, Bann S, Chang A, Darzi A. Validity and reliability of a virtual reality upper gastrointestinal simulator and cross validation using structured assessment of individual performance with video playback. Surgical Endoscopy 2004;18(2):328‐33. [DOI] [PubMed] [Google Scholar]

[CD006575-bib-0078] Munz Y, Kumar BD, Moorthy K, Bann S, Darzi A. Laparoscopic virtual reality and box trainers: is one superior to the other?. Surgical Endoscopy 2004;18(3):485‐94. [DOI] [PubMed] [Google Scholar]

[CD006575-bib-0079] Newell DJ. Intention‐to‐treat analysis: implications for quantitative and qualitative research. International Journal of Epidemiology 1992;21(5):837‐41. [DOI] [PubMed] [Google Scholar]

[CD006575-bib-0080] Payne SR, Shaw MB. What impact will shortened training have on urological service delivery?. Annals of the Royal College of Surgeons of England 2005;87(5):373‐8. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD006575-bib-0081] Prabhu A, Smith W, Yurko Y, Acker C, Stefanidis D. Increased stress levels may explain the incomplete transfer of simulator‐acquired skill to the operating room. Surgery 2010;147(5):640‐5. [DOI] [PubMed] [Google Scholar]

[CD006575-bib-0082] The Nordic Cochrane Centre, The Cochrane Collaboration. Review Manager (RevMan). Version 5.2. Copenhagen: The Nordic Cochrane Centre, The Cochrane Collaboration, 2012.

[CD006575-bib-0083] Royle P, Milne R. Literature searching for randomized controlled trials used in Cochrane reviews: rapid versus exhaustive searches. International Journal of Technology Assessment in Health Care 2003;19(4):591‐603. [DOI] [PubMed] [Google Scholar]

[CD006575-bib-0084] Salminen PT, Hiekkanen HI, Rantala AP, Ovaska JT. Comparison of long‐term outcome of laparoscopic and conventional nissen fundoplication: a prospective randomized study with an 11‐year follow‐up. Annals of Surgery 2007;246(2):201‐6. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD006575-bib-0085] Savovic J, Jones HE, Altman DG, Harris RJ, Jüni P, Pildal J, et al. Influence of reported study design characteristics on intervention effect estimates from randomized, controlled trials. Health Technology Assessment 2012;16(35):1‐82. [DOI] [PubMed] [Google Scholar]

[CD006575-bib-0086] Savovic J, Jones HE, Altman DG, Harris RJ, Jüni P, Pildal J, et al. Influence of reported study design characteristics on intervention effect estimates from randomized, controlled trials. Annals of Internal Medicine 2012;157(6):429‐38. [DOI] [PubMed] [Google Scholar]

[CD006575-bib-0087] Schulz KF, Chalmers I, Hayes RJ, Altman DG. Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA 1995;273(5):408‐12. [DOI] [PubMed] [Google Scholar]

[CD006575-bib-0088] Scott DJ, Bergen PC, Rege RV, Laycock R, Tesfay ST, Valentine RJ, et al. Laparoscopic training on bench models: better and more cost effective than operating room experience?. Journal of the American College of Surgeons 2000;191(3):272‐83. [DOI] [PubMed] [Google Scholar]

[CD006575-bib-0089] Sutherland LM, Middleton PF, Anthony A, Hamdorf J, Cregan P, Scott D, et al. Surgical simulation ‐ A systematic review. Annals of Surgery 2006;243(3):291‐300. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD006575-bib-0090] Tekkis PP, Fazio VW, Lavery IC, Remzi FH, Senagore AJ, Wu JS, et al. Evaluation of the learning curve in ileal pouch‐anal anastomosis surgery. Annals of Surgery 2005;241(2):262‐8. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD006575-bib-0091] Tekkis PP, Senagore AJ, Delaney CP, Fazio VW. Evaluation of the learning curve in laparoscopic colorectal surgery: comparison of right‐sided and left‐sided resections. Annals of Surgery 2005;242(1):83‐91. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD006575-bib-0092] Thorlund K, Devereaux PJ, Wetterslev J, Guyatt G, Ioannidis JP, Thabane L, et al. Can trial sequential monitoring boundaries reduce spurious inferences from meta‐analyses. International Journal of Epidemiology 2009;38(1):276‐86. [DOI] [PubMed] [Google Scholar]

[CD006575-bib-0093] Thorlund K, Anema A, Mills E. Interpreting meta‐analysis according to the adequacy of sample size. An example using isoniazid chemoprophylaxis for tuberculosis in purified protein derivative negative HIV‐infected individuals. Clinical Epidemiology 2010;2:57‐66. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD006575-bib-0094] Thorlund K, Engstrøm J, Wetterslev J, Brok J, Imberger G, Gluud C. User manual for Trial Sequential Analysis (TSA). ctu.dk/tsa/files/tsa_manual.pdf 2011 (accessed 20 August 2013).

[CD006575-bib-0095] Venkatesh R, Belani JS, Chen C, Sundaram CP, Bhayani SB, Figenshau RS, et al. Prospective randomized comparison of laparoscopic and hand‐assisted laparoscopic radical nephrectomy. Urology 2007;70(5):873‐7. [DOI] [PubMed] [Google Scholar]

[CD006575-bib-0096] Watterson JD, Beiko DT, Kuan JK, Denstedt JD. Randomized prospective blinded study validating acquisition of ureteroscopy skills using computer based virtual reality endourological simulator. Journal of Urology 2002;168(5):1928‐32. [DOI] [PubMed] [Google Scholar]

[CD006575-bib-0097] Wetterslev J, Thorlund K, Brok J, Gluud C. Trial sequential analysis may establish when firm evidence is reached in cumulative meta‐analysis. Journal of Clinical Epidemiology 2008;61(1):64‐75. [DOI] [PubMed] [Google Scholar]

[CD006575-bib-0098] Wetterslev J, Thorlund K, Brok J, Gluud C. Estimating required information size by quantifying diversity in random‐effects model meta‐analyses. BMC Medical Research Methodology 2009;9:86. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD006575-bib-0099] Wilkiemeyer M, Pappas TN, Giobbie‐Hurder A, Itani KM, Jonasson O, Neumayer LA. Does resident post graduate year influence the outcomes of inguinal hernia repair?. Annals of Surgery 2005;241(6):879‐84. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD006575-bib-0100] Wood L, Egger M, Gluud LL, Schulz KF, Juni P, Altman DG, et al. Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: meta‐epidemiological study. BMJ (Clinical Research Ed.) 2008;336(7644):601‐5. [PUBMED: 18316340] [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD006575-bib-0101] Gurusamy K, Aggarwal R, Palanivelu L, Davidson BR. Systematic review of randomized controlled trials on the effectiveness of virtual reality training for laparoscopic surgery. British Journal of Surgery 2008;95(9):1088‐97. [PUBMED: 18690637] [DOI] [PubMed] [Google Scholar]

[CD006575-bib-0102] Gurusamy KS, Aggarwal R, Palanivelu L, Davidson BR. Virtual reality training for surgical trainees in laparoscopic surgery. Cochrane Database of Systematic Reviews 2009, Issue 1. [DOI: 10.1002/14651858.CD006575.pub2] [DOI] [PubMed] [Google Scholar]

PERMALINK

Virtual reality training for surgical trainees in laparoscopic surgery

Myura Nagendran

Kurinchi Selvan Gurusamy

Rajesh Aggarwal

Marilena Loizidou

Brian R Davidson

Abstract

Background

Objectives

Search methods

Selection criteria

Data collection and analysis

Main results

Authors' conclusions

Plain language summary

Summary of findings

for the main comparison.

Background

Description of the condition

Description of the intervention

How the intervention might work

Why it is important to do this review

Objectives

Methods

Criteria for considering studies for this review

Types of studies

Types of participants

Types of interventions

Types of outcome measures

Primary outcomes

Secondary outcomes

Search methods for identification of studies

Electronic searches

Searching other resources

Data collection and analysis

Selection of studies

Data extraction and management

Assessment of risk of bias in included studies

Sequence generation

Allocation concealment

Blinding of participants and personnel

Blinding of outcome assessors

Incomplete outcome data

Selective outcome reporting

Vested interest bias

Measures of treatment effect

Unit of analysis issues

Dealing with missing data

Assessment of heterogeneity

Assessment of reporting biases

Data synthesis

Trial sequential analysis

Subgroup analysis and investigation of heterogeneity

Sensitivity analysis

Results

Description of studies

Results of the search

1.

Included studies

Excluded studies

Risk of bias in included studies

2.

3.

Effects of interventions

Virtual reality training versus no supplementary training

Operating time

1.1. Analysis.

1.2. Analysis.

Operative performance

1.3. Analysis.

1.4. Analysis.

Virtual reality training versus box‐trainer training

Operating time

Operative performance

2.1. Analysis.

Subgroup analysis

Funnel plot

Discussion

Summary of main results