Systematic review of team performance in minimally invasive abdominal surgery

Background Adverse events in the operating theatre related to non‐technical skills and teamwork are still an issue. The influence of minimally invasive techniques on team performance and subsequent impact on patient safety remains unclear. The aim of this review was to assess the methodology used to objectify and rate team performance in minimally invasive abdominal surgery. Methods A systematic literature search was conducted according to the PRISMA guidelines. Studies on assessment of surgical team performance or non‐technical skills of the surgical team in the setting of minimally invasive abdominal surgery were included. Study aim, methodology, results and conclusion were extracted for qualitative synthesis. Results Sixteen studies involving 677 surgical procedures were included. All studies consisted of observational case series that used heterogeneous methodologies to assess team performance and were of low methodological quality. The most commonly used team performance objectification tools were ‘construct’‐ and ‘incident’‐based tools. Evidence of validity for the assessed outcome was spread widely across objectification tools, ranging from low to high. Diverse and poorly defined outcomes were reported. Conclusion Team demands for minimally invasive approaches to abdominal procedures remain unclear. The current literature consists of studies with heterogeneous methodology and poorly defined outcomes.


Introduction
A substantial contribution to morbidity among surgical patients can be attributed to adverse events occurring in the operating theatre 1 . Increasing evidence shows that a considerable portion of these adverse events cannot be attributed solely to deficient technical skills 2 -4 . Adverse events related to non-technical skills and team performance are common and estimated to be twice as frequent as errors in surgical technique 5 . Poor teamwork and lack of vigilance appear to be essential factors influencing procedural flow and increasing error rates 6 .
Surgical teams demand specific infrastructure, resources and competencies to perform effectively and maintain patient safety 7 . Effective team performance depends on physical and social interactions, including back-up behaviour and leadership 8 . These demands, competencies and interactions encompassing effective team performance create a domain that is difficult to objectify and quantify 7,9 . In recent years, minimally invasive techniques have become the benchmark for a large number of abdominal surgical procedures 10 . These approaches introduce complex equipment, increased numbers of instrument changes and larger teams to the operative environment, resulting in increased demands in levels of coordination, anticipation, planning and communication 11 . The impact of the variation in procedural approaches on team demands, error rates and patient safety remains unclear. In highly complex abdominal procedures, associated with learning curves for surgical technique, minimally invasive approaches could also have a significant impact on team performance and non-technical skills 12 .
Recent studies have used a variety of methodologies to observe and objectify team performance. Consensus on the most efficient and methodically correct way of analysing and rating effective team performance is yet to be reached. The development of benchmarks for team observations and assessment of team performance will allow an accurate comparison of demands relative to surgical techniques. This will facilitate the development of effective, evidence-based training programmes for surgical teams, directed to increase team performance, decrease error rates and increase patient safety.
The aim of this systematic review was to assess the methodology used to objectify surgical team performance in minimally invasive abdominal surgery and explore team demands in relation to non-technical skills.

Methods
This study was performed according to the PRISMA guidelines 13 . Two researchers were involved in the search, inclusion, critical appraisal and data extraction of the articles selected for this study.

Eligibility criteria
Studies on assessment of surgical team performance and non-technical skills of the entire surgical team (including surgeons, anaesthesia and nursing staff) in the setting of minimally invasive abdominal surgery were included. Abdominal surgery was defined as any urological, gynaecological or general surgical procedure performed intra-abdominally. Minimally invasive techniques consisted of minimal-access approaches to the abdominal cavity, including laparoscopic, video-or robot-assisted methods.
Exclusion criteria consisted of non-original research, research performed in a simulated environment or non-human subject research, and language of publication other than English.

Study selection
Two authors performed a systematic literature search using PubMed, Embase, the Cochrane Library and Google Scholar to identify articles published before 11 October 2017. Search terms were based on subject ('teamwork', 'team learning', 'team efficiency', 'non-technical skills') and setting ('minimally invasive abdominal surgery', 'laparoscopic surgery', 'robotic surgery'). After the initial search, duplicates and non-English studies were removed. Articles were screened for eligibility by title, abstract and then full text. Reference lists and citations of the included studies were screened for missed articles. Discrepancies in study selection between the two authors were discussed with other review team members until consensus was reached.

Critical appraisal
The methodological quality of the included studies was assessed using the Oxford Centre for Evidence-Based Medicine levels of evidence, ranging from 1 (systematic review of RCTs) to 5 (expert opinion) 14 . Evidence of validity of the tools used to objectify and rate operative team performance was assessed using Messick's framework 15 , where a test should be rated for construct validity in each specific context in which the test is employed, defining five sources of evidence (content, process response, internal structure, relations to other variables, and consequences) each rated on a three-point scale 16 . Evidence of validity was classified as low (0-5), moderate (6-10) or high (11)(12)(13)(14)(15).

Data collection
The following data were extracted from the included studies: aim, design, setting, studied procedures, observational method, observer characteristics, outcomes, conclusion, and assessment of team performance and non-technical skills.

Results
The systematic search identified 2591 manuscripts, from which 69 duplicates and 193 non-English publications were removed. Based on screening of title and abstract a further 2198 articles were excluded. The remaining 138 full-text publications were reviewed, resulting in the selection of seven studies. Subsequent review of citations and references lists led to the inclusion of a further nine articles, so that a total of 16 studies were finally included in this systematic review (Fig. 1).

Study aims, designs and settings
All included studies consisted of single-centre observational series, with the exception of one dual-centre series 17 of four robot-assisted procedures. Two studies 18,19 investigated the influence of a non-technical skills training intervention on surgical team performance. Four studies 19 -22 consisted of subanalyses of results from observed cohorts published in previous work (Table S1, supporting information).
According to the Oxford Centre for Evidence-Based Medicine, all studies provided level 4 evidence (case series, poor-quality cohort studies, case-control studies). No subgroup analysis or meta-analysis of outcomes data was attempted due to heterogeneous methodologies and outcome measurements across the included studies. All studies observed complete surgical teams (including surgery, anaesthesia and nursing staff) with the goal of evaluating team performance through surgical workflow analysis and evaluation of disruptive events (9 studies) 20,23 -30 , the relationship between team performance and technical outcomes (3 studies) 18,21,22 , the relationship between anticipation of surgical steps and team efficiency (1 study) 31 or novel tools to rate team performance (3 studies) 17,19,32 .
Of the 16 studies, 11 focused exclusively on minimally invasive procedures, whereas five included both open and minimally invasive approaches to abdominal surgical procedures. Ten studies investigated laparoscopic techniques and six a robot-assisted approach. A total of 677 procedures (281 laparoscopic, 236 robot-assisted and 160 open) were observed across the included studies, with a mean of 42.3 operations per study (Table S1, supporting information).

Observational methodology
The majority of studies (12) observed team performance directly, and the four remaining studies performed a postoperative review of audiovisual recordings. Most (13) used multiple observers to evaluate team performance, with 14 reporting on methodological training of observers before the study and five including experts trained in human factor assessment or psychologists in their observing teams. Nine studies quantified interobserver reliability using a variety of methodologies; reliability was deemed good to excellent (Table S1, supporting information).
Most studies (14) observed team performance for the entire duration of the patient being present in the operating theatre. Six studies subdivided the procedure into preoperative, intraoperative and postoperative phases, of which four also defined a robot-docking phase.

Assessment of team performance
Seven studies used 'construct-based' team performance assessment tools that rated a number of behaviour constructs to create an overall score at the end of a case. Construct-based tools included: Oxford Non-Technical Skills (NOTECHS) 19 (4 studies) or Observational Teamwork Assessment for Surgery (OTAS) 33 (3 studies). These tools contained moderate evidence of validity for the assessed outcome according to Messick's framework, with a range of 8-10 of 15 (Table 1). Nine studies used an 'incident-based' team performance assessment methodology, classifying non-technical procedural errors or disruptions of surgical flow in order of causation 34 . The evidence of validity for these outcome tools was spread widely, ranging from low to high (4-12 of 15) ( Table 1).
No study used the same categories of surgical flow disruption. The most frequent flow disruption categories defined across the nine studies were related to equipment (8 studies), external factors (8), communication (6), supervision/training (5), environment (5) and procedure (5) (Tables 2 and 3).
Three studies included workload assessments in their methodology consisting of the National Aeronautics and Space Administration -Task Load Index (NASA-TLX) 35 (2) or the Surgery Task Load Index (SURG-TLX) 36 (1). These tools contained low to moderate evidence of validity (2-10 of 15) for the assessed outcome. Four studies also assessed technical performance using the Observational Clinical Human Reliability Assessment (OCHRA) 37 (3) and Operative Technical Errors (OTE) 18 (1) tools, which both have low to moderate evidence of validity (2-9 of 15).

Team performance relative to surgical approach
Three studies 26,28,29 compared team demands and/or performance in relation to a laparoscopic or open  18 Healey et al. 25 Catchpole et al. 24 Catchpole et al. 20 * Jain et al. 27 Zheng et al. 30 Allers et al. 23 Healey et al. 26 Weigl et al. 29 Total  approach to abdominal procedures, reporting different results. No study compared robot-assisted techniques with other approaches. Five studies investigated robot-assisted techniques and found that this approach to abdominal procedures increases team demands 24 that surgical teams were not always able to address effectively 20 , resulting in increased operating times 27 .
The identification and analysis of flow disruptions can provide an evidence base for improving the efficiency and safety of robot-assisted procedures 23,31 (Table S1, supporting information).

Discussion
The primary determinants of surgical outcome are generally perceived to be the patient's condition and the performance of the individual surgeon. Once corrected for patient risk factors, surgeons' technical skills are held accountable for variation in outcome. A number of different factors are important in achieving safe and effective surgical care, including infrastructure, equipment and surgical team performance 38 . A minimally invasive surgical procedure is conducted in a sophisticated environment combining patient factors, complex equipment and a large number of individuals set to do independent and team-based tasks. This systematic review included 16 studies objectifying and rating surgical team performance during minimally invasive abdominal procedures. The studies were of low methodological quality, heterogeneous design, and utilized a number of different tools to objectify team performance.
In four studies, data were obtained via audiovisual recordings of the surgical environment. Despite apparent benefits of reviewing audiovisual recordings, the majority of studies collected their data through direct observation in the operating theatre. Benefits of data obtained through audiovisual recording include that data can be assessed by multiple, independent observers and incidents can be reviewed multiple times, increasing the validity and reliability of findings 39 . In addition, during direct observation the focus of the observer may decrease due to fatigue, potentially resulting in failure to record important events 40 . It is also possible that during direct observation findings are affected by the Hawthorne effect (change of behaviour in response to the awareness of being observed) 41,42 . A prerequisite for accurate evaluation through review of audiovisual recordings is the quality of the recordings. Ethical considerations and potential hazards to team privacy and liability issues in case of adverse events may also be a limitation of its use 43 .
Another source of variation across the reviewed studies was the number and type of observers collecting observational data. Interobserver reliability should be quantified to guarantee the quality of observations and objectivity of rating tools used 44 .
This systematic review has demonstrated that, in the current literature, construct-and incident-based team performance objectification tools are used most commonly. Construct-based tools, including the OTAS and NOTECHS, rate a number of behavioural constructs on set Likert scales. These tools were developed for conventional approaches to surgery, providing global ratings for set constructs, and need to be validated for the identification of non-technical skills in minimally invasive surgery. Although used by a number of studies with similar aims, studies used different categories of flow disruption, with a broad range of validity. Some variation in validity can be related to the nature of Messick's framework.
According to Reason's organizational accident model, an adverse event is preceded by a chain of individually unimportant errors and/or latent threats that in sequence lead to an adverse event or breach of patient safety 45 . Incident-based team performance objectification methodology can provide valuable insight into these patterns and the interplay of complex minimally invasive surgical equipment.
None of the included studies was able to relate team performance to patient outcomes. This could be caused by insufficient power of individual studies. However, team performance as a determinant of morbidity and mortality is heavily biased by patient factors and technical performance. Surrogate markers for team performance could include operating time, intraoperative adverse events, and the number and duration of procedure flow disruptions. Larger, well designed studies are needed to display the true influence of minimally invasive techniques on team performance.
The major limitation of this review was the number and quality of available studies, providing insufficient data for a subgroup analysis or meta-analysis of outcomes. The majority of studies examined a heterogeneous group of operations, providing limited validity for the identification of unique non-technical skills related to specific procedures. Future studies should therefore analyse multiple approaches (open, laparoscopic, robot-assisted) in relation to a single procedure, use multiple trained observers to collect data, preferably from audiovisual recordings of the surgical environment, quantify interobserver reliability, objectify team performance using incident-based methodology with a predefined outcome set including causation and consequences of procedural flow disruptions, and analyse team performance in relation to direct (operating time, intraoperative adverse events) and indirect (patient morbidity and mortality) performance metrics. Such well designed studies are needed to gain insight into team performance demands unique to minimally invasive surgery in order to develop structured, evidence-based training programmes that enhance patient safety and procedural flow.

Disclosure
The authors declare no conflict of interest.