Technical and non-technical skills can be reliably assessed during paramedic simulation training

BACKGROUND: Medical teams depend on technical skills (TS) as well as non-technical skills (NTS) for successful management of critical events. Simulated scenarios are an opportunity for presentation of similar crisis situations. The aim of this study was to test whether TS and NTS are assessable with satisfactory interrater reliability (IRR) during a regular paramedic training. METHODS: Thirty paramedics were rated by two independent observers using video-recording and previously validated checklists while managing two simulated emergency scenarios as a team of two. The observed items of the team's TS included type, order, and time of adequate medical care. The NTS were restricted to six team-oriented dimensions. The IRR was quantified by calculating the intraclass correlation coefficient (ICC). The z-transformed values of the TS and NTS were correlated by Pearson's correlation. Internal consistency was controlled using Cronbach's alpha. RESULTS: The average measures ICC for the IRR was between 0.97 [95% confidence interval (CI) 0.91-0.99] and 0.98 (95% CI 0.94-0.99) for the TS sum-score, and was 0.94 (95% CI 0.87-0.97) for the NTS sum-score; the Cronbach's alpha of this NTS sum-score was 0.86. There is a positive correlation between the normalised TS and NTS sum-scores (r=0.53; P>0.05). CONCLUSION: Assessment of TS and NTS is feasible and reliable during paramedic training in emergency scenarios. TS can be reliably assessed by one trained observer; for NTS, two trained raters provide a suitable condition for excellent observations. There is a significant positive correlation between TS and NTS.