Figure - uploaded by Stella Polikarpus
Content may be subject to copyright.
Source publication
Situational awareness (SA), the ability to perceive, comprehend and predict situation around you and it is a key in attending any incident as critical foundation for successful decision-making. Because incidents are solitary events, development and assessment of SA presents a significant challenge. In this article we analyze the authoring process o...
Contexts in source publication
Context 1
... scenarios were built using CAPM and used more than ten times for assessments. Table 4 presents the average scores (means) and standard deviations (SD) of ten assessment scenario SA levels scores. The average of SA level 1 score is 66.70 and SD is 7.20 for these ten scenarios. ...Context 2
... level 2 is 65.85 (SD 7.89) and SA level 3 is 63.68 with highest SD 8.51. In all ten cases SA levels perception and prediction score means differ from each other, meaning that perception is better than prediction (see Table 4). In table 4 is seen that four cases (TR218; JM218; RT118 and TM118) SA level 1 score is below average, three cases (TR218; JM218; RT118) SA level 2 is below average and four cases is SA level 3 below average (TR218; JM218; RT118 and AS118). ...Context 3
... studied the inconsistency in SA scores relating to scenarios (see Table 4 blue coloured cells). In two scenarios (JM218 and TM218) the average SA level 2 comprehension was higher than level 3 prediction (see Table 4 column Diff SA2, SA3). ...Context 4
... studied the inconsistency in SA scores relating to scenarios (see Table 4 blue coloured cells). In two scenarios (JM218 and TM218) the average SA level 2 comprehension was higher than level 3 prediction (see Table 4 column Diff SA2, SA3). The storyline (JM 218) might be difficult for ICs to understand the development of the fire inside the building because the person, who later shot himself, first purposefully set his house into fire. ...Context 5
... in electrical substation requires a specific response and that should be practiced. In three cases (RT118, TM118 and AT119) SA level 2 comprehension was higher than SA level 1 perception (see table 4 column Diff SA1, SA2). Truck loader collision, fire (TM118, used 55 times) and gas explosion in building under construction (RT118, used 46 times) should not be used any more as they are overused (see Table 3 above). ...Context 6
... found that the CAPM is feasible because of twelve scenarios co-authored and used. It helps to implement authentic virtual scenarios based SA assessment for assessments (SA level 1 mean 66.70; SA 2 mean 65.85 SA 3 is 63.68, see also Table 4). The virtual scenarios created with CAPM are used 341 times for formal assessments (see Table 2) and new scenarios ready to be used indicate successful implementation of CAPM in the organization. ...Citations
... So far it is unrealistic to measure their situational awareness in real time when they need to act in fast developing high risk situations like emergencies are. Despite the difficulties of training and assessing rescue incident commanders situation awareness is hard task for a trainers [1], it seems to be possible to do with help of virtual simulations using virtual reality technology [2]. Virtual simulations could be co-authored by trainers themselves using the Collaborative Authoring Process Model (CAPM), that gives step-by-step guide on how to do it in a training organization context [1]. ...
... Despite the difficulties of training and assessing rescue incident commanders situation awareness is hard task for a trainers [1], it seems to be possible to do with help of virtual simulations using virtual reality technology [2]. Virtual simulations could be co-authored by trainers themselves using the Collaborative Authoring Process Model (CAPM), that gives step-by-step guide on how to do it in a training organization context [1]. In The Estonian Academy of Security Sciences the Effective Command behavioural marker framework is used to assess rescue incident commanders situational awareness [3]. ...
... Dynamic decision-making could be defined "as making a series of interdependent decisions in an environment that changes over time due to the consequences of the taken decisions or due to autonomous changes in the environment" [10, p. 284]. The research on how to create these virtual simulations for rescue incident commanders' dynamic decision-making training is rather episodic and only a few articles address this issue [1,11]. ...
Training and measurement of situation awareness (SA) in a dynamic decision-making context is a complex task and depends on personal factors as well as time, space, and situation. Therefore, to measure first-level incident commanders’ SA several authentic and immersive virtual simulations of rescue incidents should be used. In this paper, we report on Effective Command and the dynamic decision-making assessment methodology as a theoretical framework for SA assessment and implementation of virtual reality software using sixteen different virtual simulations in an organization. First, we report four aspects needed to be implemented for virtual simulation-based assessment of SA. Next, we analysed the differences between three SA ascending levels: perception, comprehension, and prediction that might be influenced by the scenario storyline. According to prior research, we would expect the difficulty of the three SA levels to be in ascending order and we evaluated that by analysing N = 665 assessment results. We confirmed such an ascending order for most scenarios, and we also identified these scenarios for which this assumption was not met. Finally, we discuss the possibilities of analysing the scenario-based SA level differences in the future when assessing SA with more automated measures. We argue that by improving the assessment of SA, we can foster the training possibilities for incident commanders’ dynamic decision-making.KeywordsSituation Awareness (SA)Effective Command (EC)Virtual simulationsThe Collaborative Authoring Process Model (CAPM)
... For example, a trainee might encounter a fire scene and needs to decide on a course of action. The actions of he/she in the VS have an influence on the course of the VS scenario, e.g., actions taken (firehose versus fire extinguisher) contribute to stopping the fire (Polikarpus & Ley, 2021). In VET, VS can be used to train dynamic decisionmaking competencies (Reis & Neves, 2020) or to gain awareness of multi-professional collaboration (Prasolova-Forland et al., 2017). ...
... For VSBT to be implemented in an organisation, VS has to be created (Polikarpus & Ley, 2021) or existing ones used (Bourdeau et al., 2021). In both cases, skilled trainers play a key role in effectively implementing VS in training organisations (Keskitalo, 2022). ...
... We conducted our study in EASS to measure the working trainers' personal TPACK. Because earlier studies in the same organisational context show that trainers can participate in co-designing VS in EASS (e.g. by using The Collaborative Authoring Process Model for Virtual Simulation Scenarios (Polikarpus & Ley, 2021)), we assume that this might lead to the development of collective TPACK (Yeh et al., 2021) in EASS. Trainers co-authoring of VS using different theoretical approaches to train situational awareness (Polikarpus & Ley, 2021;Polikarpus et al., 2022) has led to the implementation process of VSBT in EASS (Polikarpus & Danilas, 2021). ...
Virtual simulations (VS) are gaining more popularity in vocational education and training (VET) to train professional competencies. However, implementing VS into organisations requires the involvement of trainers in, and the development of their knowledge to fulfil a meaningful approach to integrating this technology into teaching. In the Estonian Academy of Security Sciences (EASS) VS have been used for more than a decade to train and assess rescue and police staff work and related competencies such as decision-making and problem-solving. As there is a lack of research as to the role of trainers in the implementation process of virtual simulation-based training (VSBT) in organisations, a survey among EASS trainers (N = 146) was used to reveal their role in the implementation, their attitudes towards the use of VS and their Technological Pedagogical Content Knowledge (TPACK). While attitudes grow more positive with the use of VS and especially with training others to use it, integrated knowledge about using technology in training seems to depend on the active participation of trainers in its creation. We discuss the important role of involving trainers in the implementation of VS-based training to ensure the successful use of VS in vocational education.
... For that incentive, we created a second scenario (we call "national scenario"-NS) following the Collaborative Authoring Process Model (CAPM), a methodology for involving trainers into the creation of VS scenarios (Polikarpus & Ley, 2021). CAPM represents an efficient and tightly-specified method for generating VS scenarios. ...
... To be able to measure ASA/U, the incident commander needs to be presented with a realistic and immersive scenario with which they can feel they are involved . The CAPM was developed to be used to assess incident commanders' dynamic decision-making skills by means of VS and it is practically applicable in fire service training organizations as it involves trainers in the building process (Polikarpus & Ley, 2021). Table 1 presents the main steps of CAPM that were used to build the NS. ...
... The NS was fully developed in XVR On-Scene (2020 version) software for incident commanders' assessment using the CAPM (Polikarpus & Ley, 2021) as follows: ...
Measuring situation awareness (SA) and situation understanding (SU) is an important topic for Command & Control research. Virtual simulations (VS) have been proposed as a suitable method for measuring SA/U, but, there is little research into how to build scenarios for VS so that SA/U can be measured reliably. In this study, we used two different VS scenarios and the Quantitative Analysis of Situation Awareness (QASA) method to measure, and provide feedback on, actual and perceived SA/U. Two VS scenarios were tested in Estonia with 36 trainees. The results of the different scenarios were compared to establish whether the scenario storylines and authoring process resulted in differences in SA or SU. We conclude that reliable assessment results were produced with both authoring processes, and further suggest that the Collaborative Authoring Process Model for Virtual Simulations (CAPM) be used for VS creation.
