Advancing How The World Learns

featured project

Shiphandling Skills Automated Assessment

In collaboration with the Surface Warfare Officers School (SWOS), in Newport, Rhode Island, CRESST is developing an automated assessment engine (AAE) capable of evaluating shiphandling skills as part of a larger Navy training initiative known as NSTAT (Navigation and Shiphandling Training and Assessment Tools).

Despite its critical importance to the Navy, shiphandling training is becoming increasingly more challenging as Surface Warfare Officers are spending less time at sea, and consequently experiencing fewer opportunities to conn a ship under the apprenticeship of a more experienced Master Mariner. This can lead to less confidence and diminished competence in shiphandling among Surface Warfare Officers, which in turn increases the Navy’s risk for accidents (collisions, allisions, groundings, etc.) when carrying out mission-critical tasks.

To address these concerns, the Navy has invested in the Conning Officers Virtual Environment (COVE) and the associated intelligent tutoring system that can be paired with it, COVE-ITS. Together, these synthetic training systems provide opportunities for students to practice shiphandling tasks with spoken coaching and feedback, based on their actions. However, in its current form, the COVE-ITS is incapable of providing deeper, more aggregated assessments of student performance, such as those that could (1) identify skill areas that require remediation; and (2) identify root causes for misconceptions or gaps in knowledge as they contribute to diminished skills.

The CRESST automated assessment engine serves to complement and enhance the spoken coaching and feedback capabilities of the COVE-ITS. This software module is designed to make inferences of student shiphandling skills based on real-time, observed performance of students working within the COVE. The development of the AAE is being done in collaboration with the developers of the COVE-ITS, Dr. Stanley Peters and his team at Stanford University. Overall, the AAE will enable instructors and students to identify strengths and weaknesses within the shiphandling domain more accurately and efficiently, and guide remediation (when necessary) to focus on root cause misconceptions and/or gaps in knowledge. The AAE addresses the shiphandling topics of (1) Mooring at a pier (docking); (2) Getting underway from a pier; (3) Underway Replenishment (UNREP); and (4) Harbor Transit.

featured project

Anesthesiology Skills Automated Assessment

CRESST will work with a team from the UCLA Medical School and individuals from the UCLA Graduate School of Education and Information Studies to develop and research a medical simulation to assess teamwork skills. The simulation will be accessible online (via secure server) or offline, permit repetitive practice, be engaging (game-like), and provide feedback in the form of summative assessment after each scenario module. The skills to be addressed within the simulation are relevant to general aspects of healthcare and specifically to military healthcare, where deployments often disrupt designated teams and heighten the need to form cohesive units quickly. Individual practice in synthetic training environments such as this one can supplement face-to-face team training.

Our project objective is to research learning and performance differences between two primary modes to simulate team training: (1) an Evaluator Mode in which students interact with the simulation system by providing annotations on observed team interactions without affecting the outcome of the event; and (2) a Game-Play Mode in which students play the role of a team member (leader or follower) and contribute to the outcome of the simulated scenario. Both groups will receive identical instruction on one or more specific team training skills prior to each scenario, and both will receive summative feedback and scoring after each scenario. A primary outcome of this research is focused on the training effectiveness differences between the two modes (evaluator vs. game-play). The reason for this comparison of training modes is to examine the development of synthetic team training from a cost-benefit perspective. This includes examining the development effort vs. learning and performance expectations. Our contentions are (1) that optimal team training relies on immersive participation which our game-play mode will emulate; and (2) that for team training, development of an evaluator-style simulation is less costly (regarding development time, programming expertise needed, resources required, etc.) compared to a game-play style simulator.