“Better Performance and Cost Data Needed to More Fully Assess Simulation-Based Efforts”

Last week the U.S. Government Accountability Office (GAO) released their Report to Congressional Committees titled, “Army and Marine Corps Training: Better Performance and Cost Data Needed to More Fully Assess Simulation-Based Efforts“. The report is in response to the increased use of computer-based simulators to train service members operating today’s modern warfare technology, and is the third report they have produced to assess the services’ mix of live and simulation-based training.

GAO examples of current Army and Marine Corps simulators and simulations.

While there’s no doubt that simulators provides excellent learning environments that are cost-effective, the GAO recommends that the military services need to develop more strict metrics and methodologies in order to compare simulation-based, and live training, head to head. The report focuses on the efforts that will be required in a fiscally constrained environment in order to meet training requirements. While the assumption is that training simulators are less expensive than live training, the GAO report points out that without properly measured metrics it is impossible to show precisely how much more cost-effective they are.

Department of Defense Comments

The U.S. Department of Defense (DOD) only partially concurred with the recommendations of the report, agreeing that performance metrics would be useful when assessing the impact of simulation-based training, however they stated in their evaluation of the report that:

“Given the magnitude and scope of training tasks, varying competencies of the training audience, and ever-changing technology, the problem set contains many independent variables.”

The DOD said they would study the problem set and develop a plan to include performance metrics to assess the impact of simulation-based training. They also said that they currently capture all of the relevant cost-based metrics required for decision making, however they agreed that having a more comprehensive collection of data would be helpful. The report concludes that without better data, the services lack the information they need to make informed simulation-based training investment decisions.